mlsplogo MLSP2013
IEEE International Workshop on
Machine Learning for Signal Processing

September 22-25, 2013  Southampton, United Kingdom

Sparsity-Aware Adaptive Learning: A Set Theoretic Estimation Approach
Sergios Theodoridis Sergios Theodoridis
Department of Informatics & Telecommunications, University of Athens, Greece
Personal homepage


Sergios Theodoridis is currently Professor of Signal Processing and Communications in the Department of Informatics and Telecommunications at the National and Kapodistrian University of Athens. His research interests lie in the areas of Adaptive Algorithms and Communications, Machine Learning and Pattern Recognition. He is the co-editor of the book “Efficient Algorithms for Signal Processing and System Identification”, Prentice Hall 1993, the co-author of the bestselling book “Pattern Recognition”, Academic Press, 4th Ed. 2009, co-author of the book “Introduction to Pattern Recognition: A MATLAB approach”, Academic Press, 2010, and the coauthor of three books in Greek, two of them for the Greek Open University. He is the co-author of six papers that have received best paper awards, including the IEEE Computational Intelligence Society Transactions on Neural Networks Outstanding Paper Award. He has also served as Distinguished Lecturer of the IEEE Signal Processing Society. He has served as President of the European Association for Signal Processing (EURASIP), as a member of the Board of Governors for the IEEE Circuits and Systems (CAS) Society and he currently serves as a member of the Board of Governors of the IEEE Signal Processing Society. He was the general chairman of EUSIPCO-98, the Technical Programme co-chairman of ISCAS- 2006 and 2013 and the co-chairman and co-founder of CIP-2008. He has served as an Associate Editor in all major Signal Processing related journals, including IEEE Transactions on Signal Processing, IEEE Signal Processing Magazine, IEEE Transactions on Neural Networks, IEEE Transactions on Circuits and Systems, Signal Processing. He is currently the Editor-in-Chief of the EURASIP Signal Processing book series of Academic Press, and Editor-in-Chief, with Rama Chellappa, of the Signal Processing E-Reference project, Elsevier.

He was a member of the Greek National Council for Research and Technology and Chairman of the SP advisory committee for the Edinburgh Research Partnership (ERP). He has served as vice chairman of the Greek Pedagogical Institute and he was for four years member of the Board of Directors of COSMOTE (the Greek mobile phone operating company). He is Fellow of IET, a Corresponding Fellow of RSE a Fellow of IEEE and a Fellow of EURASIP.


Learning sparse models has been a topic at the forefront of research for the last ten years or so. Considerable effort has been invested in developing efficient schemes for the recovery of sparse signal/parameter vectors. However, most of these efforts have focused on batch processing, via the compressed sensing or sampling (CS) framework. It is only very recently that online/time-adaptive algorithms have been developed, where the training data are processed sequentially, and the sparse signal/system to be recovered has the freedom to be time-varying. Besides time variation, online schemes are becoming very popular in the context of Big Data applications, where processing as well as memory requirements for batch processing may become excessively large. Our stage of discussion will be that of set theoretic estimation. Instead of a single optimal point, we are searching for a set of solutions that are in agreement with the available information, which comprises a set of training points and a set of constraints. Each training point is associated with a convex set, built around concepts borrowed by the robust statistics loss functions family.

In its more “standard” formulation, the sparsity constraint is imposed via convex sets such as l_1 or weighted l_1 balls. The solution is searched in the intersection of all the previously mentioned sets, via the use of a sequence of projections on the respective convex sets. Both of the previous constraint sets are equivalent to a soft thresholding operation rule. Moreover, the interesting characteristic of the weighted l_1 ball approach is that it corresponds to an optimizing task with time varying constraints. Convergence proofs are established via the rich fixed point theory. The respective algorithms are of linear complexity with respect to the number of unknowns.

Beyond convex constraints, more recent methods are also discussed, where sparsity is imposed via mappings; these are inspired by generalized thresholding rules, which are associated with non-convex penalty functions. To this end, the existing theory had to be extended with a new concept, that of partially quasi-non-expansive mappings, whose fixed point is a union of subspaces; an object that lies at the heart of sparse modellearning. This new family of sparsity promoting algorithms scales, also, linearly with the number of unknowns.

The presentation will be based on geometric arguments by-passing a mathematical formulation path. Comparative results will be discussed with respect to other sparsity promoting algorithms, which build upon arguments based on regularization in the context of the LMS and RLS-type schemes. Moreover, versions of our novel algorithms for distributed learning will be presented and their relative merits will be discussed.

Join work with Prof. K. Slavakis, Dr Y. Kopsinis, Dr S. Chouvardas

Challenges in Source Separation
Christian Jutten Christian Jutten
GIPA-lab, University Joseph Fourier of Grenoble, France
Personal homepage
Download lecture slides


Prof. Christian Jutten is full professor in University Joseph Fourier of Grenoble. He has been deputy director of the Grenoble images, speech, signal and control laboratory (GIPSA, 300 people) and director of the Department Images-Signal (DIS, 100 people) from 2007 to 2010. For 30 years, his research interests are blind source separation, independent component analysis and learning in neural networks, including theoretical aspects (separability, source separation in nonlinear mixtures, sparsity) and applications in signal processing (biomedical, seismic, hyperspectral imaging, speech).

He is co-author of more than 70 papers in international journals, 4 books, 24 invited plenary talks and 150 communications in international conferences. He is currently deputy director of Institute for Information Sciences and Technologies of CNRS. For his contributions in source separation and independent component analysis, he received the Medal Blondel (1997) from the French Electrical Engineering society, was elevated as a Fellow IEEE and as a senior Member of Institut Universitaire de France in 2008


Separation and extraction of sources are wide concepts in information sciences, since sensors provide signal mixing and an essential step consists in separating/extracting useful information from unuseful one, the noise. Althought intensively investigated during the two last decades, in this talk, I present a few challenges that have still to be addressed.

In many areas like brain imaging, hyperspectral imaging, due to various kinds of sensors, there are many ways for recording the same physical phenomenon leading to sets of multimodal data. Multimodality has been studied in human-computer interface or in data fusion, but never at the signal level. A first challenge is to provide a general framework of multimodal source signals.

There exist a few cases where the mixtures are essentially nonlinear, e.g. with chemical sensors. However, up to now, most of the source separation/extraction results and methods are restricted to linear mixtures. A second challenge is to enlarge theoretical results on identifiability and algorithms in nonlinear source separation, especially for new classes of nonlinearities (e.g. multilinear) and priors on sources.

In high-dimension data (e.g. EEG or MRI in brain imaging), separating all the sources is neither tractable nor relevant, and one would like to only extract the useful sources. Conversely, for a small number of sensors, especially smaller than the number of sources, it is again necessary to only focus on the useful signals. A third challenge is to develop generic framework for only extracting useful signals, based on coarse reference signals or priors.

Sequential Inference for Dynamically Evolving Points, Groups and Clusters
Simon Godsill Simon Godsill
SigProC Laboratory, University of Cambridge, United Kingdom
Personal homepage
Download lecture slides


Simon Godsill is Professor of Statistical Signal Processing in the Engineering Department at Cambridge University and a Professorial Fellow and tutor at Corpus Christi College Cambridge. He coordinates an active research group in Signal Inference and its Applications within the Signal Processing and Communications Laboratory at Cambridge, specializing in Bayesian computational methodology, multiple object tracking, audio and music processing, and financial time series modeling. A particular methodological theme over recent years has been the development of techniques for optimal Bayesian filtering and smoothing, using Sequential Monte Carlo or Particle Filtering methods. Prof. Godsill was technical chair of the IEEE NSSPW workshop in 2006 on sequential and nonlinear filtering methods, was Technical Chair for Fusion 2010 in Edinburgh,and has been on the conference panel for numerous other conferences/workshops. Prof. Godsill has served as Associate Editor for IEEE Tr. Signal Processing and the journal Bayesian Analysis. He was Theme Leader in Tracking and Reasoning over Time for the UK's Data and Information Fusion Defence Technology Centre (DIF-DTC) and Principal Investigator on grants funded by the EU, EPSRC, QinetiQ, General Dynamics, MOD, Microsoft UK, Citibank, Google and Mastercard. In 2009-10 he was co-organiser of an 18 month research program in Sequential Monte Carlo Methods at the SAMSI Institute in North Carolina, and will co-organise an Isaac Newton Institute Programme on Sequential Monte Carlo in 2014. He is a Director of CEDAR Audio Ltd. (which has received a technical Oscar for its audio processing work) a company which has utilised the research results from his laboratory.


In this talk I will describe recent advances in methods and applications for sequential inference about dynamically evolving structures such as stochastically linked groups, networks and multiple interacting objects. The problems are posed within a Bayesian framework and inference is carried out using modern computational methods such as sequential Monte Carlo, Markov chain Monte Carlo and combinations of the two. Examples will be discussed in which we learn the dynamic structures of small groups from noisy tracking data, learn automatically the linkage forces between members of a group, and learn about the environment using Gaussian process models of a potential field representation. I will also discuss methods for tracking large numbers of objects in dymamic clusters.  
Powered by CONWIZ, © Copyright 2013. Page maintained by Jan Larsen (Email)