Everything in Compressed Sensing is basically reducible to these slogans.
1. Slogan: undersampling is just like adding noise.
if you randomly undersample
and then do traditional linear reconstruction, it's just like complete acquisition followed
by adding gaussian noise.
2. Slogan: sparsity allows noise suppression by thresholding. If the reconstruction is done
in a domain where the signal is sparse, think wavelets or some other multiscale repn, then
thresholding will kill the noise but not the signal.
3. Slogan: iteration allows to correct any errors at earlier stages. If after thresholding, you
take the approximate reconstruction so far and then subtract it away from the data you
collected, and then
repeat the process,
you will find that the noise level is lower and you can find more
signal than before,
and you can subtract that and repeat.
The key point is that if you generate a nice incoherent matrix eg using randomness, and
then follow these slogans, you get just as far as you can get using more abstract
approaches. (This point of view is seen in the papers I sent you)
But it opens lots of great connections.
1. In the theory, we often use true random sensing matrices because they are so
convenient to analyze. In many applications we can't actually generate really random
sensing matrices. Finding matrices which are convenient to use but still allow the above
slogans to work is the essence of solving real problems.
In MRI implementing a random sampling of Fourier space would actually be far slower
than a traditional cartesian scan. However, in the papers I sent you, a real-world speedup
of factor 7 was achieved by constructing scans that traverse fourier-space along smooth
randomly perturbed spirals (in some cases) and along collections of randomly-oriented
line segments (in other cases)
I have been told by Felix Herrmann of UBC Vancouver and
his students that they have been able to radically undersample in geophysical data
collection by using randomly-scattered sampling and a StOMP-like algorithm for
reconstruction. That is, instead of traditional cartesian sampling in seismic data
collection, they undersample at randomly jittered sites, and they reconstruct in a waveletbasis using interference cancellation. This is very much like the MRI story I told you
about -- abandon Cartesian sampling for something much thinner and more exotic.
What is needed is a research program that will achieve
a very good understanding of non-random matrices
that can be explicitly constructed in applications
and yield good properties for CS. They needn't be theoretically analyzable, but for
example, they may be heuristically constructed. However, very good metrics for the
quality of such heuristic schemes need to be invented and used so there is a systematic
design criterion. Here is an example: if an image is sparsely represented in the 2d
Wavelet domain, what families of curves in Fourier space will lead to good recovery
from such samplings? How can we predict when this is true? How can we document this
using understood metrics etc.
2. The theory so far has mostly emphasized either geometric banach space techniques or
else heavy-hitter arguments in theoretical Comp Sci. Many of the contributors are not
really aware of electrical engineering, signal processing, and etc. However, in truth,
Compressed Sensing can be very closely connected with existing communities in EE/Info
Thry. Putsuing the connection farther will bring in a lot of smart people who really can
impact the navy because they know about signals and systems. It will also make the
banach spacers smarter as well as the Theoretical CS people.
The ideas of compressed sensing can be linked to
things going on in wireless communication -- mutual access interference cancellation and
wireless communication -- iterative decoding algorithms. Many of the same tools are
relevant.
In each case we are trying to solve a system of linear equations with noise. In each case
the underdetermined system is in some sense random or pseudo random. Many of the
same computational and mathematical techniques are relevant.
A significant underlying difference is that in Compressed Sensing we have sparsity as a
driving force, i.e. we know that the solution we are looking for is sparse. Sparsity ought
to be useful also in interference cancellation. Thomas Strohmer of UC Davis tells me he
was inspired by the StOMP paper to develop interative cancellation tools in wireless
based on the idea that interference will be sparse -- concentrated among relatively small
numbers of potential interferers.
It would be interesting for ONR to promote study of the
use of sparsity in solving random underdetermined systems of equations in other signal
processing contexts. Specifically, develop practical
fast algorithms for
sparse multiple access interference cancellation.
3. At the same time, it is well understood that there are phenomena of phase transition
where the slogans 1-3 above do not work anymore. Once the sparsity is lacking or the
noise is too high, the slogans all fail. Careful
probing of this effect
will result in new kinds of fundamental sampling theorem, that replace the classical
shannon theorems in this new setting. This requires probing the connections with wireless
communication -- iterative decoding algorithms , where ideas from statistical physics
have been used to precisely identify phase transitions in the success
of such algorithms.
It would be interesting for ONR to promote the fundamental theory of precisely
understanding phase transitions in reconstruction of sparse sequences from random
underdetermined systems by various algorithms. This could build a bridge to statistical
physics to information theory and also to theoretical CS who work on
combinatorial optimization.
I have been working from this point of view for example with Jared Tanner of Utah and
with Iain Johnstone of Stanford, and have to say that there are many things going on with
non-sparse spin glass models, that if reformulated in the sparsity context, could be very
valuable in such an effort. There may be very experienced probabilists and spin glass
researchers who are available to attack such problems, if not people like me and Jared
and Iain would have to do.
4. The original CS ideas were based on l1 minimization, but now
there is a flood
of other algorithmic ideas. Setting up good frameworks for
performance comparison
both in reconstruction effort and in reconstruction accuracy, and carefully creating
baselines and carefully comparting algorithms could lead to
a very good
set of options for practical use and for further algorithmic development.