From Gene to Meme
Example of Memes:
-Cultural ideas, symbols
-Religions
-Languages
-Mathematics (e.g.,
Gaussian distribution)
-Scientific understanding (e.g.,
Quantum mechanics)
-Technological achievements (e.g.,
Atomic force microscopy)
-Engineering designs (e.g., ipod,
Wii, iphone)
2016/3/23
Xin Li LDCSEE WVU Spring 2009
An Objective Measurement of
Meme’s Fitness
Academia
Industry
2016/3/23
Xin Li LDCSEE WVU Spring 2009
What is in Common?
Steve Jobs
2016/3/23
Herbert Simon
Xin Li LDCSEE WVU Spring 2009
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Theme of This Talk
“Life is about connecting dots”
– in “Staying Hungry, Staying Foolish” Stanford
Commencement Address by Steve Jobs in
2005
Scientific research is also about connecting
dots
– Search is an important component part of
scientific research (where are the dots? how
are they related?)
– Re-search often reveals hidden relationship
among isolated dots that is not known before
Xin Li LDCSEE WVU Spring 2009
Image Processing as the Showcase
Science
Technology
Statistical
physics
MRI/PET
STM/AFM
Chemical
oscillation
Cognitive
Science
PRAM/Microarray
Image
processing
Mathematics
Engineering
analysis
Networking
geometry
Control
algebra
Communication
2016/3/23
statistics
Xin Li LDCSEE WVU Spring 2009
Image Processing: at the Intersection
of Science, Technology, Engineering
and Mathematics (STEM)
our starting
point
+
2016/3/23
Xin Li LDCSEE WVU Spring 2009
So how should you choose your
technical field?
Outside environment plays some role
– Emerging areas tend to attract more
resources than traditional fields
– Every dept. has its focused areas
Learn yourself better
– Good at theory or experiment/application?
– Good at algebraic or geometric thinking?
– Good at depth-first or width-first reasoning?
Find a good match
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Taking Myself as an Example
Entered Princeton ISS group in 1996
Very little research experience in my
undergraduate study (BS thesis is on speech
coding)
Information theory or signal processing?
– Princeton EE is really strong in theory (not to mention
Math and Physics)
– Majority of ISS students will take the theory path even
by doing TAs due to limited research funding in the
area of information theory
– Graduate students in Princeton EE from India are
also really good at theory
Xin Li LDCSEE WVU Spring 2009
Information Theory vs. Image
Processing
IEEE TIP
– H-index=148
– 1992-present
– Most influential papers:
image watermarking,
image coding, image
segmentation
IEEE TIT
– H-index=214
– 1963-present
– Most influential papers:
cryptography, space-time
codes, error-correcting
codes, wavelets, ...
Xin Li LDCSEE WVU Spring 2009
Highly-cited Papers related to
Image Processing
Markov Random Field (Geman and
Geman 1984) >8000 citations
Wavelet theory (Daubechies, Mallat,
Vetterli …) >180,000 citations
Why do they last?
“the most fruitful areas for the growth of sciences
were those which had been neglected as a no-man’s
land between the various established fields.”
–Norbert Wiener
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Connection 1: MRF
Pixels vs. Particles
Pixel value = 0 or 1
2016/3/23
Spin direction = up or down
Xin Li LDCSEE WVU Spring 2009
A Little Bit History of Ising
Model
Proposed by Ising in his PhD thesis in 1925
2D Ising model was analytically solved by L.
Onsager in 1944 (who won the Nobel Prize
in 1968)
Phase transition behavior investigated by
Yang and Lee in 1950s
related to renormalization theory pioneered
by RG Wilson (who won the Nobel Prize in
1982)
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Apply Ising Model to Images
Applied to image restoration by Geman
and Geman in 1984 (bring statistical
mechanics to engineering)
Stirred up lots of interest
– More powerful image models (line process,
higher-order MRF)
– More efficient optimization algorithms (Gibbs
sampling, Swendsen-Wang, Wolff algorithm)
– New applications
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Image Example
original
Monte-Carlo
Optimization
(minimize E)
noisy
2016/3/23
restored
Xin Li LDCSEE WVU Spring 2009
Connection to Hopfield Networks
Why is this model so influential?
The first-order approximation of
associative memory in brain theory
Prof. Hopfield gave a talk at WVU on
Mar. 13, 2007 titled “How Do We Think So
Fast? From Neurons to Brain Computations,”
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Statistical Mechanics and IT
Shannon was the first to recognize the connection
between statistical mechanics and communication theory
Connection with statistical mechanics also exists for
Turbo codes (belief propagation is related to Bethe free
energy)
“Multiuser detection and statistical mechanics” (Guo and
Verdu’ 2003)
“Evolution and structure of the Internet: A statistical
physics approach” (R Pastor-Satorras and A Vespignani‘
2004)
“Statistical mechanics of complex networks” (R. Albert’
PhD thesis in 2001)
2016/3/23
Xin Li LDCSEE WVU Spring 2009
If you think you have understood
entropy
“My greatest concern was what to call it. I thought of calling it
‘information’, but the word was overly used, so I decided to call it
‘uncertainty’. When I discussed it with John von Neumann, he had
a better idea. Von Neumann told me, ‘You should call it entropy,
for two reasons. In the first place your uncertainty function has
been used in statistical mechanics under that name, so it already
has a name. In the second place, and more important, nobody
knows what entropy really is, so in a debate you will always have
the advantage. ”
-Conversation between Claude Shannon and John von Neumann
regarding what name to give to the “measure of uncertainty” or
attenuation in phone-line signals (1949)
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Connection II: Wavelet Theory
and Image Processing
Wavelet theory was established in late
1980s by mathematicians, computer
scientists and electrical engineers together
The most successful application of
wavelets is likely to be lossy image
compression (e.g., JPEG2000)
– Also popular in other processing tasks such
as segmentation, denoising and retrieval
The question is: Why?
Xin Li LDCSEE WVU Spring 2009
Where do Wavelets Come from?
Before wavelet, people used Short-Time
FT to analyze transient signals
J. Morlet – a geophysical engineer at a
French oil company came up with an
alternative approach which was
recognized by Grossmann – Daubechies’
advisor
S. Mallat – a graduate student at Penn
met Y. Meyer’s student and recognized its
connection to multi-resolution analysis
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Laplacian Pyramids invented by
RCA Engineers
Xin Li LDCSEE WVU Spring 2009
At the Intersection of Math, CS
and EE
Math: construction of basis functions with good
localization property in both time and frequency
CS: decomposes images under a multiresolution analysis framework in analogy to HVS
EE: analysis-and-synthesis filter banks used by
TV engineers
Merge of roots: a new tool for data/signal
analysis
Different perspectives: deterministic (Besovspace functions) vs. statistical (heavy-tail
2016/3/23
distributions) Xin Li LDCSEE WVU Spring 2009
Why Wavelets for Images?
Math: Besov-space function, statistics: sparse component analysis, neuroscience:
Independent components of natural scenes
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Beyond Image Processing
Statistics: nonparametric regression
Graphics: progressive mesh compression
Turbulence: one of the most complicated
phenomenon in nature
Astronomy: hierarchical clustering theory
of galaxy formation
Biomedical: MRI, EEG, PET,
mammography
Acoustic: computer music analysis
2016/3/23
Xin Li LDCSEE WVU Spring 2009
What is Missing in Wavelet
Models?
DWT
sign flip
IWT
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Connection III: Complex
Networks and Image processing
Common assumption made by MRF and
wavelet models: locality or Markovian
Most existing physical laws are defined
locally; but what about nonlocality?
A great mystery in brain science is how it
collectively processes local information
– Speed of nerve impulse transmission is much
slower than that of logic gates
– The power consumption of neural system is
also much more efficient
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Networks of Neurons
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Complex Networks
Internet
World Wide Web
Movie actor collaboration network
Science collaboration network
Citation networks
Cellular networks
Ecological networks
Power networks
2016/3/23
Xin Li LDCSEE WVU Spring 2009
How is it related to Image
Processing?
Parallel and Distributed Processing (PDP)
or connectionism was at the foundation of
neural networks
Bij 1 2 3 4
j
f3
f2
1
2
f1
3
4
B11
i
B21
B31
B41
2016/3/23
B22
B32
B42
B12
B13
B23
B33
B43
B24
B34
B44
Xin Li LDCSEE WVU Spring 2009
B14
How is it Different from NN?
What role does time play?
– Temporal binding hypothesis in neuroscience
– Synchronization of nonlinear oscillators in chemical,
biological and physical systems
What role does feedback play?
– As important as feedforward
– Mountcastle’s uniformity principle in psychology
Why does the network have to be hierarchical?
– Natural world is organized in a hierarchical fashion
– Our perception of natural world is the consequence of mapping
from outside (physical stimuli) to inside (synaptic connections)
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Experiment 1: Compressed Sensing
x
y
DT
29.06dB
28.46dB
25% kept
KR
FG1
31.56dB
34.96dB
31.16dB
36.51dB
26.04dB
24.63dB
29.91dB
17.90dB
18.49dB
29.25dB
1X.
Li, “Patch-based image interpolation: algorithms and applications,”
Inter. Workshop on Local and Non-Local Approximation (LNLA)’2008
Xin Li LDCSEE WVU Spring 2009
DT- Delauney
Triangle-based
(griddata under
MATLAB)
KR- Kernal
Regression-based
(Takeda et al.
IEEE TIP 2007
w/o parameter
optimization)
Experiment 2: Image Coding
SFG-enhanced
at rate of 0.32bpp
(PSNR=33.22dB)
JPEG-decoded
at rate of 0.32bpp
(PSNR=32.07dB)
SPIHT-decoded
at rate of 0.20bpp
(PSNR=26.18dB)
SFG-enhanced
at rate of 0.20bpp
(PSNR=27.33dB)
Maximum-Likelihood
(ML) Decoding
Maximum a Posterior
(MAP) Decoding
Xin Li LDCSEE WVU Spring 2009
Unfulfilled Connections (I)
Bayer Pattern
(US3,971,065)
Cone distribution in human retina
CCD sensor design (engineering) could benefit from the organizational
principle of cones in human retina (biology)
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Unfulfilled Connection (II)
Q: Can we generate a HDR image (16bpp) by a standard camera?
A: Yes, adjust the exposure and fuse multiple LDR images together
Xin Li LDCSEE WVU Spring 2009
34
High Dynamic Range Imaging
Note that any commercial display devices we see these days are NOT HDR
Xin Li LDCSEE WVU Spring 2009
35
Unfulfilled Connection (III)
Visual perception might be the first small step towards human intelligence
but it will be a huge leap in human intelligence (can brain understand brain?)
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Summary and Conclusions
Every theory, technology or system has its own
evolution path
– Understand its connection is the most difficult yet
important task
– My learning about image processing is still evolving,
but I am hoping its principle can be also applied to
other technical field such as communication
“If you truly believe that God creates this world in
a unified fashion, when you get stuck with a
problem, seek your inspiration from around:
nature, art and other sciences. Essentially, the
principles are the same. ”
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Research vs. Development
Good development/programming skills are a
plus but secondary to good analytical/logical
reasoning skills in my own assessment
Implementation skills should be viewed at the
same level as mathematical skills; they are both
technical tools but cannot replace scientific
vision/understanding
``Knowledge and productivity are like compound
interest.'' –Richard Hamming
Xin Li LDCSEE WVU Spring 2009
How Good do You Need to at
Mathematics?
“Never be overwhelmed by the
mathematics other people are boasting in
their papers if you are in engineering: the
more equations, the fewer ideas”
Mathematics is a language – you cannot
communicate well if you don’t master it;
but you cannot advance science by simply
playing with mathematics
http://masterxinli.wordpress.com/2008/09/15/how
-good-do-you-need-to-be-at-mathematics/
2016/3/23
Xin Li LDCSEE WVU Spring 2009
Why does
engineering/math/science
education in the US suck?
http://headrush.typepad.com/creating_passionate_users/2006/11/why_does_engine.html
Xin Li LDCSEE WVU Spring 2009