REvIEWS
Deep optical imaging within complex
scattering media
Seokchan Yoon1,2, Moonseok Kim3,4, Mooseok Jang1,2,5, Youngwoon Choi6, Wonjun Choi1,2,
Sungsam Kang 7 and Wonshik Choi 1,2 ✉
Abstract | Optical imaging has had a central role in elucidating the underlying biological and
physiological mechanisms in living specimens owing to its high spatial resolution, molecular
specificity and minimal invasiveness. However, its working depth for in vivo imaging is extremely
shallow, and thus reactions occurring deep inside living specimens remain out of reach. This
problem originates primarily from multiple light scattering caused by the inhomogeneity of tissue
obscuring the desired image information. Adaptive optical microscopy, which minimizes the
effect of sample-​induced aberrations, has to date been the most effective approach to addressing
this problem, but its performance has plateaued because it can suppress only lower-​order
perturbations. To achieve an imaging depth beyond this conventional limit, there is increasing
interest in exploiting the physics governing multiple light scattering. New approaches have
emerged based on the deterministic measurement and/or control of multiple-​scattered waves,
rather than their stochastic and statistical treatment. In this Review, we provide an overview of
recent developments in this area, with a focus on approaches that achieve a microscopic spatial
resolution while remaining useful for in vivo imaging, and discuss their present limitations and
future prospects.
Center for Molecular
Spectroscopy and Dynamics,
Institute for Basic Science,
Seoul, Korea.
1
2
Department of Physics,
Korea University, Seoul,
Korea.
3
Department of Medical Life
Sciences, College of Medicine,
The Catholic University of
Korea, Seoul, Korea.
4
Department of Biomedicine
and Health Sciences, College
of Medicine, The Catholic
University of Korea, Seoul,
Korea.
5
Department of Bio and Brain
Engineering, KAIST, Daejeon,
Korea.
6
School of Biomedical
Engineering, Korea
University, Seoul, Korea.
7
Laser Biomedical Research
Center, Massachusetts
Institute of Technology,
Cambridge, MA, USA.
✉e-​mail: wonshik@korea.ac.kr
https://doi.org/10.1038/
s42254-019-0143-2
Nature Reviews | Physics
Optical microscopy is widely used to investigate the
mechanisms underlying various physical and biological
phenomena and to enable minimally invasive disease
diagnosis and treatment. Unlike other imaging modalities that rely on high-​energy sources, such as X-​rays
and electron beams, this approach does little harm to
samples of interest, while producing valuable molecular
information at a subcellular spatial resolution. Optical
microscopy is thus used in various biological and medi­
cal applications, including the investigation of protein
interactions and signalling pathways1, monitoring of the
dynamics of neural circuits at the cellular level and at
timescales ranging from milliseconds to months2, and the
non-invasive imaging of cancer for intraoperative surgical guidance and screening3. However, a dis­advantage
of optical imaging is that it is extremely near-sighted
because the light waves are deflected multi­ple times
when propagating through complex scattering media.
Owing to this multiple light scattering, the spatial resolving power of optical imaging degrades significantly
even at depths of just a few hundred micrometres when
used in bioimaging (Fig. 1a). Although there have been
advances in the past two decades in improving the
resolving power beyond the diffraction limit, the difficulties associated with overcoming the higher-​order
perturbations caused by multiple light scattering have
slowed progress in increasing imaging depth4–7. Thus,
advanced applications of optical imaging in the fields of
life science and medicine have yet to be realized.
To understand the challenges associated with microscopic imaging within complex scattering media, it is
important to understand the way that light waves pro­
pagate in reflection-​mode imaging, which is the relevant
modality for in vivo imaging. Consider a target object
located at a depth z0 with a plane wave with transverse
wave vector ki incident on the medium (Fig. 1b). The
propagation direction of the internal wave changes
owing to the inhomogeneity of the medium, leading to
the wave being scattered through various pathways. The
signal of most interest is the single-​scattered (SS) wave,
often called ballistic light, denoted as E SS(k o; τ0 ) with
the output transverse wave vector ko and the f light time
τ0 = 2z0/c′, where c′ is the average speed of light in the
medium. The SS wave experiences a single scattering
event by the target object but no scattering events in
the intervening medium. An ideal diffraction-​limited
object image can be reconstructed with the SS wave
because its momentum change, ko − ki, is directly related
to the structure of the object. Notably, the intensity of
the SS wave is attenuated by a factor of exp(−2z/ls),
where ls is the scattering mean free path of the medium
(see section I in the Supplementary Information).
Reviews
Key points
• Optical microscopy is an indispensable tool in biology and medicine owing to its high
spatial resolution, molecular specificity and minimal invasiveness, but it is limited to
the interrogation of superficial layers for in vivo imaging.
• The intensity of single-​scattered waves used in conventional imaging decreases
exponentially with depth; thus, the imaging depth limits are set by the detector
dynamic range and the efficiency of the gating operations.
• Approaches that make deterministic use of the abundant multiple-​scattered (MS)
waves have been proposed to enable deep optical imaging while maintaining the
microscopic spatial resolving power.
• Recording and controlling the wavefront of MS waves enables a complex scattering
layer to be converted into a focusing lens, leading to the development of an ultrathin
endoscope.
• Acousto-​optic interactions and wavefront sensing and/or control are integrated
to exploit the large penetration depth of ultrasound and high spatial resolution of
optical imaging.
• Reflection-​matrix approaches that record and process all MS waves enable the
correction of sample-​induced aberrations, exploitation of multiple scattering
signals and suppression of multiple scattering noise, allowing for imaging at depths
greater than those accessible with conventional confocal and adaptive optics
microscopy.
Isoplanatic patch
The area over which the
wavefront error remains almost
the same.
Owing to this exponential attenuation, the SS wave is
obscured by multiple-​scattered (MS) waves even at a
depth of just a few ls.
Except for the SS wave, the incident wave is fully
converted into MS waves. There are several types of
MS wave that return from the scattering medium
with the same ko as that of the SS wave and are therefore indistinguishable from the SS wave based on their
output propagation direction. Of these, two MS waves
E TM(k o; τ0 ) and E BM(k o; τ0 ) have the same flight time
as the SS wave. Here, E TM(k o; τ0 ) describes an MS wave
that interacts with the target object at least once. Its
momentum change is not identical to the object’s spatial frequency spectrum owing to scattering events in
the complex media but is loosely related to the object’s
structure because the deflection angles are ≪1 rad. This
MS wave consists of ‘snake photons’ and is responsible for the finite size of the isoplanatic patch in adaptive
optics (AO)8. E BM(k o; τ0 ) indicates an MS wave that
travels at depths shallower than that of the target object
and thus has no interaction with the object. E TM and
E BM are stronger than SS waves because their intensity is
attenuated exponentially as exp(−2z/l′), where the decay
length l′ is longer than ls (refs9,10). It is difficult to estimate l′ because it depends on various factors, such as the
specifications of the imaging optics and the properties
of the scattering medium. Nevertheless, measurements of
l′ (ref.10) and the analytical estimation of the ratio between
the SS wave and MS waves11 have provided insight into
the interaction between these factors. The remaining
MS waves are described by E DC(k o; τ ≠ τ0 ) and travel
along longer or shorter path lengths than that of the SS
wave and thus have different flight times. The relative
intensities of the SS wave and various MS waves (Fig. 1c)
demonstrate the difficulty in achieving high-​resolution
deep optical imaging. For example, at a target depth of
z0 = 10ls, which corresponds to ~1 mm in biological tissue, the intensity of the SS wave is attenuated by a factor
of e–20 with respect to E DC.
In the past two decades, various deep optical imaging
approaches have been developed to overcome the detrimental effects of complex media on optical imaging.
Early attempts at deep optical imaging have been previously reviewed12. The most straightforward approach has
been to filter out the MS waves using gating operations.
For example, in confocal detection13, the SS waves are
selectively collected along the propagation direction,
although a considerable portion of the MS waves (E TM,
E BM and E DC) bypass confocal gating. Temporal gating
based on low-​coherence interferometry removes E DC,
but E TM and E BM still contribute to the noise14,15. In fluo­
rescence imaging, E BM and E DC can be removed using
spectral filters because their wavelengths differ from
those of the fluorescence emissions16. Alternatively, ultrasound waves are focused on the target object to modulate
the light waves that interact with it17,18. Selective detection of the modulated wave can eliminate a large fraction
of E DC. In many cases, multiple gating operations are
combined to maximally reject unwanted multiple scattering noise19–21. In addition to gating approaches, multi­
photon imaging based on nonlinear excitation22–28 has
proved to be effective in increasing the imaging depth
(Fig. 1a). Longer-​wavelength excitations reduce multiple
light scattering, and the nonlinear response of the emission is favourable for excitation localization. However,
multiphoton excitation is hampered by the attenuation
of the excitation-beam intensity by scattering and aberrations, which is currently the main reason for imaging
depth limitations.
Despite these previous efforts, the imaging depth of
high-resolution optical microscopy, which has a spatial resolving power close to the ideal diffraction limit,
has largely been limited to the superficial layer of bio­
logical tissue (<1 mm). This limitation is primarily due
to the reliance on the readily obscured SS waves. The
detector dynamic range and SS gating efficiency are two
major factors in determining the achievable imaging
depth in the case of optical coherence imaging10, and
the sample-induced aberrations pose an additional
limit (Box 1).
In addition to the noise created by the MS waves,
the SS wave is prone to perturbation by the scattering
medium. The SS wave experiences angle-​dependent
phase retardations, referred to as sample-​induced aberrations, due to the inhomogeneity of the medium. The
aberrations cause the focal spot to blur and degrade
the peak intensity of the point spread function (PSF; see
section II of the Supplementary Information). Sample-​
induced aberrations are unavoidable in deep optical
imaging, and it is difficult to reach the gating limit
because these aberrations attenuate the intensity of the
SS wave, E SS 2, in the image-formation step. Their effect
is characterized by the Strehl ratio, which is defined as
the ratio of the peak intensity of the PSF in an aberrated
system to that of the ideal diffraction limit (the dashed
blue line in the figure in Box 1 indicates the depth limit
with sample-​induced aberrations). AO has been adopted
to remove sample-​induced aberrations (see section II
of the Supplementary Information)8,29 and raise the
effective imaging depth limit (Box 1 figure). However,
most AO microscopy methods are either prone to
www.nature.com/natrevphys
Reviews
a
10 mm
Imaging depth
PAT
MPM
1 mm
UOT
100 μm
OCT and OCM
STED, PALM
and SOFI
Confocal
10 μm
10 nm
100 nm
1 μm
10 μm
100 μm
1 mm
Spatial resolution
b
Ei (ki)
ESS
ETM EBM
c
1.2
EDC
Total reflection
z0
Target
object
Normalized intensity
1.0
0.8
Multiple scattering with
different flight times
0.6
0.4
Time-gated
multiple
scattering
0.2
Single scattering
0
0
5
10
Penetration depth (z0/ls)
15
Fig. 1 | Effects of multiple light scattering on imaging depth. a | Imaging depth of various imaging modalities: stimulated
emission depletion (STED), photoactivated localization microscopy (PALM), super-​resolution optical fluctuation imaging
(SOFI), optical coherence tomography (OCT), optical coherence microscopy (OCM), multiphoton microscopy (MPM),
ultrasound-​modulated optical tomography (UOT) and photoacoustic tomography (PAT). b | An input wave (with a signal E i
and a wave vector ki) incident on a scattering medium with a target object at depth z0 undergoes scattering events,
producing various types of scattered wave with the same output wave vector ko for the given ki. E SS(k o; τ0) denotes the
signal of a single-​scattered (SS) wave (where τ0 is the flight time). E TM(k o; τ0) and E BM(k o; τ0) are multiple-​scattered (MS)
waves that do and do not interact with the target object, respectively, while having the same flight time as the SS wave.
By contrast, E DC(k o; τ ≠ τ0) are MS waves that have different flight times to that of the SS wave. c | The relative intensities
as a function of z0 for an SS wave (E SS), time-​gated MS waves (E TM and E BM) and other MS waves in typical cases in which
the transport mean free path, lt, is ~10 times larger than the scattering mean free path of the medium, ls. The decay lengths
for the curves were derived from ref.10. Panel a adapted from ref.149, Springer Nature Limited. Panel b adapted from ref.10,
Springer Nature Limited.
strong multiple scattering noise or are unable to correct
high-​order aberrations.
A direct approach to increasing imaging depth is to
devise new types of gating mechanism, such as acoustic
gate spacing30, that more effectively filter out MS waves.
Another approach is to exploit the E TM waves to form
an image, which raises the signal-to-noise ratio of the
image, suggesting that the maximum achievable imaging
depth for a given dynamic range of a detector can be
improved (Box 1). Diffuse optical tomography and photo­
acoustic imaging are good examples of this approach,
because their imaging depth is much greater than that
of high-​resolution optical microscopy owing to the use
of MS waves31–33. However, their spatial resolution is worse
than the diffraction limit because the stochastic treatment of MS signals results in the loss of spatial resolution. In overcoming these limitations, the main question
Nature Reviews | Physics
is whether E TM waves that interact with the target object
can be deterministically used to increase the imaging
depth while maintaining the ideal diffraction-limited
spatial resolution.
About a decade ago, it was experimentally demonstrated that MS waves can be controlled deterministically using wavefront shaping to form a sharp focus
and to construct an object image even through a highly
scattering medium34,35. Although this approach is not
yet directly applicable to the optical imaging of targets embedded within a scattering medium, it has led
to the development of various deep-optical-imaging
approaches that perhaps would not otherwise have been
conceivable. For example, concepts related to meso­scopic
scattering theory36–39, such as short- and long-​range
correlations and open eigenchannels, have been developed for optical focusing and imaging40. The scattering
Reviews
Box 1 | Depth limit of optical coherence imaging
Maximum imaging depth (ls)
22
The principal factors that
determine the imaging
20
depth of optical coherence
18
imaging10 are discussed
Exploit
16
here; however, note that
New
MS
14
gating
the depth limit depends
12
on the imaging modality
AO
and the specifications of
10
the system. There are two
8
imaging depth limits: the
6
detector dynamic range
Dynamic range limit
4
Gating limit
limit and the singleGating limit with aberrations
2
scattered (SS) gating limit.
0
The detector sensor should
100
101
102
103
104
105
resolve the SS wave in
Dynamic
range
the presence of strong
multiple-​scattered (MS) waves, requiring the detector dynamic range to exceed the
intensity ratio of the SS wave to all MS waves, SMR = ∣E SS∣2∕ ∣E TM∣2 + ∣EBM∣2 + ∣EDC∣2
(where ∣E TM∣2, ∣EBM∣2 and ∣EDC∣2 are the intensities of the different MS waves).
In inter­ferometric detection, the dynamic range should be larger than the square root
of the SMR, which sets the dynamic range limit (solid red line in the figure). For the
typical sensor dynamic range of ~104, the imaging depth can reach ~18ls (where ls is the
scattering mean free path of the medium). Even if the SS wave is detected by the sensor,
diffraction-​limited imaging is not possible unless the SS wave can be distinguished from
the MS waves. Therefore, the intensity ratio of the SS wave to the gated MS waves,
SMRG = ∣E SS∣2∕ ∣E TM∣2 + ∣EBM∣2 , should be >1, which sets the gating limit (solid blue line
in the figure) for typical aberration-​free optical coherence microscopy equipped with
temporal, confocal and polarization gating10,11. Both the dynamic range and gating limits
are affected by factors such as the spatial resolving power, temporal gating width and
the scattering parameters of the scattering medium. For a spatial resolution of ~1 μm, the
gating limit is set at ~12ls, which is much lower than the dynamic range limit for the typical
dynamic range of ~104 (black dashed line in the figure). Therefore, in state-of-the-art
imaging, the maximum imaging depth is mainly set by the gating limit. In practice,
the imaging depth is often shallower than the gating limit owing to sample-induced
aberrations, which decrease the peak intensity of the point spread function, resulting in
the attenuation of the SMRG. The dashed blue line in the figure indicates the depth limit
when the Strehl ratio is 0.01, and the black arrows indicate the potential gain in imaging
depth through adaptive optics (AO), the new single-scattering gating method (new
gating) or the deterministic use of multiple-​scattered waves (exploit MS).
(
(
Guide stars
Bright, point-​like light sources
or scatterers that provide a
wavefront reference for
measuring and correcting
wavefront distortions in
adaptive optics systems.
Point optimization
A method or algorithm that
optimizes the point spread
function by minimizing
wavefront errors in adaptive
optics systems.
Spatial light modulator
A device that modulates the
amplitude, phase or polarization
of light waves in space.
Optical memory effect
The phenomenon that speckle
patterns of scattered light
through thin and diffusive
media are invariant to small
tilts or shifts in an incident
wavefront of light.
)
)
matrix formalism (see section III of the Supplementary
Information) has also been revisited as a more complete
description of the light–medium interaction, opening
new avenues for the manipulation of MS waves for
optical focusing and imaging in scattering media41.
Although these developments have not yet enabled
the ultimate goal of the full deterministic use of MS
waves for in vivo deep microscopic optical imaging,
advances are being made towards the better use of MS
waves and increasing the practicality of the proposed
approaches. In this Review, we describe the principles
of several key experimental methodologies that both
attain microscopic spatial resolution and can potentially be applied to in vivo imaging. Because this research
area is still in its developmental stage, we conclude by
discussing present challenges and future prospects.
Imaging through a scattering medium
In this section, we discuss experimental approaches
that make deterministic use of MS waves transmitted
through a scattering layer with finite thickness. In these
approaches, it is assumed that a detector can be placed
on the plane behind the scattering layer for measuring
the transmitted MS waves. We first discuss optical wavefront shaping techniques for focusing transmitted MS
waves, along with their applications for imaging through
scattering media. Subsequently, we discuss generalized
approaches based on the measurement of the transmission matrix (TM) of a scattering layer and their use for
endoscopic reflectance and fluorescence imaging.
Focusing multiple-​scattered waves. In AO microscopy,
wavefront-shaping has been used to control weakly scattered waves only, owing to the practical constraints in
bioimaging, such as the deficiency of ideal guide stars
and the requirement for a short image acquisition
time. Control of MS waves induced by a large number
of scattering events has been considered possible in
principle and previously studied in acoustics42–44 and
microwaves45,46. However, this control was realized only
comparatively recently in optics. Two noteworthy examples are the demonstration of optical focusing behind
a highly scattering layer34 by iterative feedback control
of the incident wavefront, a process often referred to
as point optimization, and the demonstration of phase
conjugation through thick biological tissues35.
In the input–output relationship of a linear system,
vt = Tuin (where T is the transmission matrix of the system; see section III of the Supplementary Information),
the output wave vt can be controlled by adjusting the
incident wave uin in the point-​optimization method.
A random speckle pattern is generated when a strongly
scattering medium is illuminated with a plane wave
(Fig. 2a). The point-​optimization method adjusts the
phases of the incident wavefront by using a wavefront-​
shaping device such that the light waves arriving at a
predefined point on the opposite side of the medium
constructively interfere (Fig. 2b). Consequently, the intensity at the target point can be substantially increased relative to the surrounding region. Although optimization is
conducted for a single point, the overall transmission can
be increased47 (Fig. 2c–e), because the point-​optimization
process increases the contribution of eigenchannels with
high transmittance48.
One application based on point optimization is
instantaneous image delivery though a scattering layer49.
Point optimization through a spatial light modulator
(SLM; Fig. 2f) enables the recovery of the image not only
at the optimized point but also at adjacent points owing
to the optical memory effect37,38,50. In one example, point
optimization was used to refocus scattered light from a
point source49 (Fig. 2g,h). With the optimized SLM correction, an object image that was invisible before the point
optimization (Fig. 2i) could be clearly resolved (Fig. 2j).
The point-​optimization method has also been used to
generate a focal spot sharper than the diffraction-​limited
size determined by the numerical aperture of a focus­ing
lens51. Placing a scattering layer between the focusing
lens and a detector increases the deflection angles, which
enables the generation of a focal spot that is smaller than
the diffraction limit set by a conventional lens. Indeed,
using a scattering layer with a high refractive index, a
sub-100-nm focal spot was generated52. Light has also
been focused at a subwavelength scale by controlling the
near field generated in a metamaterial53, and the control
www.nature.com/natrevphys
Reviews
Micromirrors used for high-​
speed, efficient and reliable
spatial-​light modulation;
originally invented to create
video displays in digital
projectors.
of surface plasmon polaritons has been demonstrated
for a disordered array of nanoholes in a metallic film54–56.
In addition to iterative feedback optimization,
other point-​optimization algorithms have been proposed based on TM recording and optical phase conjugation57. Although liquid-​crystal SLMs have been
widely used (Fig. 2f), they are not fast enough to focus
light in dynamic scattering media. Over the past
decade, notable progress has been made in increasing
the speed of focus generation and image acquisition
using digital micromirror devices (DMDs)58,59, high-​speed
micro-​electro-mechanical systems60 and ferroelectric
SLMs61, with the operating time reduced to 10 μs using
an acousto-​optic modulator62.
Point optimization demonstrates the full control of
MS waves in the sense that a diffraction-​limited spot
is formed, but it cannot be directly applied to deep-​
tissue imaging because the detector needs to be placed
in the object plane. To find a wavefront that forms a
focus within the scattering medium, the detector has
to be embedded within the subject of interest, which is
a
b
Random speckle
Plane wave
c
105
104
103
102
101
100
Focused light
Shaped wave
d
Strongly
scattering
sample
f
10
104
103
102
101
100
5
Strongly
scattering
sample
R
f
SLM
not compatible with most in vivo optical bioimaging.
However, the concepts underlying point optimization
have been adopted to increase the imaging depth in
two-​photon microscopy. In general AO microscopy, several low-​order Zernike modes are controlled to increase
image sharpness. In a different approach, the segments of
the incident wavefront can be controlled to optimize the
two-photon fluorescence from a target beacon63, similar
to the point-optimization method. Conjugate AO fluorescence microscopy was also demonstrated in the in vivo
imaging of neurons through an intact mouse skull over
an extended corrected field of view, achieved by placing
a wavefront-​shaping device in the plane conjugate to the
dominant aberration-​inducing layer (skull), rather than
in the pupil plane of the microscope64. However, these
approaches work only for relatively weak scattering or
a thin aberrating layer because the target beacon has to
be visible to initiate the optimization process. Various
approaches have been explored to overcome this limit­
ation and increase the viability of point optimization for
use in imaging within more strongly scattering media65–68.
g
106
105
104
–15
–10
0.025
h
0.015
0.010
Image
of object
Object
Thin
scattering
medium
BP Lens
filter
3 mm
0.005
–5
0
x (μm)
5
10
15
1.0
0.8
0.6
0.4
3 mm
0.2
Intensity (norm.)
0.020
Intensity (norm.)
Thermal
source
e
Intensity (counts per column)
Digital micromirror devices
0.0
i
j
3 mm
3 mm
Fig. 2 | Point optimization by wavefront shaping and its imaging applications. a,b | Speckled transmission of a plane
wave through a strongly scattering medium (panel a). A focus can be generated by shaping the wavefront of the incident wave
(panel b). c,d | Speckled transmission image of a plane wave through a scattering medium (panel c) and the corresponding
transmission image through a scattering medium after point optimization (panel d). e | Intensity-summed profiles in the
vertical direction of panels c and d. The grey dashed line is the transmission of the plane wave in panel c, and the blue solid
line is the transmission of the point-​optimized wavefront in panel d. f | Instantaneous image delivery through a scattering
medium. R is the distance between the object and the scattering medium, and f is the focal length of the lens. The thin
scattering medium in conjunction with the spatial light modulator (SLM) serves as a ‘scattering lens’ with focal length R.
The SLM shapes the wavefront of the scattered light from the object such that the image of the object is refocused on a
camera. The band-​pass (BP) filter rejects uncorrected wavelengths. g,h | Images of a point produced with a flat SLM (panel g)
and after optimization of the SLM pattern (panel h). The inset shows the phase pattern written on the SLM for optimization.
i,j | Transmitted images of an object (the letter A, shown in the inset) before (panel i) and after (panel j) point optimization.
Panels a and b adapted with permission from ref.34, OSA Publishing. Panels c–e adapted with permission from ref.47,
APS. Panels f–j adapted from ref.49, Springer Nature Limited.
Nature Reviews | Physics
Reviews
Transmission-​matrix approach to image delivery
through scattering media. The TM approach is a general­
ization of point optimization because point optimization is equivalent to recording one column of a TM (see
section III of the Supplementary Information). In the
relationship vt = Tuin, the operator T is measured by
sending an incident wave to each basis vector in uin. With
knowledge of T, the initial object field uin at the input
plane can be recovered from the distorted field vt at the
output plane using the relationship uin = T–1vt. In 2010,
wide-​field image transfer through a scattering medium
using TM measurements was first demonstrated69,70.
Subsequently, the integration of a fast steering mirror
into digital holographic microscopy greatly improved
the resolution and sensitivity of image transfer though
a scattering medium with unprecedented TM information71,72. In addition, the counterintuitive use of multiple scattering to enhance the resolution and extend the
view field has been demonstrated in wide-​field imaging71. These studies confirmed that the TM approach
can transform the scattering medium into a high-​quality
optical element that can surpass conventional optics in
terms of its functionality73.
For the same reason as point optimization, the TM
approach cannot be directly applied to deep optical
imaging. However, it is used in ultrathin endoscopes
in combination with multimode optical fibres. A single multimode optical fibre can, in principle, transmit
images through numerous propagating modes but
scrambles the original image information owing to the
dispersion and mixing of modes. Recording the TM of
the fibre resolves this problem because it converts the
fibre into imaging optics. Because the mode density of a
multimode fibre is 1–2 orders of magnitude greater than
that of the core density of bundled fibres, the diameter
of an endoscope can be greatly reduced. We briefly discuss the demonstration of endoscopic reflectance and
fluorescence imaging based on the TM approach.
Reflectance endoscopic imaging through a single
multimode fibre was first realized in 2012 (ref.74). In
endoscopic imaging, the image is distorted by the fibre
twice — once on the way in and once on the way out.
In this implementation, the distortion on the way in is
removed using the speckle imaging technique and that
on the way out is recovered by applying an inversion of
the TM. A light wave was illuminated through the input
plane of a fibre proximal to a camera (Fig. 3a). After being
transmitted, the light was reflected by the sample at the
object plane and recaptured by the same fibre. To use the
fibre as an image guide, its transmission properties were
calibrated using its TM measured from the object plane
to the image plane (Fig. 3b). Owing to image distortions,
the observed images exhibited complex speckle patterns
(Fig. 3c). By applying the inverse of the measured TM, the
distortion on the way out was removed. After reconstruction, however, the images still retained speckle patterns
(Fig. 3d) because the distortion on the way in had not yet
been resolved. As this distortion arose prior to the light–
object interaction, a clean image was obtained using the
speckle-imaging method, which involves the averaging
of images taken from various illuminations (Fig. 3e).
To demonstrate the ex vivo imaging of tissues, a rat
intestinal tissue was imaged with bright-​field micro­
scopy (Fig. 3f) and with the fibre endoscope (Fig. 3g,h).
Although changes in the conformation of the fibre
cause a loss of TM reconstruction, an object image was
recovered when the tip of the fibre facing the object
was scanned over an extended field of view (Fig. 3g,h).
The TM approach has also been applied to endoscopic
reflectance imaging using a bundled fibre, wherein each
fibre core acts as an image pixel. The main benefit of the
TM approach is the removal of image pixellation caused
by the gap between neighbouring fibres in a bundle75,76.
Compared with reflectance endoscopic imaging, fluorescence endoscopic imaging using a multimode optical
fibre is relatively straightforward because only the distortion on the way in needs to be corrected. From the
measured TM of the fibre, an incident wave uin is identified that generates a focus at the distal end, where the
sample is located. By shaping the incident wavefront as
the identified uin, a focus is physically generated at the
desired position. For point-​scanning imaging, several
incident wavefronts are sequentially shaped by the SLM
to scan a focus at the object plane, and fluorescence emissions at each focus are collected by the same fibre77–80.
A representative configuration of a focus-​s canning
endoscope using a multimode optical fibre is shown in
Fig. 3i. After TM measurement from the DMD to the
object plane located at the distal end, the appro­priate
phase modulations are applied to the DMD to scan the
focus. DMDs have been widely used for high-​speed
image acquisition81–83, although liquid-​crystal SLMs
were used in earlier set-​ups77 (sometimes in conjunction
with an acousto-​optic deflector)79. By taking advantage
of the ultrathin multimode fibre probe, minimally invasive in vivo fluorescence imaging within a mouse brain
was demonstrated82,84 (Fig. 3j). Confocal imaging85 and
two-​photon fluorescence imaging86 have also been realized using multimode fibres for deep-​tissue imaging.
Intensity-​only TMs have been explored to eliminate the
reference arm required for complex-​field detection87,88,
and the fluorescence imaging of a planar object was
demonstrated using intensity-​based speckle correlation
through a fibre bundle89.
Despite recent technological advances, the bending and twisting of the fibre, which leads to deterioration of the TM, still limit the widespread application
of fibre-​based endoscopy. Because pre-​calibration is
readily undermined by small deformations or twisting,
it is crucial to maintain the fibre geometry during the
imaging process74,79. To dynamically compensate for
fibre bending while focusing light through a multimode
fibre, a virtual point source was generated at the tip of
the fibre, and the speckled wavefront generated through
the fibre by this virtual point source was measured at the
camera plane and phase-​conjugated90. Moreover, under
certain bending conditions, the fibre output characteristics can be conserved and the shaped focus remains
intact91. In addition, the transmission-mode properties
of deformed multimode fibres have been numerically
predicted92, and image reconstruction has been demonstrated under the bending deformation of a graded-​
index fibre93. These efforts to resolve the problems
associated with bending and deformation will enable
www.nature.com/natrevphys
Reviews
a
c Reflected object images
b Transmission matrix
OP
GM
Laser
BS1
Single fibre
IP
Object
BS3
Camera
g
f
d Reconstructed images
(θξ,θη)T
BS2
x
100 μm
y
OL
(θx ,θy)S
y
…
100 μm
(θx,θy)S
η
…
x
ξ
h
e
Speckle imaging
A
A
B
B
50 μm
100 μm
i
Step 1
DMD
HW
L
L
HW
L
L
M
HW
Haemorrhaging blood
cells (time-lapse images)
L
OL
Laser
L
ISO
M
L
QW
L
HW BS
Camera
Fig. 3 | TM approach to image transfer and its application in endomicroscopy. a | Schematic showing the set-up for wide-field endoscopic imaging
using a multimode optical fibre (where OP is the object plane and IP is
the input plane). b | Transmission-​matrix (TM) elements measured along the
direction from the OP to the IP shown in panel a. (θξ,θη)T is the illumination
angle of the beam impinging on the fibre at the OP. c | Reflection images of
an object taken at the IP using the set-​up shown in panel a for various illumination angles (θx,θy)S. d | Object images reconstructed by multiplying the
images in panel b by the inverse of the TM. e | Object image recovered after
averaging the reconstructed images in panel d. f | Transmission image of rat
villi taken using a conventional microscope. g,h | Image of the villi shown in
panel f taken using endoscopic imaging through a multimode fibre (panel g)
and an image obtained by numerical refocusing of the image in panel g by
40 μm towards the fibre end (panel h). Arrows A and B indicate the villus in
focus in panels h and g, respectively. Owing to the weak absorption by the
villi, the reflectance endoscopic images in panels g and h provide stronger
contrast than that of the transmission image in panel f. i | Schematic showing
the set-up for scanning fluorescence endoscopy through a multimode optical fibre. In step 1, the TM of the fibre is measured between the input from the
digital micromirror device (DMD) and the output at the camera. In step 2,
images of an object (in this case, a mouse) are acquired in vivo using the cali­
brated fibre. j | Images of neuronal somata and neuronal processes (indicated
by the white arrows) taken through direct insertion of the fibre probe 2–3 mm
into a mouse brain. Time-​lapse images of blood cells haemorrhaging from
a burst blood vessel (bottom) acquired using the method shown in panel i.
BS, beam splitter; DM, dichroic mirror; Fil, filter; GM, two-​axis galvanometer
scanning mirror ; HW, half wave plate; ISO, isolator ; L , lens; M, mirror ;
OL, objective lens; PBS, polarizing beam splitter; PMT, photomultiplier tube;
QW, quarter-​wave plate. Panels a–h adapted with permission from ref.74,
APS. Panels i and j adapted from ref.84, CC-BY-4.0.
the application of ultrathin multimode fibre endoscopy
in biological and biomedical fields.
In situ imaging using memory effects
The MS waves generated by a thin scattering layer have
a unique correlation characteristic: the speckle patterns
rotate in angle with the rotation of the illumination
wave. This angular memory effect has been reported
in the form of short-​range angular correlations in the
mesoscopic physics of wave propagation in dis­ordered
media37,38,94,95. A tilted plane wave with an angle Δθ (Fig. 4a)
Nature Reviews | Physics
Neuronal processes
10 μm
Fil
DM
Fibre
L
HW
Neuronal somata
PMT
L
Fil
Signal
HW
PBS
Iris
j
Imaging
QW
Laser module
M
Step 2
Calibration
can be regarded as a superposition of point sources with
their relative phases increasing linearly along the lateral
plane. When the scattering layer is thin enough, the
transmitted waves from these indivi­dual point sources
maintain the same phase difference induced by the tilt.
Therefore, the overall transmitted speckle pattern is
also tilted by the same Δθ. After propagating over a distance s, the transmitted speckle pattern projected on a
screen is shifted by Δr ≈ sΔθ. The effective range of the
memory effect depends on the thickness L of the scattering layer and the wavenumber k of the incident light.
Reviews
a
b
Screen
g Polarizing beam splitter
Screen
SLM
HW
Laser
1
θy (°)
1.0
0.5
0
0.5
e Hidden object
θx (°)
1.0
1.5
0.8
Scan lens
Phase
stepper
d
1
0.5
0
–0.5
–0.5
0
θx – θx, 0 (°)
Tube lens
To photomultiplier
tube
Dichroic mirror
Objective lens
0
0.5
Sample
f Retrieved object
1
Polarizing
beam splitter
Piezo scanner
Correlation coefficient
1.5
Integrated intensity (norm.)
c
From
Ti:sapphire
laser
Shift
Shift
Shift
∆r ≈ s∆θ
Tilt
L ∆θ
θy – θy, 0 (°)
Tilt
∆θ
0
GM
Laser
1
h
Intensity (norm.)
Intensity (norm.)
=
|EPSF(r)|3 e–iϕ
=
EPSF(r)
(r)
PSF
Erecon(r)
~EPSF(r)
~δ(r)
100 μm
i
0
j Conventional two-photon image
F-SHARP
Skull
Brain
20 μm
0
Fig. 4 | Non-​invasive imaging using speckle correlations and memory
effects. a | Angular memory effect for a strongly scattering layer of thickness L. Here, s is the distance between the scattering layer and the screen,
Δθ is the tilt of the angle of the incident plane wave, and Δr is the shift of the
speckle pattern at the screen due to the tilt. b | Translational memory effect
for an anisotropic scattering medium. When the incident wave is laterally
shifted, the transmitted speckle pattern is shifted by the same amount.
c–f | Exploitation of multiple-​scattered waves for non-invasive imaging using
the angular memory effect. Panel c shows the integrated fluorescence intensity I(θ) measured as a function of the incident angle θ = (θx,θy) for a fluor­escent
target located at the screen shown in panel a. Panel d shows the autocor­
relation I I averaged over nine scans with different starting incident angles,
θ0 = (θx,0,θy,0). Panel e shows a fluorescence microscopy image of an object
taken without the scattering layer. Panel f shows the image of the object
retrieved from the autocorrelation in panel d using a Gerchberg–Saxton-​type
algorithm. g,h | Scattering compensation by focus scanning holographic
aberration probing (F-​SHARP). Panel g shows the experimental set-​up for
F-SHARP. The half-​wave plate (HW) and the first polarizing beam splitter split
the excitation laser beam into a strong beam and a weak beam. A strong
scanning field Escan(r′ − r) is generated at the focus plane of the objective lens
k F-SHARP
1
20 μm
Intensity (arb. units)
Coverslip
Intensity (arb. units)
Objective
1
0
l
1.4
Intensity
(arb. units)
0
0.0
0
5
10
x (μm)
System correction
15
F-SHARP
by the beam passing through a phase stepper and is scanned by a piezo scanner across a coordinate r. A stationary field Estat (r′) is generated by the weak
beam illuminating the sample at a location r′ via the galvo scanning arm
(where GM is a two-axis galvanometer scanning mirror). In the experimental
demonstration, the strong and weak beams were switched. For two-​photon
imaging, the strong beam is sent through the spatial light modulator (SLM) for
aberration correction and raster scanned at the sample by the GM. Panel h
shows the theoretical description of F-​SHARP for measuring the scattered
electric-​field point spread function (EPSF). The cubic term of Escan, a δ-like function, is convolved with Estat during the two-​photon fluorescence recording by
the combined excitation of Estat and Escan. Both Estat(r′) and Escan(r′) are scaled
by EPSF(r′), and thus Estat(r′) ∝ Escan(r′) ∝ EPSF(r′). Thus, the reconstructed electric
field Erecon(r) in this recording gives EPSF(r). ϕPSF(r) is the phase of EPSF(r). i–l | Panel i
shows the set-up for in vivo imaging of a mouse brain using F-​SHARP. Panels j
and k show the two-​photon imaging of an interneuron that expresses green
fluorescent protein 480 µm below the dura mater before (panel j) and after
(panel k) the application of F-​SHARP. Panel l shows the intensity plot along
the dashed lines in panels j and k. Panels a and b adapted with permission
from ref.96, OSA Publishing. Panels c–f adapted from ref.97, Springer Nature
Limited. Panels g–l adapted from ref.105, Springer Nature Limited.
www.nature.com/natrevphys
Reviews
This range can be determined using the theoretical
correlation function:
 k Δθ L 2

C ( Δθ , L ) = 
 sinh(k Δθ L ) 
(1)
Similar to the angular memory effect, a translational
memory effect50,96 has been reported in scattering media,
with a transmitted speckle pattern maintaining its
correlation along the transverse direction (Fig. 4b).
An approach that uses the angular memory effect
has been reported for the non-​invasive imaging of fluor­
escent objects hidden behind a scattering layer89,97–102.
A plane wave incident at an angle θ = (θx,θy) was used
to illuminate a fluorescent object placed at a distance s
behind an opaque scattering layer, and the fluor­escence
emissions from the object were recorded on the illuminated side. The recorded signal is described by the convolution between the intensity PSF, IPSF(r), and the object
function, O(r), where r = (x,y) is the position vector in
object coordinates. Because IPSF(r) is similar to the δ function in conventional microscopy, the measured intensity
directly resembles O(r). In the presence of aberrations
and multiple scattering, IPSF(r) is highly scrambled into
a speckle pattern, Isp(r), making it difficult to extract
O(r). Nevertheless, the total backscattered fluorescence
intensity as a function of θ (Fig. 4c) can be written in convolution form within the finite translational invariant
range of Isp(r), that is, the memory range:
∞
I (θ) = ∫ O(r)Isp(r − θs )d2r = O ∗ Isp(θ)
−∞
(2)
O(r) was retrieved from the autocorrelation of I(θ)
(Fig. 4d), which is expressed as
⟨I ⋆I ⟩(Δθ) = [O ⋆O] ∗ ⟨I sp ⋆Isp⟩
Epi-​detection geometry
An imaging configuration in
which an objective is used
for both illumination
and detection.
Nature Reviews | Physics
(3)
where ⋆ denotes the cross-​correlation product, and
the angle brackets denote the average over different
I(θ) taken at different starting incident angles. Because
Isp is a speckle pattern, its autocorrelation is approximately a δ function. Therefore, ⟨I ⋆I ⟩ can be written as
⟨I ⋆I ⟩(Δθ) ≅ [O ⋆O]. O(r) can then be obtained from the
autocorrelation using a Gerchberg–Saxton-​type algorithm103,104. The object image retrieved from the averaged
autocorrelation resembles the fluorescence image taken
without a scattering layer (Fig. 4e,f).
This method is an important step towards deep optical imaging as it makes full deterministic use of MS waves
and is applicable to strongly scattering media in which
SS waves are almost invisible. Furthermore, it can support non-​invasive imaging under epi-​detection geometry
without guide stars. However, this method needs a
finite distance between the scattering layer and the
object to secure a sufficient imaging area, and the view
field is reduced as the thickness of the scattering layer is
increased and, thus, the memory-​effect range decreased.
The range of the translational memory effect is
related to the size of the isoplanatic patch in AO microscopy. If high-resolution wavefront correction is conducted for numerous scattered modes, a tight focus
inside an anisotropic scattering medium is achieved and
maintained over a small but finite translational memory-​
effect range50. An approach termed focus scanning holographic aberration probing (F-​SHARP) achieves rapid,
high-​resolution wavefront correction of both strong
aberrations and higher-​order scattered modes in living
tissue105. With this approach, it was possible to measure
the phase and amplitude of the scattered electric-​field
PSF (EPSF) and generate a tightly focused excitation beam
by transmitting the optical phase conjugation of EPSF.
To measure EPSF , a nonlinear interaction between
two excitation beams can be exploited (Fig. 4g). In the
theoretical description of this approach, a weak stationary beam first illuminates the sample at a location r′
to form the stationary field Estat(r′). A strong beam is
then scanned across coordinate r using a piezo scanner
to form the scanning field Escan(r′–r). The two-​photon
fluorescence intensity generated by the superposition of
the two beams can be written as
I (r) ∝ ∫ ∣E scan(r ′ − r) + E stat (r ′)∣4 dr ′
(4)
When E stat is much weaker than E scan, E stat terms
with a power ≥2 can be ignored and the cubic term of
Escan(r′) can be considered a highly peaked δ function
(Fig. 4h). The captured fluorescence intensity can then be
approximated as
∗
I (r) ∝ Ibackground + E PSF(r) + E PSF
(r)
(5)
where Estat(r) has been replaced by EPSF(r), which is
extracted from I(r) using phase-shifting interferometry
with a phase stepper (Fig. 4g) . In the experimental
demonstration of this approach, the strong beam is set
to be stationary and the weak beam is scanned by a piezo
scanner. The strong beam is then corrected by applying
the complex conjugate of the Fourier transform of the
measured EPSF to the SLM. After a finite number of consecutive correction steps, the strong beam can be transformed quickly into a sharp focus. After all the correction
steps, the weak beam is blocked and the strong beam
is raster scanned by a galvanometer scanning mirror to
obtain two-photon images.
The effectiveness of F-​SHARP microscopy was
demonstrated for in vivo imaging of a mouse brain. In an
anaesthetized GAD67 mouse, interneurons labelled with
green fluorescent protein were imaged 480 µm below the
brain surface following a craniotomy (Fig. 4i). Compared
with that of conventional two-​photon microscopy
(Fig. 4j), the image quality was greatly enhanced after
correction (Fig. 4k). The F-​SHARP images exhibited a
fivefold increase in the signal from the corrected region
and a higher resolution (Fig. 4l). In many ways, F-​SHARP
resembles AO microscopy that directly detects the
wavefront of fluorescence from nonlinear guide stars106.
However, the interferometric detection of EPSF by introducing the nonlinear interaction between two beams
can greatly increase the sensitivity of wavefront sensing.
Unlike existing iterative wavefront-shaping AO methods,
F-​SHARP determines the wavefront correction by raster
scanning EPSF using fast scanning mirrors. This approach
decouples the wavefront measurement speed from the
Reviews
resolution limit of TRUE, two approaches have been proposed to date based on the non-​uniform pressure profile
of focused ultrasound waves: the iterative conjugation of
frequency-​shifted light and the time reversal of variance-​
encoded light. In the iterative approach, the measurement and conjugation process for frequency-​modulated
Ultrasound-​assisted optical imaging
light is repeated to find an open eigenchannel that maxUnlike light waves, which are prone to multiple light imizes the light transmitted through the ultrasound
scattering in biological tissue, ultrasound can propa- focus118,119. Owing to the Gaussian-​like shape of the
gate through tissue without notable distortion up to ultrasound focus, the maximally transmissive channel is
depths of 1–100 mm (ref.107). The greater penetration confined to an area close to the peak of the ultrasound
depth can be translated into optical imaging through focus, where the modulation is strongest. In the variance-​
ultrasound–light interactions. Two major interaction encoding method, several input–output matrices are
mechanisms have been actively employed to date: the measured at multiple positions adjacent to ultrasound
frequency shift of optical waves using ultrasound and foci for thousands of different input modes120. The optithe generation of ultrasonic waves using optical waves. mal wavefront is then calculated to maximize the variIn bioimaging, the former mechanism has led to the ance ratio between the sum and difference fields using
development of ultrasound-​modulated optical tomo­ singular-​value analysis. Typical frequency-​modulated
graphy (UOT), in which an ultrasound-​induced refrac- optical fields measured at different ultrasound positions
tive index grating and movement of optical scatterers are shown in Fig. 5c, along with their sum and difference.
modulate the frequency of optical waves propagating Because a single speckle grain located at the centre of
through an ultrasonic focus18,108,109. The latter mecha- an ultrasound focus statistically has the largest variance,
nism was used to develop photoacoustic tomography this approach in principle realizes speckle-​scale optical
(PAT), in which absorbed optical energy produces heat focusing (Fig. 5c). To date, the resolution based on the
and subsequently generates ultrasound from the ther- non-​uniformity of an ultrasound focus is 5–10 μm, which
mal expansion of the medium18,110,111. The trade-​off for is a threefold to fivefold increase over the ultrasonic difthe increase in penetration depth is that the resolving fraction limit118–120. A generalized framework has been
power of these ultrasound-​mediated imaging methods presented for iterative and variance-​encoding methods
is far lower than the optical diffraction limit. The resolv- in the context of the TM121. For these approaches to
ing power is dictated by the ultrasonic diffraction limit, operate as high-​resolution microscopy techniques, howwhich is 20–1,000 μm for the commonly used ultrasound ever, they need to be applicable to target objects that are
frequency range of 1–50 MHz. Higher frequency ultra- fully embedded within scattering media, in which the
sound has been considered to improve the resolution, speckle grain size approaches half the optical wavelength.
but this approach is not practical because of the weaker The technical challenge ahead is the large number of
ultrasound-​mediated signal and the severe attenuation measurements required to guarantee statistical robustof the high-​frequency ultrasound in biological tissue112. ness; this number grows cubically with the number of
Advances in the field of wavefront shaping have optical speckle grains within the ultrasound focus121.
As an alternative to adopting wavefront-​shaping
improved the spatial resolution of UOT-​based and PAT-​
based approaches. Before discussing these approaches, it techniques, which place an emphasis on controlling MS
is important to understand how the wavefront-​shaping optical waves, ultrasonic light modulation in coherent
technique34,40,113 is used to enhance the sensitivity of confocal microscopy can be employed to reject MS
ultrasound–light interactions. When combined with waves travelling outside the ultrasound focus30. In other
UOT-​b ased or PAT-​b ased approaches, an optimal words, acousto-​optic modulation is used as a form of
wavefront can be obtained to focus light on the ultra- space gating that is similar to the temporal and confosonic guide star through approaches based on iterative cal gating used in optical coherence microscopy122. The
optimization34,57, optical phase conjugation114 or a TM70. space-​gating mediated by acousto-​optic modulation
Owing to the reciprocity of light scattering, the optimal acts inside the scattering medium in which the target
wavefront cancels the effect of light scattering in a ‘time-​ object is located, unlike temporal gating and confocal
reversed’ manner and is focused back onto the ultra- gating, which are applied to the optical waves that return
sound focus. For imaging purposes, the focused spot to the free space. Therefore, space gating can reject MS
can be scanned through the object to yield fluorescence waves that the other gating operations cannot; as a result,
or optical absorption-​contrast images.
the combined use of these available gating operations
Using UOT-​a ssisted wavefront shaping, time-​ maxi­mizes the visibility of SS optical waves. Because this
reversed ultrasonically encoded (TRUE) optical focusing method relies on the detection of SS waves, the resolubased on optical phase conjugation was demonstrated, tion is guaranteed to reach the optical diffraction limit,
in which the wavefront of a frequency-​shifted optical regardless of the speckle grain size. However, owing to
wave was directly measured and phase conjugated115. the exponential extinction of SS waves, the penetration
This approach (Fig. 5a) reverses the propagation of all depth of the gating approach will be set by the detector
frequency-​modulated optical waves, including both SS dynamic range limit (Box 1), whereas the approaches
and MS waves. Therefore, the focusing resolution is dic- based on the wavefront shaping of modulated MS waves
tated by the ultrasonic diffraction limit of a few tens to can overcome this limit if the aforementioned challenges
hundreds of micrometres115–117 (Fig. 5b). To overcome the are addressed.
limited speed of the wavefront shapers, which is crucial for correcting a large number of scattered modes
(>1,000). Further increases in imaging depth will depend
on relieving the requirement for a spatially sharp initial
EPSF for the method to converge.
www.nature.com/natrevphys
Reviews
a
Digital phase conjugation
Ultrasound
transducer (fUS)
Photodetector
d
SLM
Genetic
algorithm
sCMOS
camera
Reference beam
(f0 + fUS)
Computer
Lenses
Lens
Sample
beam (f0)
SLM
Diffuser
y
Oscilloscope
Blood
b
b
π/2
0
′
20 μm
z
Beam splitters
Dichroic
beam splitter
Sample
x
Water
Ultrasonic
transducer
c
Amplifier
e
b′1
b′2
b′2–3
b′3
b′4
b′1–4
b′1+2+3+4
Linear PAWS
Nonlinear PAWS
Fig. 5 | Ultrasound-​mediated optical wavefront shaping and imaging. a | Experimental set-up of the time-​reversed
ultrasonically encoded (TRUE) optical focusing method. The frequency of the optical wave (f0) is shifted by the ultrasound
frequency (fUS) when it travels through an ultrasound focus. The digital-​phase-conjugation system selectively measures
and conjugates the wavefront of the frequency-​shifted optical wave. Owing to the reciprocity of light scattering, the
conjugated wavefront is focused back to the position of the ultrasound. b | Images showing the typical frequency-​shifted
speckle field b′ (top part; the colour represents the phase and the luminance indicates the normalized amplitude) and
ensemble-​averaged amplitude profile (bottom part) at the plane of the ultrasound focus. c | Frequency-​shifted speckle
fields b′1, b′2, b′3 and b′4 were recorded at four different acoustic foci. The optimal wavefront that maximizes the variance
ratio between the sum field (b′1+2+3+4) and the difference fields (b′2–3 and b′1–4) is phase conjugated to focus the ultrasound-​
modulated wave to a single speckle grain located inside the ultrasound focus (bottom right part). d | Experimental set-up
of a wavefront-​shaping method, known as photoacoustically guided wavefront shaping (PAWS), that iteratively optimizes
the nonlinear photoacoustic signal excited by dual laser pulses. e | Images showing the optimized nonlinear signal
(far right), obtained using the method shown in panel d, in which light is focused on a single speckle grain inside the
ultrasound detection area. sCMOS, scientific complementary metal-​oxide semiconductor; SLM, spatial light modulator.
Panel a adapted from ref.116, Springer Nature Limited. Panels b and c adapted from ref.120, Springer Nature Limited.
Panels d and e adapted from ref.124, Springer Nature Limited.
Unlike the UOT-​based approach, the PAT-​based
wavefront-​shaping approach cannot directly measure
the optimal wavefront. Instead, the acoustic wave generated by optical absorption is iteratively optimized or
the photoacoustic TM measured by injecting light into
different free-space modes at the input plane of the scattering medium. By measuring the photoacoustic TM,
selective focusing on absorbing targets was demonstrated together with the scanning of an optical focus
based on the optical memory effect123. However, similar
to TRUE, the resolution of this approach is intrinsically
limited to the ultrasound diffraction limit owing to the
linearity of photoacoustic signals. To overcome this
resolution limit, nonlinear photoacoustic signals have
been introduced with a dual-​pulse excitation based on
the Grueneisen relaxation effect124. A genetic algorithm
was then used to identify the optimal wavefront that
maximizes the nonlinear signal. This method is referred
to as photoacoustically guided wavefront shaping
(PAWS; Fig. 5d). Unlike the linear photoacoustic approach,
the optimization of the nonlinear signal converges to the
wavefront solution confined within a single speckle
Nature Reviews | Physics
grain inside the ultrasonic detection area (Fig. 5e). The
Gaussian-​shape sensitivity profile of the detection area
can also be used without inducing a nonlinear signal125,
although the enhancement is quite limited. Similar to
iterative TRUE or the variance-​encoding method, the
improved resolution in the PAT-​based approaches can
reach ~5 μm (refs 124,125). For further improvement,
the low measurement sensitivity when the size of the
speckle grains approaches half the optical wavelength
needs to be addressed. Furthermore, in imaging applications, photoacoustic approaches need to be developed to enable arbitrary control of the position of the
speckle-scale focus113.
Ultrasound–light interactions are uniquely valuable in biophotonics because both UOT and PAT provide direct access to the optical flux inside a scattering
medium. The incorporation of wavefront-​shaping
techni­ques has added the superior resolving power of
optics to ultrasound-​mediated approaches, thus overcoming the conventional ultrasonic diffraction limit.
Although there are practical challenges, advances in
sensor and SLM technology may soon realize the full
Reviews
potential of ultrasound-​mediated wavefront shaping and
imaging in microscopic applications.
Deep imaging using a reflection matrix
Epi-​detection geometry is essential for most in vivo
applications, which demand the recording of a reflection matrix. Unlike conventional confocal microscopy,
in which only the signal filtered by the confocal pinhole
is collected for image reconstruction, reflection-​matrixbased approaches make deterministic use of all of the
backscattered signals arriving at non-​confocal positions
as well as those at confocal positions (see section III of
the Supplementary Information). In this section, we first
introduce methods that simultaneously suppress multiple scattering and sample-​induced aberrations without
using guide stars and then describe approaches based on
the coherent superposition of MS waves.
Solving the inverse scattering problem using a reflection matrix. Diffraction-limited imaging requires the
momentum change ko − ki induced by the target object
to be measured for all possible angles supported by the
objective lens. However, MS waves and sample-​induced
aberrations prevent the accurate tracking of ki and ko.
The first experimental system for measuring a wide-​field
time-​gated reflection matrix, R(τ0), was developed in 2015
(ref.10) (Fig. 6a). Using R(τ0), a method referred to as collective accumulation of single scattering (CASS) micro­
scopy was proposed to selectively identify SS waves that
preserve the momentum change induced by the target
object. Two unique properties of SS waves are used in this
process: the depth-​dependent flight time and momentum
conservation. Using CASS microscopy, an imaging depth
of 11.5ls was demonstrated with a near-diffraction-limited
spatial resolution of 1.5 μm (ref.10).
Subsequently, a method termed closed-loop accumulation of single scattering (CLASS)126 was developed that
separately identifies the aberrations in SS waves incident
on and reflected from an object, even in the presence of
multiple light scattering. Using an experimental set-up
similar to that used in CASS microscopy, complex-field
images of backscattered waves, E(ro,ki;τ0), were measured
at a conjugate image plane for a set of wavevectors {ki} that
cover all of the orthogonal input free modes, where ro is
the position vector in image coordinates. By taking the
inverse Fourier transform with respect to ki, the matrix
E(ro,ri;τ0) can be obtained in position space (Fig. 6b).
Here, E(ro,ri;τ0) corresponds to the matrix element of
R(τ0) (see section III of the Supplementary Information).
The appearance of strong off-diagonal elements is due to
multiple scattering and aberrations. By taking the Fourier
transform of the original images, E(ro,ri;τ0) with respect to
ro, the time-gated reflection matrix E(k o, k i ; τ0 ) in k space
is obtained (Fig. 6c). The k basis is a good basis because the
phase-retarded SS wave can be decoupled from the MS
waves. That is, the matrix element can be decomposed by
E(k o, k i ; τ0 ) = P ao(k i + Δk)O(Δk)P ia(k i)
+ E M(k o, k i ; τ0 )
(6)
Here, the first term on the right-​hand side describes
an SS wave scattered once by the target object. Therefore,
its amplitude is proportional to the amplitude of the
reflectance of the object for the spatial frequency
Δk = ko − ki, but its phase is retarded by complex pupil
functions for aberrations in the illumination and imaging
paths,P ai (k i) = exp[− iϕi(k i)]and P ao(k o) = exp[− iϕo(k o)],
respectively. Here, ϕi(ki) and ϕo(ko) are the sample-induced
phase retardation functions in the illumination and
imaging paths, respectively. The second term on the
right-​hand side is the contribution of all time-​gated MS
waves with the same wavevector as that of the SS wave:
E M(k o, k i ; τ0 ) = E TM(k o, k i ; τ0 ) + E BM(k o, k i ; τ0 ).
The main concept underlying the CLASS algorithm
(Fig. 6d,e) is iterative processing of the reflection matrix
to determine the phase corrections θi(ki) and θo(ko)
that compensate for ϕi(k i) and ϕo(ko), respectively.
In every nth iteration, the input and output correction
processes occur in sequence. In the input correction pro­
cess (Fig. 6d), θi(n)(k i) is optimized to maximize the total
intensity of the reconstructed target image. In the same
way, θo(n)(k o) is identified in the output correction process (Fig. 6e). The correction process mainly involves the
SS waves, with the MS waves having little role because
they are taken at different angles of illumination and are
uncorrelated with respect to one another. As the iteration continues, the contribution of the correlated SS
waves to the phase correction increases, while that of the
uncorrelated MS waves remains the same. Therefore, the
maximization of total intensity is almost exclusively due
to the aberration correction of the SS waves. After applying the accumulated phase corrections, an aberration-​
corrected target image is obtained by summing the
angular spectra for all of the illuminations, that is,
E CLASS(Δk) = ∑ k i E(k o, k i ; τ0 ). In short, the multi-​angular
illuminations used in the k-​space reflection matrix offer
a way to eliminate the influence of MS waves. In comparison with an image taken before aberration correction
(Fig. 6f), a signal in an aberration-​corrected CLASS image
(Fig. 6g) was increased more than 20-fold126. The finest
structures in the resolution target are also clearly visible,
confirming the recovery of near-​diffraction-limited spatial resolution. The input and output phase-​correction
maps identified independently by the CLASS algorithm
show high-order aberration modes that ordinary AO
microscopy cannot (Fig. 6h,i).
In initial studies, a drawback of CLASS microscopy
was the slow acquisition rate for recording the time-​
gated reflection matrix owing to the use of a SLM for
multi-​angular illuminations. To resolve this issue, adaptive optical synchronous angular scanning microscopy
(AO-​SASM)127 was proposed, in which a pair of scanning mirrors is used to increase the scanning speed up
to 10,000 modes per second. The high-​speed recording
of R(τ0) enabled aberration-​free imaging of a living larval zebrafish over the entire volume of the hindbrain.
Coherence imaging of neural fibres before aberration
correction at a depth of 160 μm, corresponding to conventional en-​face optical coherence tomography (OCT)
imaging, produces a blurred image (Fig. 6j) in which the
fine neural fibre structures are not well resolved because
the zebrafish has many organs that are optically heterogeneous. Indeed, the sample-​induced aberrations are so
pronounced that they vary from one position to another.
www.nature.com/natrevphys
Reviews
a
Incidence BS2
wave
BS1
SLD
j
x
OL
k
π
z
Phase (rad)
y
Reference
wave
Reflected
wave
Sample
PZT
20 μm
DG
ro
0.4
0.0
ri
θi(n) (ki)
c
E(ko,ki;τ0)
Eo(ko)
0
–2
e
Ei* (ki)
–π
CCD
–1
ki
20 μm
–3
m
n
Dorsal
Posterior
70
θo(n) (ko)
Ventral
220
Depth (μm)
0.2
BS3
Amplitude (log scale)
0.6
E(ro,ri;τ0)
Amplitude (log scale)
b
First order
ko
SM
d
l
Anterior
50 μm
200 μm
p
o
Target object
Input phase correction
Initial
0.0
20
15
10
5
Final
0
π
4 μm
Phase (rad)
0.5
4 μm
g
Intensity (arb. units)
1.0
Intensity (arb. units)
f
Output phase correction
–π
Fig. 6 | scattering and aberration suppression using a time-​gated
reflection-​matrix approach. a | Experimental set-​up of the collective
accumu­lation of single scattering (CASS) microscope (the closed-​loop accumulation of single scattering (CLASS) method adopts a similar set-​up). Low-​
coherence interferometry is used for time-​gated detection, and off-​axis
digital holographic microscopy is integrated for the complex field recording.
b,c | Time-​gated reflection matrices obtained in position space (E(ro,ri;τ0);
panel b) and wavevector k space (E(k o, k i; τ0); panel c). ri and ro are the input
(illumination) and output (detection) position vectors in the object coordinates, respectively; ki and ko are the corresponding input and output transverse wavevectors, respectively, and τ0 is the flight time. d | The nth correction
of an input pupil aberration. The phase correction θi(n)(k i) is applied to each
column of E(k o, k i; τ0) in panel c. e | The nth correction of an output pupil
aberration. The phase correction θo(n)(k o) is applied to each row of E(k o, k i; τ0).
The iteration is repeated until the total intensity of the retrieved CL ASS
image increases sufficiently to reach a given criterion. f,g | Reconstructed
images from the diagonal elements of E(ro,ri;τ0) before aberration correction
(panel f) and the corresponding aberration-​corrected CLASS image after
five rounds of iterations (panel g). h,i | The accumulated phase-​correction
maps for the sample-​induced aberrations in the illumination path (panel h)
50 μm
and detection path (panel i) after five rounds of iterations. j–l | The time-​gated
reflection matrix was obtained by adaptive optical synchronous angular
scanning microscopy (AO-​SASM) at a depth of 160 μm in the hindbrain of a
zebrafish. Images showing neural fibres reconstructed before (panel j) and
after (panel k) aberration correction. The white dotted boxes in panel j show
segmented areas where the CLASS algorithm was individually applied. The
local aberration maps in the pupil plane (panel l) correspond to the segments
indicated in panel k. m | Imaging configuration (top) and bright-​field image
(bottom) for a 21-days-post-fertilization (dpf) living larval zebrafish. The yellow dotted box indicates the imaging area. n | In vivo volumetric imaging of
the deep hindbrain of a 10-dpf larval zebrafish. For 3D rendering, aberration-​
corrected slice images were retrieved from the consecutive time-​gated
reflection matrices along axial intervals at the length of the coherence gating, then numerical propagation was applied with sufficient sampling along
the depth. o,p | Dorsal-view maximum-​intensity-projection images from
confocal microscopy (panel o) and AO-​SASM (panel p). BS, beam splitter;
CCD, charge-​coupled device; DG, diffraction grating; OL, objective lens;
PZT, piezoelectric transducer; SLD, superluminescent diode; SM, scanning
mirror. Panel a adapted from ref.10, Springer Nature Limited. Panels b–i
adapted from ref.126, CC-BY-4.0. Panels j–p adapted from ref.127, CC-BY-4.0.
To address these position-​dependent aberrations, the
CLASS algorithm was individually applied to each segmented area (Fig. 6j). The combined aberration-​corrected
image and the local aberration maps in the pupil plane
(Fig. 6k,l, respectively) show the fine neural fibres and
spatial complexity of the aberrations. A clear 3D image
of the whole neural network throughout the hindbrain
Nature Reviews | Physics
50 μm
of a 10-days-​post-fertilization zebrafish was obtained by
AO-​SASM (Fig. 6m,n), even though the dark epidermal
layer, complex skin architectures and optically heterogeneous internal organs disturb the light propagation
in the imaging configuration. Only dense fascicles are
visible in the confocal microscopy image in the dorsal
view of the hindbrain (Fig. 6o), whereas anatomical
Reviews
details, including fine neuronal fibres and other subcellular architectures, are visible in the corresponding
AO-​SASM image (Fig. 6p).
Deep optical imaging based on the reflection matrix
has an advantage over conventional AO microscopy in
that it can suppress sample-​induced aberrations without the need for guide stars and under the influence
of strong MS noise. These features enable deep optical
imaging to reach the ideal SS gate limit (Box 1 figure).
These advantages are made possible because the reflection matrix deterministically records both the aberrated
SS waves and MS waves arriving at non-​confocal positions. Compared with computational AO approaches
that rely on full-​field imaging, the reflection-​matrix
approach offers more robust image reconstruction with
respect to pupil aberrations and multiple light scattering.
Conventional computational AO approaches often use a
single 2D complex-​field image for a specific depth128 or
a 3D tomogram for a volume129,130 to correct the aberrations, which is equivalent to the acquisition of the diagonal elements only of the position-​space reflection matrix.
Owing to this limited acquisition, the optimization problem becomes underdetermined or ill posed, especially
for reflection imaging in epi-​detection geometry, for
which both the input and output wavefront aberrations
need to be identified. Indeed, the complete recording
of the input–output response through the reflection
matrix has provided the opportunity to resolve difficulties that cannot be handled using conventional imaging.
However, there is still room to explore the use of the
reflection matrix to solve the inverse problem associated with ∣E TM∣2 MS waves that interact with the target object. Extending the reflection-​matrix approach to
solve shift-​variant aberrations64,131,132 could be one option
for the complete deterministic use of multiple scattering.
The identification of position-​dependent local aberrations (Fig. 6l) is a good starting point as they contain a
small fraction of ∣E TM∣2.
Deep imaging based on reflection-​matrix eigenchannels.
Identifying eigenchannels in the reflection matrix provides a unique opportunity to control the interference of
MS waves. Mathematically, eigenchannels are retrieved
through the singular value decomposition of the reflection matrix into R = UΣV∗, where Σ is a rectangular diagonal matrix with real non-negative singular values, and
V and U are unitary matrices with columns that are the
eigenchannels at the input and output planes, respectively.
V∗ denotes the conjugate transpose of V. Coupling light
to eigenchannels with large singular values leads to constructive interference between the scattered waves133, thus
increasing the reflectance in proportion to the square of
the singular values. Indeed, the eigenchannels with large
singular values guide incident waves to relatively high
reflective target objects embedded within a scattering
medium. This characteristic of eigenchannels has previously been employed in acoustics to detect reflecting
objects embedded within a scattering medium134,135 in a
process known as the decomposition of the time-reversal
operator (abbreviated to DORT in French).
In 2011, the first optical analogue of the DORT
method was reported for selectively focusing light on
gold nanoparticles inside disordered media136. Owing
to multiple light scattering, the particles were invisible
in the reflection image (Fig. 7a). The reflection matrix
R was measured using a continuous-wave laser and
its singular-value distribution obtained (Fig. 7b). Four
large singular values associated with the bright gold nano­
particles were identified and their output eigenchannels
(v1–v4) derived from the columns in V, enabling the individual nanoparticles to be clearly resolved (Fig. 7c). The
images recorded on the transmission side confirmed
that the waves generated through eigenchannel coupling
were strongly focused on the target particles. However,
one technical limitation of this early method based on a
steady-​state reflection matrix is its susceptibility to MS
noise. The envelope of small singular values shown in
Fig. 7b is due to background noise, such as stray reflections and laser fluctuations. The eigenchannels of these
small singular values could be distinguished from those
of target particles in a relatively weak scattering medium.
In general, for a highly scattering medium, MS waves
that have no interaction with the nanoparticles dominate
those that do, which may leave the targets invisible, even
with the use of the DORT method.
To enhance the sensitivity in the presence of strong
MS noise, a time-​gated reflection matrix R(τ0) can be
considered10,137, in which a large fraction of background
MS noise is eliminated by time gating (Fig. 7d,e). This
approach was used to demonstrate the extraction of
time-​gated reflection eigenchannels associated with
target objects and the enhancement of light energy
delivered to the target138. Unlike the wave trajectories
for the uncontrolled incident wave (Fig. 7f), most of the
waves coupled through the eigenchannels were guided
to the target, which increased reflection from the target (Fig. 7g). This observation verifies that eigenchannel
coupling controls ∣E TM∣2 MS waves that interact with the
target such that they are focused on the target.
In another method for deep optical imaging, referred
to as smart OCT, eigenchannels of a time-​gated reflection matrix are used for image reconstruction139. For
ZnO particles covered by a 12.25ls-thick scattering
layer, R(τ0) was obtained in the position basis (Fig. 7h).
To reveal the target objects, an MS filter was devised to
remove matrix elements far from the diagonal (Fig. 7i).
The en face OCT image obtained from the diagonal elements of R(τ0) shows only speckle noise (Fig. 7j), whereas
the three particles were clearly resolved in the image
reconstructed using the first three eigenchannels of the
filtered matrix (Fig. 7k). To demonstrate the feasibility of
imaging an extended target, a USAF target behind thick
biological tissue was tested. Compared with the conventional en face OCT image (Fig. 7l), the three bars in the
image reconstructed using 250 eigenchannels (Fig. 7m)
are more clearly resolved.
The image reconstruction techniques based on
eigenchannels discussed so far have proved to be powerful tools for ultra-​deep optical imaging. MS waves
that interact with target objects are deterministically
used in image reconstruction. Therefore, the imaging
depth can exceed that of methods based on SS waves,
such as confocal microscopy, multiphoton micro­
scopy and OCT. Theoretically, the imaging depth of
www.nature.com/natrevphys
Reviews
the eigenchannel-​based methods is expected to reach
22ls compared with 12ls for conventional OCT11,139.
Moreover, these methods also have a better spatial
resolution than those of fully stochastic approaches.
However, neither the input nor output eigenchannels can
represent the real electric field at the plane of the object.
Furthermore, image reconstruction does not keep track
of the momentum change induced by the target object.
Therefore, the spatial resolution is lower than the optical
diffraction limit, especially in the presence of sample-​
induced aberrations. Future research is expected to
0.04
0.02
0.00
1
0
1
0
2
2
2
1
3
Normalized singular values
3
4
5
0.8
0.2
5 μm
0.9
10 μm
xout
g
Random input
Eigenchannel coupling
30
20
|U3 · V3|
+
=
0.3
10 μm
0.2
0.1
x
0.0
m
80
60
40
10 μm
10
20
0
0
Fig. 7 | Eigenchannels of a reflection matrix and their use in imaging.
a–c | Selective focusing on gold nanoparticles through an aberrating layer
using the eigenchannels of the steady-​state reflection matrix. The gold
nanoparticles are not visible in the reflection image obtained with plane-​
wave illumination (panel a). In the singular-​value distribution of the measured
reflection matrix (panel b), there are a few high singular values. The reflection
images obtained using the output eigenchannels of the four largest singular
values (panel c) reveal the presence of the gold nanoparticles. d–g | Time-​
gated reflection eigenchannels. Panel d shows representative trajectories of
multiple-​scattered waves with different flight times (labelled 1–4) within a
scattering medium. Panel e shows the corresponding intensity (I) profile of
backscattered waves as a function of the time of flight (τ). By coupling light
to the time-​gated reflection eigenchannels at the flight time indicated as (3),
the intensity at the selected time-​gating window increases (the red
curve). The blue dotted curve is the intensity profile for the uncontrolled
wave. Images of the wave propagation calculated by the finite-​difference
time-​domain method (where Zs is the depth of the target object) show
Nature Reviews | Physics
0.0
l
τ (ps)
Target
0.6
(4)
|U2 · V2|
+
0.3
(2)
Zs
|U1 · V1|
(3)
(1)
x
k
ri = ro
Zs
Target
z
j
Intensity (arb. units)
Random input
Eigenchannel
0.6
0.4
0.2
0.0
–0.2
–0.4
–0.6
ri
Intensity (arb. units)
e I(τ)
(2)
f
ro
ri
0.0
(3) (4)
0.6
0.4
0.2
0.0
–0.2
–0.4
–0.6
1.0
4
0.4
(1)
ri
lc
43
10
0.6
d
ro
ri
Noise
20
Intensity (norm.)
Reflection image
c
30
ro
0.06
yout
Intensity (norm.)
0.08
i
h
40
y
Singular values (counts)
b
0.10
Future perspectives
A surge in research related to deep optical imaging has
been witnessed since the first demonstration of focusing
MS waves to a sharp spot using wavefront shaping. Many
approaches for using MS waves have emerged based on
the mesoscopic physics of wave propagation, and those
covered in this Review have provided opportunities
that were not previously conceivable. In particular, the
ro
a
harness the benefits of eigenchannels in controlling MS
waves for high-​resolution optical imaging.
10 μm
the differences in wave propagation for a random input (panel f) and
eigenchannel coupling (panel g). h–k | Smart optical coherence tomography
(OCT) for deep optical imaging. Panel h shows the time-​gated reflection
matrix R(ro,ri) (bottom) measured for a scattering layer covering three ZnO
beads with diameters of 5 μm (top), where ri and ro are the incident-​focusing
and output-​focusing points, respectively. Panel i shows the single-​scattered
matrix RS obtained by applying a multiple-​scattered filter to the raw matrix
R with a virtual pinhole mask with a size (lc) of 5 μm. Panel j shows an en face
OCT image deduced from the diagonal of R. The images reconstructed using
the first three eigenchannels of RS, |Ui·Vi| (panel k, top), yield the image of the
three beads in the presence of a scattering layer (bottom). l,m | Panel l shows
an en face OCT image of a USAF target under 800-μm-​thick rat intestinal
tissue. Panel m shows the corresponding eigenchannel image of the
resolution target obtained from the first 250 eigenchannels of the filtered
matrix. Panels a–c adapted with permission from ref.136, APS. Panels d–g
adapted from ref.138, Springer Nature Limited. Panels h–m adapted with
permission from ref.139, AAAS.
Reviews
deterministic use of MS waves for optimal image formation has raised the prospect of overcoming the imaging depth limit of conventional high-​resolution optical
microscopy. Because high-​resolution optical imaging
relies mainly on SS waves, its depth limit is set by the
efficiency of the single-​scattering gating methods that
distinguish SS signal waves from strong MS noise.
Although this depth limit varies from system to system,
it is typically ~10ls. The deterministic use of MS waves
converts some of the waves considered to be noise into a
signal, thus increasing the effective signal-​to-noise ratio.
This approach will ultimately lead to an imaging depth
that is greater than the single-​scattering gating limit and
one that may even exceed the fundamental limit set by
the dynamic range of the detector, which is ~18–20ls.
At present, however, most studies have been proof-​
of-concept and are subject to assumptions and constraints that are not fully compatible with in vivo
imaging, and/or the improvement relative to conventional imaging has been marginal. Thus, the proposed
methodologies have not yet reached a level at which
MS waves are used fully deterministically to form a
diffraction-​limited optical image under standard imaging conditions. For these newly proposed methodologies to mature, the aforementioned assumptions and
constraints need to be accounted for. Indeed, there are
many ongoing studies that are addressing the limitations
of each modality. Specifically, researchers working on
deep imaging based on memory effects are combining
their efforts to find ways to effectively generate a focus
within a scattering medium and to extend the working
range of the memory effect140,141. Studies are also underway to address the degradation of the TM due to the
bending and twisting of multimode fibres, which will
enable multimode-​fibre ultrathin endoscopes to create images from curved pathways90–93. Optical-​scale
Luker, G. D. & Luker, K. E. Optical imaging: current
applications and future directions. J. Nucl. Med. 49,
1–4 (2008).
2.
Wilt, B. A. et al. Advances in light microscopy for
neuroscience. Annu. Rev. Neurosci. 32, 435–506
(2009).
3.
Hadjipanayis, C. G., Jiang, H., Roberts, D. W. & Yang, L.
Current and future clinical applications for optical
imaging of cancer: from intraoperative surgical
guidance to cancer screening. Semin. Oncol. 38,
109–118 (2011).
4.
Betzig, E. et al. Imaging intracellular fluorescent
proteins at nanometer resolution. Science 313,
1642–1645 (2006).
5.
Rust, M. J., Bates, M. & Zhuang, X. Sub-diffraction-limit
imaging by stochastic optical reconstruction microscopy
(STORM). Nat. Methods 3, 793–796 (2006).
6.
Klar, T. A., Jakobs, S., Dyba, M., Egner, A. & Hell, S. W.
Fluorescence microscopy with diffraction resolution
barrier broken by stimulated emission. Proc. Natl
Acad. Sci. USA 97, 8206–8210 (2000).
7.
Hell, S. W. Far-Field Optical nanoscopy. Science 316,
1153–1158 (2007).
8.
Tyson, R. K. Principles of Adaptive Optics (CRC Press,
2015).
9.
Liu, F., Yoo, K. M. & Alfano, R. R. Transmitted photon
intensity through biological tissues within various
time windows. Opt. Lett. 19, 740–742 (1994).
10. Kang, S. et al. Imaging deep within a scattering medium
using collective accumulation of single-​scattered waves.
Nat. Photonics 9, 253–258 (2015).
The first report of the measurement of a wide-​field
time-​gated reflection matrix.
11. Badon, A., Boccara, A. C., Lerosey, G., Fink, M. &
Aubry, A. Multiple scattering limit in optical microscopy.
Opt. Express 25, 28914–28934 (2017).
1.
imaging using acousto-​optic interactions and the selective detection of single optical speckles is advancing
towards realistic conditions in which the speckle grain
size reaches half of the wavelength121,124. High-​resolution
imaging based on a time-​gated reflection matrix is also
starting to gain control of MS waves in reconstructing
the object image126,127,139. Additionally, improvements
in the sensitivity and speed of cameras and the increase in
the number of pixels and control speed of the SLM will
help to resolve some of the practical issues.
The formation of an ideal image using MS waves is
an underdetermined problem, and advances in computational tools, such as convolutional neural networks,
may help in finding a solution by taking advantage of
prior knowledge from mesoscopic physics. Deep learning based on neural networks has been actively used in
recent years for image denoising, spatial and spectral
deconvolution, super-​resolution imaging, scattering
noise reduction, image registration and tomographic
reconstruction142–144. In the context of MS waves, neural networks have been used to train the input–output
response for scattering media and multimode optical
fibres145,146 and to reconstruct refractive index maps
for spatially confined objects from the MS waves of the
objects themselves147,148. It remains to be seen whether
these approaches can be extended to selectively train
MS waves that interact with the target object to form
a near-​diffraction-limited image under the strong MS
noise induced by a large-​volume scattering medium. We
expect that these efforts will improve the practicality of
deep optical imaging in complex scattering media and
make it possible to investigate physical and biological
systems in a more detailed but less invasive manner
than before.
Published online xx xx xxxx
12. Dunsby, C. & French, P. M. W. Techniques for depthresolved imaging through turbid media including
coherence-​gated imaging. J. Phys. D. Appl. Phys. 36,
R207–R227 (2003).
An early study on the development of singlescattering gating methods for deep optical imaging.
13. Minsky, M. Microscopy apparatus. US Patent
3013467 (1961).
14. Huang, D. et al. Optical coherence tomography.
Science. 254, 1178–1181 (1991).
15. Bouma, B. E. & Tearney, G. J. (eds) Handbook of
Optical Coherence Tomography (Marcel Dekker,
2002).
16. Pawley, J. (ed.) Handbook of Biological Confocal
Microscopy (Springer, 2010).
17. Yao, G. & Wang, L. V. Theoretical and experimental
studies of ultrasound-​modulated optical tomography
in biological tissue. Appl. Opt. 39, 659–664 (2000).
18. Wang, L. V. Ultrasound-​mediated biophotonic imaging:
a review of acousto-​optical tomography and photo-​
acoustic tomography. Dis. Markers 19, 123–138
(2004).
19. Izatt, J. A., Hee, M. R., Owen, G. M., Swanson, E. A.
& Fujimoto, J. G. Optical coherence microscopy in
scattering media. Opt. Lett. 19, 590–592 (1994).
20. Beaurepaire, E., Boccara, A. C., Lebec, M., Blanchot, L.
& Saint-​Jalmes, H. Full-​field optical coherence
microscopy. Opt. Lett. 23, 244–246 (1998).
21. Oh, W. Y. et al. Ultrahigh-resolution full-​field optical
coherence microscopy using InGaAs camera.
Opt. Express 14, 726–735 (2006).
22. Denk, W., Strickler, J. H. & Webb, W. W. Two-​photon
laser scanning fluorescence microscopy. Science 248,
73–76 (1990).
23. Hell, S. W. et al. Three-​photon excitation in fluorescence
microscopy. J. Biomed. Opt. 1, 71–74 (1996).
24. Zipfel, W. R. et al. Live tissue intrinsic emission
microscopy using multiphoton-​excited native
fluorescence and second harmonic generation.
Proc. Natl Acad. Sci. USA 100, 7075–7080 (2003).
25. Barad, Y., Eisenberg, H., Horowitz, M. & Silberberg, Y.
Nonlinear scanning laser microscopy by third
harmonic generation. Appl. Phys. Lett. 70, 922–924
(1997).
26. Papagiakoumou, E. et al. Functional patterned
multiphoton excitation deep inside scattering tissue.
Nat. Photonics 7, 274–278 (2013).
27. Rowlands, C. J. et al. Wide-​field three-​photon
excitation in biological samples. Light. Sci. Appl. 6,
e16255 (2016).
28. Adrià, E.-M. et al. Wide-​field multiphoton imaging
through scattering media without correction. Sci. Adv.
4, eaau1338 (2018).
29. Booth, M. J. Adaptive optical microscopy: the ongoing
quest for a perfect image. Light. Sci. Appl. 3, e165
(2014).
30. Jang, M., Ko, H., Lee, W. K., Lee, J.-S. & Choi, W.
Coherent space-​gated microscopy: a step towards
deep-​tissue phase imaging of biological cells.
Preprint at arXiv https://arxiv.org/abs/1811.02755
(2018).
31. Durduran, T. et al. Diffuse optical measurement of
blood flow, blood oxygenation, and metabolism in a
human brain during sensorimotor cortex activation.
Opt. Lett. 29, 1766–1768 (2004).
32. Ntziachristos, V., Yodh, A. G., Schnall, M. & Chance, B.
Concurrent MRI and diffuse optical tomography of
breast after indocyanine green enhancement.
Proc. Natl Acad. Sci. USA 97, 2767–2772 (2000).
33. Wang, L. V. & Hu, S. Photoacoustic tomography:
in vivo imaging from organelles to organs. Science
335, 1458–1462 (2012).
www.nature.com/natrevphys
Reviews
34. Vellekoop, I. M. & Mosk, A. P. Focusing coherent light
through opaque strongly scattering media. Opt. Lett.
32, 2309–2311 (2007).
The first demonstration of optical-​wave focusing
through a scattering layer.
35. Yaqoob, Z., Psaltis, D., Feld, M. S. & Yang, C.
Optical phase conjugation for turbidity suppression
in biological samples. Nat. Photonics 2, 110–115
(2008).
36. Dorokhov, O. N. On the coexistence of localized and
extended electronic states in the metallic phase.
Solid. State Commun. 51, 381–384 (1984).
37. Feng, S., Kane, C., Lee, P. A. & Stone, A. D.
Correlations and fluctuations of coherent wave
transmission through disordered media. Phys. Rev.
Lett. 61, 834–837 (1988).
38. Freund, I., Rosenbluh, M. & Feng, S. Memory effects
in propagation of optical waves through disordered
media. Phys. Rev. Lett. 61, 2328–2331 (1988).
39. Beenakker, C. W. J. Random-​matrix theory of quantum
transport. Rev. Mod. Phys. 69, 731–808 (1997).
40. Mosk, A. P., Lagendijk, A., Lerosey, G. & Fink, M.
Controlling waves in space and time for imaging and
focusing in complex media. Nat. Photonics 6,
283–292 (2012).
41. Rotter, S. & Gigan, S. Light fields in complex media:
mesoscopic scattering meets wave control. Rev. Mod.
Phys. 89, 015005 (2017).
42. Fink, M. Time reversal of ultrasonic fields. I. Basic
principles. IEEE Trans. Ultrason. Ferroelectr. Freq.
Control. 39, 555–566 (1992).
43. Derode, A., Roux, P. & Fink, M. Robust acoustic
time reversal with high-​order multiple scattering.
Phys. Rev. Lett. 75, 4206–4209 (1995).
44. Fink, M. Time reversed acoustics. Phys. Today 50,
34–40 (1997).
45. Lerosey, G. et al. Time reversal of electromagnetic
waves. Phys. Rev. Lett. 92, 193904 (2004).
46. Lerosey, G., de Rosny, J., Tourin, A., Derode, A. &
Fink, M. Time reversal of wideband microwaves.
Appl. Phys. Lett. 88, 154101 (2006).
47. Vellekoop, I. M. & Mosk, A. P. Universal optimal
transmission of light through disordered materials.
Phys. Rev. Lett. 101, 120601 (2008).
48. Kim, M., Choi, W., Yoon, C., Kim, G. H. & Choi, W.
Relation between transmission eigenchannels and
single-​channel optimizing modes in a disordered
medium. Opt. Lett. 38, 2994–2996 (2013).
49. Katz, O., Small, E. & Silberberg, Y. Looking around
corners and through thin turbid layers in real time
with scattered incoherent light. Nat. Photonics 6,
549–553 (2012).
50. Judkewitz, B., Horstmeyer, R., Vellekoop, I. M.,
Papadopoulos, I. N. & Yang, C. Translation correlations
in anisotropically scattering media. Nat. Phys. 11,
684–689 (2015).
51. Vellekoop, I. M., Lagendijk, A. & Mosk, A. P. Exploiting
disorder for perfect focusing. Nat. Photonics 4,
320–322 (2010).
52. van Putten, E. G. et al. Scattering lens resolves sub100 nm structures with visible light. Phys. Rev. Lett.
106, 193905 (2011).
53. Park, J.-H. et al. Subwavelength light focusing using
random nanoparticles. Nat. Photonics 7, 454–458
(2013).
54. Choi, W. et al. Control of randomly scattered surface
plasmon polaritons for multiple-​input and multiple-​
output plasmonic switching devices. Nat. Commun. 8,
14636 (2017).
55. Gjonaj, B. et al. Focusing and scanning microscopy
with propagating surface plasmons. Phys. Rev. Lett.
110, 266804 (2013).
56. Seo, E. et al. Far-​field control of focusing plasmonic
waves through disordered nanoholes. Opt. Lett. 39,
5838–5841 (2014).
57. Vellekoop, I. M. Feedback-​based wavefront shaping.
Opt. Express 23, 12189–12206 (2015).
58. Wang, D. et al. Focusing through dynamic tissue with
millisecond digital optical phase conjugation. Optica
2, 728–735 (2015).
59. Conkey, D. B., Caravaca-​Aguirre, M. A. & Piestun, R.
High-​speed scattering medium characterization with
application to focusing light through turbid media.
Opt. Express 20, 1733–1740 (2012).
60. Blochet, B., Bourdieu, L. & Gigan, S. Focusing light
through dynamical samples using fast continuous
wavefront optimization. Opt. Lett. 42, 4994–4997
(2017).
61. Liu, Y., Ma, C., Shen, Y., Shi, J. & Wang, L. V. Focusing
light inside dynamic scattering media with millisecond
digital optical phase conjugation. Optica 4, 280–288
(2017).
Nature Reviews | Physics
62. Feldkhun, D., Tzang, O., Wagner, K. H. & Piestun, R.
Focusing and scanning through scattering media in
microseconds. Optica 6, 72–75 (2019).
63. Ji, N., Milkie, D. E. & Betzig, E. Adaptive optics via
pupil segmentation for high-​resolution imaging in
biological tissues. Nat. Methods 7, 141–147 (2009).
64. Park, J.-H., Sun, W. & Cui, M. High-​resolution in vivo
imaging of mouse brain through the intact skull.
Proc. Natl Acad. Sci. USA 112, 9236–9241 (2015).
65. Frostig, H. et al. Focusing light by wavefront shaping
through disorder and nonlinearity. Optica 4,
1073–1079 (2017).
66. Ruan, H. et al. Focusing light inside scattering media
with magnetic-​particle-guided wavefront shaping.
Optica 4, 1337–1343 (2017).
67. Florentin, R. et al. Shaping the light amplified in a
multimode fiber. Light Sci. Appl. 6, e16208 (2017).
68. Shen, Y., Liu, Y., Ma, C. & Wang, L. V. Focusing light
through biological tissue and tissue-​mimicking
phantoms up to 9.6 cm in thickness with digital
optical phase conjugation. J. Biomed. Opt. 21,
085001 (2016).
69. Popoff, S., Lerosey, G., Fink, M., Boccara, A. C.
& Gigan, S. Image transmission through an opaque
material. Nat. Commun. 1, 81 (2010).
70. Popoff, S. M. et al. Measuring the transmission matrix
in optics: an approach to the study and control of light
propagation in disordered media. Phys. Rev. Lett.
104, 100601 (2010).
The first report of the measurement of a transmission
matrix in optics.
71. Choi, Y. et al. Overcoming the diffraction limit using
multiple light scattering in a highly disordered
medium. Phys. Rev. Lett. 107, 023902 (2011).
72. Choi, Y. et al. Synthetic aperture microscopy for high
resolution imaging through a turbid medium. Opt. Lett.
36, 4263–4265 (2011).
73. van Putten, E. & Mosk, A. The information age in
optics: measuring the transmission matrix. Physics 3,
22 (2010).
74. Choi, Y. et al. Scanner-​free and wide-​field endoscopic
imaging by using a single multimode optical fiber.
Phys. Rev. Lett. 109, 203901 (2012).
75. Choi, Y., Yoon, C., Kim, M., Choi, W. & Choi, W.
Optical imaging with the use of a scattering lens.
IEEE J. Sel. Top. Quantum Electron. 20, 6800213
(2014).
76. Yoon, C. et al. Removal of back-​reflection noise
at ultrathin imaging probes by the single-​core
illumination and wide-​field detection. Sci. Rep. 7,
6524 (2017).
77. Bianchi, S. & Di Leonardo, R. A multi-​mode fiber
probe for holographic micromanipulation and
microscopy. Lab. Chip 12, 635–639 (2012).
The first report of imaging through a multimode
fibre.
78. Bianchi, S. et al. Focusing and imaging with increased
numerical apertures through multimode fibers with
micro-​fabricated optics. Opt. Lett. 38, 4935–4938
(2013).
79. Čižmár, T. & Dholakia, K. Exploiting multimode
waveguides for pure fibre-​based imaging. Nat. Commun.
3, 1027 (2012).
80. Čižmár, T. & Dholakia, K. Shaping the light
transmission through a multimode optical fibre:
complex transformation analysis and applications
in biophotonics. Opt. Express 19, 18871–18884
(2011).
81. Kim, D. et al. Toward a miniature endomicroscope:
pixelation-​free and diffraction-​limited imaging through
a fiber bundle. Opt. Lett. 39, 1921–1924 (2014).
82. Ohayon, S., Caravaca-​Aguirre, A., Piestun, R. &
DiCarlo, J. J. Minimally invasive multimode optical
fiber microendoscope for deep brain fluorescence
imaging. Biomed. Opt. Express 9, 1492–1509
(2018).
83. Caravaca-​Aguirre, A. M. & Piestun, R. Single
multimode fiber endoscope. Opt. Express 25,
1656–1665 (2017).
84. Turtaev, S. et al. High-​fidelity multimode fibre-​based
endoscopy for deep brain in vivo imaging. Light. Sci.
Appl. 7, 92 (2018).
85. Loterie, D. et al. Digital confocal microscopy through
a multimode fiber. Opt. Express 23, 23845–23858
(2015).
86. Morales-​Delgado, E. E., Psaltis, D. & Moser, C.
Two-photon imaging through a multimode fiber.
Opt. Express 23, 32158–32170 (2015).
87. Drémeau, A. et al. Reference-​less measurement of
the transmission matrix of a highly scattering
material using a DMD and phase retrieval techniques.
Opt. Express 23, 11898–11911 (2015).
88. del Hougne, P., Rajaei, B., Daudet, L. & Lerosey, G.
Intensity-​only measurement of partially uncontrollable
transmission matrix: demonstration with wave-​field
shaping in a microwave cavity. Opt. Express 24,
18631–18641 (2016).
89. Porat, A. et al. Widefield lensless imaging through
a fiber bundle via speckle correlations. Opt. Express
24, 16835–16855 (2016).
90. Farahi, S., Ziegler, D., Papadopoulos, I. N., Psaltis, D.
& Moser, C. Dynamic bending compensation while
focusing through a multimode fiber. Opt. Express 21,
22504–22514 (2013).
91. Loterie, D., Psaltis, D. & Moser, C. Bend translation
in multimode fiber imaging. Opt. Express 25,
6263–6273 (2017).
92. Plöschner, M., Tyc, T. & Čižmár, T. Seeing through
chaos in multimode fibres. Nat. Photonics 9,
529–535 (2015).
93. Boonzajer Flaes, D. E. et al. Robustness of light-​
transport processes to bending deformations in
graded-​index multimode waveguides. Phys. Rev. Lett.
120, 233901 (2018).
94. Freund, I. Looking through walls and around corners.
Phys. A 168, 49–65 (1990).
95. Akkermans, E. & Montambaux, G. Mesoscopic Physics
of Electrons and Photons (Cambridge Univ. Press, 2007).
96. Osnabrugge, G., Horstmeyer, R., Papadopoulos, I. N.,
Judkewitz, B. & Vellekoop, I. M. The generalized
optical memory effect. Optica 4, 886–892 (2017).
97. Bertolotti, J. et al. Non-​invasive imaging through
opaque scattering layers. Nature 491, 232–234
(2012).
The first study to use the optical memory effect for
epi-​fluorescence imaging.
98. Katz, O., Heidmann, P., Fink, M. & Gigan, S.
Non-invasive single-​shot imaging through scattering
layers and around corners via speckle correlations.
Nat. Photonics 8, 784–790 (2014).
99. Yang, X., Pu, Y. & Psaltis, D. Imaging blood cells through
scattering biological tissue using speckle scanning
microscopy. Opt. Express 22, 3405–3413 (2014).
100. Schott, S., Bertolotti, J., Léger, J.-F., Bourdieu, L.
& Gigan, S. Characterization of the angular memory
effect of scattered light in biological tissues.
Opt. Express 23, 13505–13516 (2015).
101. Amitonova, L. V., Mosk, A. P. & Pinkse, P. W. H.
Rotational memory effect of a multimode fiber.
Opt. Express 23, 20569–20575 (2015).
102. Stasio, N., Moser, C. & Psaltis, D. Calibration-​free
imaging through a multicore fiber using speckle
scanning microscopy. Opt. Lett. 41, 3078–3081
(2016).
103. Fienup, J. R. Phase retrieval algorithms: a comparison.
Appl. Opt. 21, 2758–2769 (1982).
104. Fienup, J. R. Reconstruction of an object from the
modulus of its Fourier transform. Opt. Lett. 3, 27–29
(2008).
105. Papadopoulos, I. N., Jouhanneau, J. S., Poulet, J. F. A.
& Judkewitz, B. Scattering compensation by focus
scanning holographic aberration probing (F-SHARP).
Nat. Photonics 11, 116–123 (2017).
106. Feng, G. et al. Imaging neuronal subsets in transgenic
mice expressing multiple spectral variants of GFP.
Neuron 28, 41–51 (2000).
107. Szabo, T. L. Diagnostic Ultrasound Imaging: Inside
Out (Elsevier, 2004).
108. Elson, D. S., Li, R., Dunsby, C., Eckersley, R. &
Tang, M. X. Ultrasound-​mediated optical tomography:
a review of current methods. Interface Focus. 1,
632–648 (2011).
109. Yao, G. & Wang, L. V. in Biomedical Optical
Spectroscopy and Diagnostics (Optical Society of
America, 2000).
110. Wang, L. V. Prospects of photoacoustic tomography.
Med. Phys. 35, 5758–5767 (2008).
111. Wang, L. V. (ed.) Photoacoustic Imaging and
Spectroscopy (CRC press, 2017).
112. Culjat, M. O., Goldenberg, D., Tewari, P. & Singh, R. S.
A review of tissue substitutes for ultrasound imaging.
Ultrasound Med. Biol. 36, 861–873 (2010).
113. Horstmeyer, R., Ruan, H. & Yang, C. Guidestar-assisted
wavefront-​shaping methods for focusing light into
biological tissue. Nat. Photonics 9, 563–571 (2015).
114. Jang, M., Sentenac, A. & Yang, C. Optical phase
conjugation (OPC)-assisted isotropic focusing.
Opt. Express 21, 8781–8792 (2013).
115. Xu, X., Liu, H. & Wang, L. V. Time-​reversed ultrasonically
encoded optical focusing into scattering media.
Nat. Photonics 5, 154–157 (2011).
The first study to combine acousto-​optic
modulation and optical phase conjugation for
optical imaging in a scattering medium.
Reviews
116. Wang, Y. M., Judkewitz, B., Dimarzio, C. A. & Yang, C.
Deep-​tissue focal fluorescence imaging with digitally
time-​reversed ultrasound-​encoded light. Nat. Commun.
3, 928 (2012).
117. Si, K., Fiolka, R. & Cui, M. Fluorescence imaging
beyond the ballistic regime by ultrasound-​pulseguided digital phase conjugation. Nat. Photonics 6,
657–661 (2012).
118. Si, K., Fiolka, R. & Cui, M. Breaking the spatial
resolution barrier via iterative sound-​light interaction
in deep tissue microscopy. Sci. Rep. 2, 748 (2012).
119. Ruan, H., Jang, M., Judkewitz, B. & Yang, C. Iterative
time-​reversed ultrasonically encoded light focusing in
backscattering mode. Sci. Rep. 4, 7156 (2014).
120. Judkewitz, B., Wang, Y. M., Horstmeyer, R., Mathy, A.
& Yang, C. Speckle-​scale focusing in the diffusive
regime with time reversal of variance-​encoded light
(TROVE). Nat. Photonics 7, 300–305 (2013).
121. Katz, O., Ramaz, F., Gigan, S. & Fink, M. Controlling
light in complex media beyond the acoustic diffraction-​
limit using the acousto-​optic transmission matrix.
Nat. Commun. 10, 717 (2019).
122. Izatt, J. A., Swanson, E. A., Fujimoto, J. G., Hee, M. R.
& Owen, G. M. Optical coherence microscopy in
scattering media. Opt. Lett. 19, 590–592 (2008).
123. Chaigne, T. et al. Controlling light in scattering media
non-​invasively using the photoacoustic transmission
matrix. Nat. Photonics 8, 58–64 (2014).
124. Lai, P., Wang, L., Tay, J. W. & Wang, L. V.
Photoacoustically guided wavefront shaping for
enhanced optical focusing in scattering media.
Nat. Photonics 9, 126–132 (2015).
125. Conkey, D. B. et al. Super-​resolution photoacoustic
imaging through a scattering wall. Nat. Commun. 6,
7902 (2015).
126. Kang, S. et al. High-​resolution adaptive optical
imaging within thick scattering media using closed-​
loop accumulation of single scattering. Nat. Commun.
8, 2157 (2017).
A report of a reflection matrix analysis algorithm
to correct aberrations without using guide stars.
127. Kim, M. et al. Label-​free neuroimaging in vivo using
synchronous angular scanning microscopy with singlescattering accumulation algorithm. Nat. Commun. 10,
3152 (2019).
128. Colomb, T. et al. Numerical parametric lens for
shifting, magnification, and complete aberration
compensation in digital holographic microscopy.
J. Opt. Soc. Am. A 23, 3177–3190 (2006).
129. Boppart, S. A. et al. Computational adaptive optics
for broadband optical interferometric tomography
of biological tissue. Proc. Natl Acad. Sci. USA 109,
933505 (2012).
130. Shemonski, N. D. et al. Computational high-​resolution
optical imaging of the living human retina.
Nat. Photonics 9, 440–443 (2015).
131. Zheng, G., Ou, X., Horstmeyer, R. & Yang, C.
Characterization of spatially varying aberrations
for wide field-​of-view microscopy. Opt. Express 21,
15131–15143 (2013).
132. Pozzi, P. et al. Anisoplanatic adaptive optics in
parallelized laser scanning microscopy. Preprint at
arXiv https://arxiv.org/abs/1809.07529 (2018).
133. Kim, M. et al. Maximal energy transport through
disordered media with the implementation of
transmission eigenchannels. Nat. Photonics 6,
581–585 (2012).
134. Prada, C. & Fink, M. Eigenmodes of the time
reversal operator: a solution to selective focusing in
multiple-target media. Wave Motion 20, 151–163
(1994).
135. Aubry, A. & Derode, A. Random matrix theory
applied to acoustic backscattering and imaging
in complex media. Phys. Rev. Lett. 102, 084301
(2009).
136. Popoff, S. M. et al. Exploiting the time-​reversal operator
for adaptive optics, selective focusing, and scattering
pattern analysis. Phys. Rev. Lett. 107, 263901 (2011).
137. Choi, Y. et al. Measurement of the time-​resolved
reflection matrix for enhancing light energy delivery
into a scattering medium. Phys. Rev. Lett. 111,
243901 (2013).
138. Jeong, S. et al. Focusing of light energy inside a
scattering medium by controlling the time-​gated
multiple light scattering. Nat. Photonics 12, 277–283
(2018).
139. Badon, A. et al. Smart optical coherence tomography
for ultra-​deep imaging through highly scattering media.
Sci. Adv. 2, e1600370 (2016).
A study analysing the time-​gated reflection
eigenchannels for deep optical imaging.
140. Kadobianskyi, M., Papadopoulos, I. N., Chaigne, T.,
Horstmeyer, R. & Judkewitz, B. Scattering correlations
of time-​gated light. Optica 5, 389–394 (2018).
141. Osnabrugge, G., Amitonova, L. V. & Vellekoop, I. M.
Blind focusing through strongly scattering media
using wavefront shaping with nonlinear feedback.
Opt. Express 27, 11673–11688 (2019).
142. McCann, M. T., Jin, K. H. & Unser, M. Convolutional
neural networks for inverse problems in imaging:
a review. IEEE Signal. Process. Mag. 34, 85–95
(2017).
143. Belthangady, C. & Royer, L. A. Applications, promises,
and pitfalls of deep learning for fluorescence image
reconstruction. Nat. Methods 16, 1215–1225
(2019).
144. Barbastathis, G., Ozcan, A. & Situ, G. On the use of
deep learning for computational imaging. Optica 6,
921–943 (2019).
145. Turpin, A., Vishniakou, I. & Seelig, J.d. Light scattering
control in transmission and reflection with neural
networks. Opt. Express 26, 30911–30929 (2018).
146. Rahmani, B., Loterie, D., Konstantinou, G., Psaltis, D.
& Moser, C. Multimode optical fiber transmission
with a deep learning network. Light. Sci. Appl. 7, 69
(2018).
147. Kamilov, U. S. et al. Learning approach to optical
tomography. Optica 2, 517–522 (2015).
148. Sun, Y., Xia, Z. & Kamilov, U. S. Efficient and accurate
inversion of multiple scattering with deep learning.
Opt. Express 26, 14678–14688 (2018).
149. Ntziachristos, V. Going deeper than microscopy:
the optical imaging. Nat. Methods 7, 603–614
(2010).
Acknowledgements
This research was supported by the Institute for Basic Science
(grant no. IBS-​R023-D1), the National Research Foundation
of Korea (grant nos. NRF-2019R1C1C1008175 and NRF2016R1A6A3A11936389), and the Catholic Medical Center
Research Foundation in the programme year of 2018.
Author contributions
The authors contributed equally to all aspects of the article.
Competing interests
The authors declare no competing interests.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.
Supplementary information
Supplementary information is available for this paper at
https://doi.org/10.1038/s42254-019-0143-2.
© Springer Nature Limited 2020
www.nature.com/natrevphys