Multifocal Multiphoton Microscopy for
Neurobiological Imaging
By
Jae Won Cha
B.S., Mechanical and Aerospace Engineering
Seoul National University (2003)
S.M., Mechanical Engineering
Massachusetts Institute of Technology (2007)
Submitted to the Department of Mechanical Engineering
in Partial Fulfillment of the Requirements for the Degree of
ARC*QVEU
MASSACHUSETTS INS1fT7WE
OF TECHNOLOGY
MAY 0 8 2014
Doctor of Philosophy
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
L
LIBRARIES
February 2014
C 2014 Massachusetts Institute of Technology. All rights reserved
Signature of Author:
Department ofMechanical Engineering
15, 2014
Certified by:
Peter T. C. So
Professor of Mechanical Engineering and Biological Engineering
Thesiupervisor.
Accepted by:
David E. Hardt
Chairman of the Departmental Committee for Graduate Students
Multifocal Multiphoton Microscopy for
Neurobiological Imaging
By
Jae Won Cha
Submitted to the Department of Mechanical Engineering
on Jan 15, 2014 in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy in
Mechanical Engineering
Abstract
Observing neuronal structures and monitoring changes in synaptic connectivity with respect to time
have a significant impact on understanding the basis of structural brain abnormalities and the
development of therapeutics for their correction. Today, multiphoton excitation fluorescence
microscopy is the method of choice for in vivo neuronal imaging with its inherent 3D resolution,
minimal photo-damage, and excellent penetration depth. The study of neuronal interactions on
dendritic arbor remodeling often requires large volume imaging demanding fast imaging speed. One
of the methods to improve imaging speed is multifocal multiphoton microscopy (MMM) that
parallelizes imaging process with multiple excitation foci. Early MMM had very limited imaging
depth due to signal-to-noise ratio (SNR) degradation resulting from the scattering of emission
photons in highly turbid biological specimens. The development of descanned MMM with
multianode photomultiplier tube has partly alleviated this problem, but it still suffers from greater
signal loss and the presence of image artifacts compared with conventional single focus
multiphoton microscopes. In this thesis, adaptive optics compensation, image post processing for
emission photon reassignment, and a novel non-descanned MMM have been investigated for SNR
improvement. In addition, spectral resolved MMM has been developed for simultaneous fast
imaging and spectral detection.
Thesis Supervisor: Peter T. C. So
Title: Professor of Mechanical Engineering and Biological Engineering
2
Acknowledgement
It was pretty long time to study at MIT and finish my Ph.D. work. During my nine-year Boston
life,
I am deeply indebted to many people for academic research as well as personal life.
First of all, I would like to thank my thesis advisor, Prof. Peter So for his generosity, guidance,
encouragement, tolerance, and trust. He has given me limitless opportunities even from when I was
a complete novice as a master candidate. In addition, whenever I faced challenging situations, he
was always supportive and provided essential advice. Not only his academic advice, but also his
thoughtful consideration was indispensable for my successful MIT life. I also would like to thank
Prof. Elly Nedivi for her support, advice, generosity, patience, and trust. She was another greatest
support for my study and life at MIT. I also would like to express my gratitude to Prof. Roger
Kamm for his key advice, guidance and generous support for successful research.
I also thank all the former and current So Lab members, especially Kihean Kim, Daekeun Kim,
Hyuksang Kwon, Euihean Chung, Maxine Jonas, Judith Su, Heejin Choi, Dimitris Tzeranis, YangHyo Kim, Christof Buehler, Hayden Huang, Timothy Ragan, Karsten Bahlmann, Christopher
Rowlands, Elijah Yew, Vijay Raj Singh, Yun-Ho Jang, Barry Masters, and Sossy Megerdichian for
their academic advice, numerous help, and warm-hearted support.
I also would like to thank all the people who have been with me in my Boston life, Korean Graduate
Student Association members in Mechanical Engineering, SNU OB Choir members in Boston,
many MIT friends, and especially Jooeun Ahn and his family for priceless friendship.
I cannot imagine how I can express my appreciation to my parents and parents-in-law. I am deeply
grateful to Sung-Chan Cha, Soon-Ja Koo, Je-Kyung Ryu, and Ja-Kyung Choi.
Especially I thank my wife, Min Young Yoo for her constant support and endless love, and our
children, Yuna, Jihyo, and Seehun with all my heart.
I would like to acknowledge my funding sources and sponsors, NIH ROl EY017656, SingaporeMIT Alliance 2, Singapore MIT Alliance for Research and Technology, and Imagine Optic.
3
Table of Contents
1. Introduction .............................................................................................................
13
1. 1. M ultiphoton microscopy for neurobiological imaging .............................................
13
1. 2. Imaging speed improvement in multiphoton microscopy..........................................
16
1. 3. Signal-to-noise ratio improvement in MMM ........................................................
18
1. 3. 1. Adaptive optics compensation..................................................................
18
1. 3. 2. Emission photon reassignment................................................................
18
1. 3. 3. Non-descanned M M M ...........................................................................
19
20
1. 4. Spectral resolved NIMM ...................................................................................
2. A Shack-Hartmann Wavefront Sensor Based Adaptive Optics System for Multiphoton
23
Microscopy ..................................................................................................................
2. 1. Introduction ..................................................................................................
23
2. 2. M ethods and Experimental Setup.....................................................................
24
24
2. 2. 1. A large dynamic range deformable mirror ..................................................
2. 2. 2. The detection of the wavefront distortion with a Shack-Hartmann wavefront sensor
..................................................................................................................
. . 25
2. 2. 3. The application confocal detection to select reflected light signal from the focal
. 25
plan e ............................................................................................................
26
2. 2. 4. Overall Instrument Configuration...............................................................
30
2. 3. Experiment and Result.......................................................................................
30
2. 3. 1. Signal loss due to aberrations as a function of imaging depth ........................
2. 3. 2. Point spread function degradation due to aberrations as a function of imaging depth
..................................................................................................................
. . 32
2. 3. 3. Mouse tongue muscle imaging using AO compensated two-photon microscopy. 34
2. 3. 4. Mouse heart muscle imaging using AO compensated two-photon microscopy.... 37
2. 3. 5. Neuronal imaging in mouse brain slices using AO compensated two-photon
microscopy....................................................................................................
40
2. 4. Conclusion....................................................................................................
42
3. Reassignment of Scattered Emission Photons in Multifocal Multiphoton Microscopy ...
47
3. 1. Introduction .................................................................................................
47
3. 2. M ethods .......................................................................................................
51
3. 2. 1. MN IM configuration ................................................................................
51
3. 2. 2. Image reconstruction methodology .............................................................
52
4
3 . 3 . Resu lts ...........
.
-----.....
- .............................................................................
57
3.3. 1. Simulation results................................................................................ 57
3.3.1. 1.4 Spots ina2by2M M
image....................................................57
3. 3. 1. 2. 4 Alphabets in a 2 by 2 MMM image ................................................
3. 3. 2. Experimental results ..............................................................................
60
61
3. 3. 2. 1. Fluorescent beads image in 6 by 6 MMM with 85 pm foci separation ..... 61
3. 3. 2. 2. Mouse brain imaging in 4 by 4 MMM with 85 pm foci separation..... 65
3. 3. 2. 3. Fluorescent beads and Mouse brain imaging in MMM with 45 Rm foci
separation ...........................................................................
67
3. 4. Conclusion...................................................................................................
68
4. Non-descanned Multifocal Multiphoton Microscopy With Multianode Photomultiplier Tube
.............................................................................................
71
4. 1. Introduction......................................................................................
..... 71
4. 2. Method ............................................................................................ 74
4. 2. 1. Non-descanned MMM with an imaging detector...........................................
74
4. 2. 2. Descanned MMM with a MAPMT .............................................................
75
4. 2. 3. Non-descanned MMM with MAPMT .........................................................
76
4. 2. 4. ZEMAX simulation for the corner and edge foci of the non-descanned MMM... 78
4. 2. 5. Detection strategy: image shifting ...........................................................
4 . 3 . Resu lt ........... ...........................................
4. 3. 1. Fluorescent beads images with 8
x
80
....................................................
81
8 MMM ..............................................
81
4. 3. 2. Field-of-view comparison .........................................................................
83
4. 3. 3. Signal attenuation comparison .................................................................
86
4. 3. 4. Mouse brain images with 4 x 4 MMM .....................................................
87
4. 4. Discussion...................................................................................................
89
5. Spectral Resolved Multifocal Multiphoton Microscopy with Multianode Photomultiplier
Tu b e ............................................................................................................................
5. 1. Introduction.................................................................................................
93
93
5.2 .M ethod ........................................................................................................ .9 6
5. 2. 1. System configuration ............................................................................... 96
5. 2. 2. Spectral decomposition .......................................................................... 98
5.3.R esult ........................................................................................................ ...
99
5. 3. 1. Detection channel calibration ................................................................
99
5. 3. 2. Three color fluorescent beads ..................................................................
100
5
5. 3. 3. Mouse kidney image ..............................................................................
101
5. 3. 4. Nerve sample image...............................................................................
103
5. 3. 5. Mouse brain image ................................................................................
105
5. 4. Discussion and Conclusion ...............................................................................
109
6. Conclusion and Outlook ..........................................................................................
6
113
List of Figures
Fig 1.1. Jablonski diagram for one-photon and two-photon excitation ..................................
14
Fig 1.2. Comparison of one- and two-photon excitation profiles..........................................
14
Fig 1.3. The schematic of two-photon excitation fluorescence microscopy.............................
15
Fig 2.1. System configuration. The components used are Ti-Sapphire (Ti-Sa) laser (Tsunami,
Spectra-Physics, Mountain View, CA), plano-convex lens LI (focal length 50 mm,
KPX082AR. 16, Newport, Irvine, CA), L2 (f 200 mm, KPX 1 06AR. 16, Newport), L3 (f 100
mm, KPX094AR. 16, Newport), L5 (f 200 mm, KPX 1 06AR. 16, Newport), L6 (f 400 mm,
KPXI15AR.16, Newport), L7 (f 50 mm, KPX082AR.16, Newport), L8 (f 20 mm, 444232,
Zeiss, Thornwood, NY), L10 (f 125 mm, KPX097AR. 16, Newport), LII (f 200 mm,
KPX 1 06AR. 16, Newport), L13 (f 100 mm, KPX094AR. 16, Newport), silver coated mirrors
(1OD20ER.2, Newport), beam splitter (IOBC17MB.2, Newport), deformable mirror
(Mirao52d, Imagine Optic, Orsay, France), scanning mirrors (6350, Cambridge Technology,
Lexington, MA), short-pass dichroic mirror (650dcxxr, Chroma Technology, Rockingham,
VT), objective lens (Fluar, 1.3 NA, Zeiss), photomultiplier detector (R7400P, Hamamatsu,
Bridgewater, NJ), pinhole (P150S, Thorlabs, Newton, NJ), and wavefront camera (HAS032,
Im agine O ptic)...............................................................................................
26
Fig 2.2. Wavefront change after the AO compensation. The sample is a mouse heart (described in
3.4) and imaged at 20 pim depth. (a) Distorted wavefront without the compensation. (b)
Wavefront after the AO compensation ................................................................
29
Fig 2.3. 3rd order aberration coefficients change after the AO compensation. (a) Uncompensated
coefficients corresponding to Fig 2.2(a). X-axis represents Zernike coefficients; 4 is Z2,2, 5
is Z2,-2, 6 is Z3,1, 7 is Z3,-i, and 8 is Z4,0. The error of the wavefront measurement is less
than 2 nm. (b) Compensated coefficients.............................................................
Fig 2.4. Signal loss experim ent sam ple ..........................................................................
29
31
Fig 2.5. Signal loss improvement. The ranges of error bars are smaller than their symbols.......... 31
Fig 2.6. PSF degradation experiment sample...................................................................
32
Fig 2.7. Resolution change after AO compensation. (a) Lateral resolution change. (b) Axial
R esolution change............................................................................................
Fig 2.8. Mouse tongue images at 80 pm depth without / with AO compensation .......................
Fig 2.9. Signal improvement after AO compensation. (a)
33
35
(d) show histograms for number of
pixels according to their intensity; x-axis represents intensity of the pixels. For example, 10
means the pixels that have 0-10% intensity of the maximum in the whole image, and 100
7
means 90~100% intensity pixels. Y-axis represents the number of pixels in the intensity
range. The blue bars show the number of pixels before the compensation, and red bars show
the result after the compensation. Each histogram was normalized to itself. (a) shows the
distribution at 20 pm depth, (b) at 40 gm, (c) at 60 gm, and (d) at 80 jm imaging depth. (e)
Percentage improvement according to imaging depth. Red line shows the improvement with
background rejection (only fluorescent area was calculated), and blue line shows the
improvement only with 90~100% intensity pixels. (f) Increment of mean photon count. Red
and blue lines show same data sets as (e). (g) Percentage improvement based on each pixel.
Any pixel with more than 2 fold improvement, that comes from mismatch of the
uncompensated and compensated image, is saturated to red color. The background is set to 1 for visualization purpose....................................................................................
Fig 2.10. Mouse heart images at 80 jm depth without / with AO compensation .....................
36
38
Fig 2.11. Signal improvement after AO compensation. (a) ~ (d) show histograms for number of
pixels according to their intensity; x-axis represents the intensity of pixels same as Fig 2.9
except 100 (100 means 40~100% intensity pixels in the image). Y-axis represents the
number of pixels in the intensity range. The blue bars show the number of pixels before the
compensation, and red bars show the result after the compensation. Each histogram was
normalized to itself. (a) shows the distribution at 20 jm depth, (b) at 40 gm, (c) at 60 jm,
and (d) at 80 jim imaging depth. (e) Percentage improvement according to imaging depth.
Red line shows the improvement with background rejection (only fluorescent area was
calculated), and blue line shows the improvement only with 90-100% intensity pixels ... 39
Fig 2.12. Mouse brain images at 50 pm depth without / with AO compensation ........................
41
Fig 2.13. Signal improvement after AO compensation. (a) and (b) show histograms for number of
pixels according to their intensity; x-axis represents intensity of the pixels, and y-axis
represents the number of pixels in the intensity range. The blue bars show the number of
pixels before the compensation, and red bars show the result after the compensation. Each
histogram was normalized to itself. (a) shows the distribution at 50 jm depth, and (b) at 150
pm imaging depth. The bottom figures of (a) and (b) are detailed distributions in the
selected range. (c) Percentage improvement according to imaging depth. Red line shows the
improvement with background rejection (only fluorescent area was calculated), and blue
line shows the improvement only with 90-100% intensity pixels .............................
42
Fig 3.1. Two experimental approaches for the estimation of a scattering matrix. (a) Sparsely
distributed beads sample in tissue phantom (2% Intralipid emulsion), and scattering matrix
measurement from its MMM image. (b) Single focus excitation on a real sample and
8
scattering matrix measurement from its MMM image ...........................................
50
Fig 3.2. Schematic of M MM based on the MAPM T.............................................................
52
Fig 3.3. Summary of the photon reassignment process for MMM........................................
57
Fig 3.4. (a) Original 4 beads image in a 2 by 2 MMM. (b) A scattering-affected image of (a); one
sub-image contains one primary bead image and three ghost bead images with the specified
proportions. (c) The processed image of (b). (d) Original alphabet image in a 2 by 2 MMM.
(e) A scattering-affected image of (d) in the same fashion. (f) The processed image of (e) 59
Fig 3.5. (a) Intensity comparison of the primary beads in the simulated MMM image (blue, main
anode + ghosts, subjected to Poisson noise) and their processed images (red). (b-d)
Scattering coefficients distribution at 2by2 MMM anodes. (b) Original distribution used in
simulation, (c) estimated distribution of the processed bead image, and (d) the alphabet
image. One sub-image contains a primary coefficient from the corresponding image and
three scattering coefficients showing the leakage of emission photons to the neighbor
anodes due to scattering ....................................................................................
61
Fig 3.6. Left column: fluorescent beads images with the 6 by 6 MMM system at three imaging
depth. Right column: Corresponding processed images .........................................
63
Fig 3.7. (a) Bead's signal comparison at different depths for original and processed image. (b)
Signal-to-Ghost ratio (SGR) of the imaging beads at different depths........................
64
Fig 3.8. (a) Plot of scattering coefficient for an anode and its distribution at neighboring anodes
corresponding to different imaging depths of fluorescence beads image, (b) Plot of emission
scattering coefficient as function of depth ............................................................
64
Fig 3.9. A mouse brain image with the 4 by 4 MMM system. Left acquired images at 100 pm and
190 gm imaging depths, and Right are the corresponding processed images...............
66
Fig 3.10. (a) Plot of scattering coefficient for an anode and its distribution at neighboring anodes
corresponding to different imaging depths of mouse brain, (b) Plot of emission scattering
coefficient as function of depth ..........................................................................
66
Fig 3.11. MMM images (with foci separation 45mm) and its processing. (a) Fluorescent beads
image with the 6 by 6 MMM system, and (b) is the corresponding processed image. (c)
Image of GFP expression neurons in mouse brain acquired using 8 by 8 MMM system at
90m depth, and (b) is the corresponding processed image .....................................
Fig 3.12. Plot of log-likelihood function for number of iterations .......................................
68
69
Fig 4.1. The schematic of MMM. The first MMM acquired signals with a camera, and further a
descanned MMM has been developed with a MAPMT..............................................
Fig 4.2. The schematic of MMM system with non-descanned detection................................
9
74
77
Fig 4.3. The f-numbers of Lens L 1 , L 2 , L 3 , and L 4 in the descanned and non-descanned MMM.... 78
Fig 4.4. The ZEMAX simulation of the excitation foci in the descanned and non-descanned
geometry, and their actual images. The scale bars in the actual images are 5 pm long ..... 79
Fig 4.5. The detection scheme of the sample shift. (a) The anode matrix of the MAPMT with 0.3mm
dead spaces [28]. (b) The scan areas in a specimen (orange) and the areas covered by the
anodes (blue). (c) One scan area covered by 4 time sample shift. (d) Overlapped scan areas
with 4 time and (e) 3 time movement in the 4
x
4 MMM case. Red: 4 time, yellow: 3 time,
green: 2 time, and violet: 1 time overlap. (f) SNR improvement of each case ................
81
Fig 4.6. Fluorescent bead images in the non-descanned MMM and the standard single focus
scanning microscope at 40, and 80 gm imaging depths. The images are 410 gm
x
410 gm
83
w ith 0.4 pm resolution ......................................................................................
Fig 4.7. Mouse kidney image. (a) Raw image of the descanned MMM, image size: 680 gm
gm with 8
x
x
680
8 foci. (b) Raw image of the non-descanned MMM image with the same FOV
with the same number of foci. (c) Raw image of the single focus scanning with the same
FOV. (d-f) Normalized images of (a-c) with (g-i). (g-i) 20 time averaged fluorescein
solution image of each system. The periodic pattern in (h) is the actual shape of the
84
faceplate in front of the cathod in the MAPMT .....................................................
Fig 4.8. Normalized intensity profile comparison of Fig 4.7 (g-i). The distance from the center to
the corner of the 680 gm
x
86
680 gm image is 481 pm..............................................
Fig 4.9. Fluorescence emission signal attenuation in the descanned and non-descanned MMM. Data
87
points are connected by lines for eye guidance .....................................................
Fig 4.10. Mouse brain images at 40, 100, and 160 pm depths. All images are 340 pm
x
0.4 pm pixel size. The left column images are descanned MMM images with 4
middle column images are non-descanned MMM images with 4
x
340 [tm with
x
4 foci. The
4 foci and the 4 time
sample movement. The right column images are the standard single focus scanned images
....................................................................................................................
. . 88
Fig 4.11. The fluorescence emission signal attenuation of the neuronal cell bodies in the descanned
and non-descanned MMM. The percentage numbers show the signal improvement, and the
error bars show ± one standard deviation at each data point. The data points are connected
by lines for eye guidance..................................................................................
89
Fig 5.1. The Schem atic of M M M ..................................................................................
94
Fig 5.2. The configuration of the spectral resolved MMM .................................................
96
Fig 5.3. Spectral distribution estimation by (a) imaging the single-color samples with 8 channels or
(b) based on the known emission spectrum of each specimen. The graph in (b) shows the
10
total detection efficiency as a function of wavelength. The spectral calibration is discussed
in section 3 .1.................................................................................................
. . 99
Fig 5.4. Three color fluorescent bead image. (a) Eight channel images in pseudo colors for display
purpose. The raw image of each channel contains only combined intensity information. (b)
The spectrum of each bead on the eight channels. (c) Processed image in pseudo colors. The
image size is 340 gm
x
340 m ...........................................................................
10 1
Fig 5.5. A mouse kidney image labeled with three colors. The red arrow indicates representative
actin labeled with phalloidin, the green arrows show glomeruli labeled with Alexa 488, and
the blue arrow for nuclei ....................................................................................
Fig 5.6. A nerve image surrounded by a scaffold labeled with three colors .............................
103
105
Fig 5.7. A mouse brain image labeled with two colors. (a) Spectral-resolved MMM image, (b)
single focus scanning image and the enlarged images of the specified areas ................
107
Fig 5.8. A mouse brain image labeled with two colors. (a) Spectral-resolved MIMM image at 184 pm
depth, and (b) at 156 gm depth. (c) The processed image of (a) with the ghost image
removal algorithm, and (d) the processed image of (b). (c) Single focus scanning image at
156 pm depth ...................................................................................................
11
10 8
Abbreviations
ID
one-dimensional
2D
two-dimensional
3D
three-dimensional
AO
adaptive optics
AOI
angle of incidence
CPC
custom parabolic light collector
DALYs
disability adjusted life years
DOE
diffractive optical element
FOV
field-of-view
FWHM
full-width at half-maximum
MAPMT
multianode photomultiplier tube
ML
maximum likelihood
MMM
multifocal multiphoton microscopy
NA
numerical aperture
PMT
photomultiplier tube
PSF
point spread function
SGR
signal-to-ghost image ratio
SNR
signal-to-noise ratio
SPAD
single-photon avalanche diode
TCSPC
Time-Correlated Single Photon Counting
WHO
world health organization
12
Chapter 1
Introduction
1.1 Multiphoton microscopy for neurobiological imaging
The economic costs of mental illness have been ranked one of the top five costly conditions in the entire
health care costs. The Agency for Healthcare Research and Quality cited a cost of $57.5B in 2006 for
mental health care in the U.S., which was equivalent to the cost of cancer care [1]. In addition, the World
Health Organization (WHO) has reported that mental illnesses are the leading causes of disability
adjusted life years (DALYs, the number of years lost due to ill-health, disability or early death)
worldwide, accounting for 37% of healthy years lost from non-communicable diseases [2]. Considering
not only the cost of health care, but also the intangible loss such as unemployment, expenses for social
supports, and a range of indirect costs due to a chronic disability, understanding of cognitive disorders,
needless to say, is important.
Numerous studies have revealed that many mental disorders exhibit some form of structural change in the
brain at the global as well as the cellular level [3-7]. It is suspected that neuronal and synaptic deficits
result in altered connectivity and thus give rise to pathophysiological circuit dysfunction. Therefore,
observing neuronal structures in relation to mental disorders and monitoring subtle changes in
connectivity with respect to time have a significant impact on understanding the basis of structural brain
abnormalities and the development of therapeutics for their correction.
Multiphoton excitation fluorescence microscopy is the method of choice for in vivo neuronal imaging in
animal models. The theory of the two-photon excitation was first introduced by Maria G6ppert-Mayer in
1931 [8]. In the two-photon excitation process, a fluorophore (fluorescent chemical compound that can reemit light upon light excitation) is brought to an electronically excited state by the simultaneous
13
absorption of two photons. Each photon has approximately half of the energy that is needed for the
transition between the ground state and the excited state [Fig 1.1]. Similarly one-photon excitation
process requires only single photon to excite a fluorophore. The one-photon excitation process typically
requires photons in the ultraviolet, blue, or green spectral range, which can easily damage biological
specimens. However, the two-photon excitation process typically uses red or infrared spectral range,
which has less energy than those for the one-photon excitation process. Therefore, two-photon excitation
microscopy is often more suitable for non-invasive biological imaging with less photon-damaging [9-11].
Si
Two-photon
absorption
One-photon
absorption
Fluorescence
So
Fig 1.1. Jablonski diagram for one-photon and two-photon excitation.
One-photon
excitation
Two-photon
excitation
Fig 1.2. Comparison of one- and two-photon excitation profiles [10].
Two-photon microscopy has inherent 3D resolution due to the quadratic dependence of the excitation
process on the incident light distribution. Two-photon excitation can be localized to a femtoliter region
14
about the focal point of a high numerical aperture objective [Fig 1.2]. In addition two-photon excitation
microscopy has the capability of deep tissue imaging. Typically tissue specimens are turbid and easily
scatter light, but scattering is inversely proportional to the wavelength to the fourth power in the Rayleigh
approximation; therefore, the longer wavelength excitation light of the two-photon excitation can better
penetrate the thick tissues compared to the one-photon excitation wavelengths.
In 1990, Denk, Webb, and co-workers first demonstrated the two-photon excitation fluorescence
microscopy [9]. As a light source, typically a femto-second pulsed laser is used. The high peak power of a
mode-locked laser pulse increases the efficiency of the two-photon excitation with minimal photoinduced thermal damage. The raster scanning is accomplished by deflecting the excitation beam using
galvanometer-driven x-y mirrors. The excitation beam is expanded by a scan lens and a tube lens to
ensure overfilling of the back aperture of an objective lens. The expanded laser beam is reflected by a
dichroic mirror towards the back aperture of the objective lens, and focused on the sample. The emission
light is collected by the same objective lens and passes through the dichroic mirror. A highly sensitive
photon detector, such as a photomultiplier tube, is used to detect the emission signal. Fig 1.3 shows the
schematic of two-photon excitation fluorescence microscopy.
"
-
Detector
Scanning
mirrors
N
Dchroic
Mirror
Laser
Source
Excitation
light
Emission
light
Objective lens
Specimen
Fig 1.3. The schematic of two-photon excitation fluorescence microscopy.
15
1.2 Imaging speed improvement in multiphoton microscopy
Multiphoton microscopy has been applied in structural plasticity studies of mature mouse brains including
the remodeling of the whole dendritic arbor of neurons [12-15]. Dendritic remodeling is a complex
process that depends on many factors including interactions between pre-synaptic and post-synaptic
neuronal partners. Since many neurons have extensive arbor structures, the study of neuronal interactions
on dendritic arbor remodeling often requires imaging over a volume of several cubic millimeters. For this
large volume imaging, fast imaging speed becomes critical for minimizing animal stress due to prolonged
anesthesia, overcoming motion artifacts, and improving imaging throughput. Several methods have been
developed to improve imaging speed beyond conventional point scanning approach
based on
galvanometric mirror scanners. One approach is based on the use higher speed scanners such as polygonal
mirrors [16], resonant mirror scanners [17], or acousto-optical deflectors [18-21]. These high speed
scanners can typically achieve frame rate up to about 1 kHz in tissues with a comparable imaging depth as
conventional multiphoton microscopy. However, the higher speed scanning requires a correspondingly
decreased pixel dwell time resulting in lower signal-to-noise ratio (SNR). This tradeoff can be partially
compensated by increasing the excitation laser power but laser power is ultimately limited by specimen
photodamage and excitation saturation [11, 22]. Another approach is two-photon wide-field imaging
based on temporal focusing [23, 24]. In this case, two-photon excitation is localized to a plane, instead of
a point, by controlling the spectrally dispersing the light pulse away from the focal plane. However, widefield two-photon imaging is often limited by the lower axial resolution and the smaller field-of-view
(FOV) due to the need for much higher peak power laser pulses. Another popular approach to improve
imaging speed is multifocal multiphoton microscopy (MMM) [25, 26]. With a lenslet array or a
diffractive optical element (DOE) [27, 28], multiple foci are generated and scanned simultaneously.
Within the limit of available laser power, about a hundred foci can be effectively generated with a
standard Ti-Sapphire oscillator resulting in approximately two orders of magnitude improvement in
imaging speed proportionally. For MMM system, simultaneous acquisition of data from many foci often
16
require the use of an area detector, such as a CCD or a CMOS sensor [25, 26]. The spatial registration is
achieved by telecentric mapping of the image plane into the image sensor plane. In the presence of a
turbid specimen, the emission photons can be scattered resulting in image blurring. More important, as
scattered emission photons are scattered away from the correct location, they contribute to an increased
background and degrading image SNR. To overcome the scattering issue, MMM with a descanned
detection configuration using a multianode photomultiplier tube (MAPMT) has been developed [29, 30].
As in most confocal microscopes, the descanned geometry guides the emission photons from each focus
backward along the same optical path shared with the excitation beams [31-34]. Since the motion of the
scanning mirror is much slower than the speed of light, the emission light rays after the scanning mirror
becomes stationary independent of scanner motion and can be separated from the excitation beams with a
dichroic mirror. The unscattered ballistic emission photons generated by each focus are focused at the
cathode which is corresponding to the center of each anode of the MAPMT. The scattered emission
photons will not be focused at the center but many can still be collected by the corresponding anode. Very
intuitively, the larger area projected by each anode at the specimen plane will results in greater collection
efficiency of these scattered emission photons. At the limit of infinitely large anode, the collection
efficiency of scattered emission photons will be comparable to that of a single focus scanning
multiphoton system. Since the infinitely large anode cannot be completely realized, if the sample is highly
scattering and imaging depth is large, the scattered emission photons will reach neighbor anodes resulting
in ghost images in the neighbor sub-images. In addition, the emission signal loss can be severe in the long
optical path associated with the descanned detection geometry due to the small size of optics and many
optical surfaces [29]. To achieve not only fast imaging speed, but also comparable SNR, adaptive optics
(AO) compensation (chapter 2), image post processing for emission photon reassignment (chapter 3), and
non-descanned MMM (chapter 4) have been investigated in this thesis. In addition, spectral resolved
MMM is further introduced for simultaneous fast imaging and spectral detection (chapter 5).
17
1.3 Signal-to-noise ratio improvement in MMM
1.3.1 Adaptive optics compensation
The imaging depth of two-photon excitation fluorescence microscopy is partly limited by the
inhomogeneity of the refractive index in biological specimen. This inhomogeneity results in a distortion
of the wavefront of the excitation light. This wavefront distortion results in image resolution degradation
and lower signal level. Using an adaptive optics system consisting of a Shack-Hartmann wavefront sensor
and a deformable mirror, the wavefront distortion can be measured and corrected. With adaptive optics
compensation, it is demonstrated that the resolution and signal level can be better preserved at greater
imaging depth in a variety of ex vivo tissue specimens including mouse tongue muscle, heart muscle and
brain in chapter 2. However, for these highly scattering tissues, it is found that signal degradation due to
scattering to be a more dominant factor than aberration.
1.3.2 Emission photon reassignment
MMM achieves fast imaging by simultaneously scanning multiple foci across different regions of a
specimen. The sub-images generated by the foci are combined as a montage to form the final image. In
order to resolve the signals from the different foci, imaging detectors, such as CCD cameras, are used.
The use of imaging detectors results in degradation of image SNR due to the scattering of emitted photons
into neighbor pixels limiting image depth. It has been shown that SNR can be partly recovered using
MAPMT [29]. In this design, however, emission photons scattered to neighbor anodes are encoded by the
foci scan location resulting in ghost images, duplicates of an image acquired by one focus added onto the
images of neighbor foci. The crosstalk between different anodes can be quantified by a scattering matrix
that can be measured a priori. With the known scattering matrix, the photons of these ghost images can be
reassigned to restore the final picture. However, a priori measurement of this scattering matrix is
18
cumbersome as scattering crosstalk depends on specimen type, location, and depth. The need to measure
the scattering matrix a priori partly negates the advantage of the high speed imaging of MMM. In chapter
3, a novel methodology is investigated for the photon reassignment process based on the maximum
likelihood estimation for the quantification of the crosstalk between the anodes of MAPMT without a
priori measurement. The method provides the reassignment of the photons generated by the ghost images
to the original spatial location thus increases the SNR of the final reconstructed image.
1.3.3 Non-descanned MMM
MMM improves imaging speed over point scanning approach by parallelizing the excitation process.
Early versions of MiMM rely on imaging detectors, such as CCD or CMOS, to simultaneously record
emission signals from multiple foci. Image registration is achieved by telecentric mapping of the
specimen plane to the image plane of the detector. For many turbid biological specimens, the scattering of
emission photons results in blurred images and degrading SNR. It has been demonstrated that a MAPMT
placed in a descanned configuration can effectively collect scattered emission photons from each focus
into their corresponding anodes significantly improving image SNR for highly scattering specimens [29].
This approach is especially effectively with high numerical aperture, low magnification objectives that
allows foci to be widely separated in the specimen plane. However, this descanned MMM approach has
several drawbacks. The longer detection path of the descanned configuration caused significant emission
photon loss up to 70 % [29]. Further, it is a challenge to design intermediate optics in the descanned
geometry that minimizes optical aberration of the foci located in the edge of the large FOV for high NA,
low magnification objectives resulting in resolution loss at image edges. Overcoming these drawbacks, a
novel non-descanned MMM based on MAPMT is introduced in chapter 4. In this non-descanned
configuration, the excitation foci can be widely separated covering the full FOV of these objectives with
minimal aberrations, and signal collection efficiency can be improved up to fourfold.
19
1.4 Spectral resolved MVM
Multiphoton excitation fluorescence microscopy is the preferred method for in vivo deep tissue imaging.
There are many biological applications that demand high imaging speed. One of the successful methods
to improve the imaging speed in highly turbid specimen is multifocal multiphoton microscopy (MMM)
based on multi-anode photomultiplier tubes (MAPMT). It improves imaging speed by using multiple foci
for parallelized excitation without sacrificing signal to noise ratio (SNR) due to the scattering of emission
photons. In this work, we demonstrate that the MAPMT based MMM approach can be extended by
implementing of spectroscopic imaging capability. Instead of generating multiple excitation foci in 2D
grid pattern, a linear array of foci is generated. This leaves one axis of the MAPMT available for spectral
dispersion and detection. The spectral-resolved MMM can detect several emission signals simultaneously
with high imaging speed optimized for high-throughput, high-contents applications (chapter 5).
20
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
Soni, A., The five most costly conditions, 1996 and 2006: Estimatesfor the US civilian noninstitutionalized
population. Statistical Brie# 248. July 2009. 2009, Rockville, MD: Agency for Healthcare Research and
Quality.
Alwan, A., Global status report on noncommunicable diseases 2010. 2011: World Health Organization.
Fiala, J.C., J. Spacek, and K.M. Harris, Dendritic spine pathology: cause or consequence of neurological
disorders?Brain Research Reviews, 2002. 39(1): p. 29-54.
Grutzendler, J. and W.-B. Gan, Two-photon imaging of synaptic plasticity and pathology in the living
mouse brain. NeuroRx, 2006. 3(4): p. 489-496.
Penzes, P., et al., Dendritic spine pathology in neuropsychiatric disorders. Nature neuroscience, 2011.
14(3): p. 285-293.
Kays, J.L., R.A. Hurley, and K.H. Taber, The dynamic brain: neuroplasticity and mental health. The
Journal of neuropsychiatry and clinical neurosciences, 2012. 24(2): p. 118-124.
Murmu, R.P., et al., DendriticSpine Instability Leads to ProgressiveNeocortical Spine Loss in a Mouse
Model of Huntington's Disease. The Journal of Neuroscience, 2013. 33(32): p. 12997-13009.
Gppert-Mayer, M., Ober elementarakte mit zwei quantensprungen. Annalen der Physik, 1931. 401(3): p.
273-294.
Denk, W., J.H. Strickler, and W.W. Webb, 2-PHOTON LASER SCANNING FLUORESCENCE
MICROSCOPY Science, 1990. 248(495 1): p. 73-76.
So, P.T.C., et al., Two-photon excitation fluorescence microscopy. Annual Review of Biomedical
Engineering, 2000. 2: p. 399-429.
Zipfel, W.R., R.M. Williams, and W.W. Webb, Nonlinear magic: multiphoton microscopy in the
biosciences. Nature biotechnology, 2003. 21(11): p. 1369-1377.
Lee, W.C.A., et al., Dynamic remodeling of dendritic arbors in GABAergic interneurons of adult visual
cortex. Plos Biology, 2006. 4(2): p. 271-280.
Lee, W.C.A., et al., A dynamic zone defines interneuronremodeling in the adult neocortex. Proceedings of
the National Academy of Sciences, 2008. 105(50): p. 19968-19973.
Chen, J.L., et al., Structural basis for the role of inhibition in facilitating adult brain plasticity. Nature
neuroscience, 2011. 14(5): p. 587-594.
Chen, J.L., et al., Clustered dynamics of inhibitory synapses and dendritic spines in the adult neocortex.
Neuron, 2012. 74(2): p. 361-373.
Kim, K.H., C. Buehler, and P.T.C. So, High-speed,two-photon scanning microscope. Applied optics, 1999.
38(28): p. 6004-6009.
Fan, G., et al., Video-rate scanning two-photon excitationfluorescence microscopy and ratio imaging with
cameleons. Biophysical journal, 1999. 76(5): p. 2412-2420.
Iyer, V., B.E. Losavio, and P. Saggau, Compensation of spatial and temporal dispersionfor acousto-optic
multiphoton laser-scanningmicroscopy. Journal of Biomedical Optics, 2003. 8: p. 460.
Reddy, G.D. and P. Saggau, Fast three-dimensional laser scanningscheme using acousto-optic deflectors.
Journal of Biomedical Optics, 2005. 10: p. 064038.
Zeng, S., et al., Simultaneous compensation for spatial and temporal dispersion of acousto-optical
deflectorsfor two-dimensional scanning with a singleprism. Optics Letters, 2006. 31(8): p. 1091-1093.
Katona, G., et al., Fast two-photon in vivo imaging with three-dimensionalrandom-access scanning in
large tissue volumes. Nature methods, 2012. 9(2): p. 201-208.
Cianci, G.C., J. Wu, and K.M. Berland, Saturation modified point spread functions in two-photon
microscopy. Microscopy research and technique, 2004. 64(2): p. 135-141.
Oron, D., E. Tal, and Y. Silberberg, Scanningless depth-resolved microscopy. Optics Express, 2005. 13(5):
p. 1468-1476.
Zhu, G., et al. Simultaneous spatial and temporalfocusing offemtosecond pulses. 2005: Optical Society of
America.
Bewersdorf, J., R. Pick, and S.W. Hell, Multifocal multiphoton microscopy. Optics Letters, 1998. 23(9): p.
655-657.
Buist, A.H., et al., Real time two-photon absorption microscopy using multi point excitation. Journal of
Microscopy-Oxford, 1998. 192: p. 217-226.
21
27.
28.
29.
30.
31.
32.
33.
34.
Sacconi, L., et al., Multiphoton multifocal microscopy exploiting a diffractive optical element. Optics
Letters, 2003. 28(20): p. 1918-1920.
Watson, B.O., V. Nikolenko, and R. Yuste, Two-photon imaging with diffractive opticalelements. Frontiers
in neural circuits, 2009. 3.
Kim, K.H., et al., Multifocal multiphoton microscopy based on multianode photomultiplier tubes. Optics
Express, 2007. 15(18): p. 11658-11678.
Martini, J., V. Andresen, and D. Anselmetti, Scattering suppression and confocal detection in multifocal
multiphoton microscopy. Journal of biomedical optics, 2007. 12(3): p. 034010-034010-6.
Minsky, M., Microscopy apparatus.1961, US Patent 3013467.
Sheppard, C. and A. Choudhury, Image formation in the scanning microscope. Journal of Modem Optics,
1977. 24(10): p. 1051-1073.
Brakenhoff, G., P. Blom, and P. Barends, Confocal scanning light microscopy with high aperture
immersion lenses. Journal of microscopy, 1979. 117(2): p. 219-232.
Carlsson, K. and N. Aslund, Confocal imagingfor 3-D digitalmicroscopy. Applied optics, 1987. 26(16): p.
3232-3238.
22
Chapter 2
A Shack-Hartmann Wavefront Sensor Based Adaptive Optics
System for Multiphoton Microscopy'
2.1. Introduction
Two-photon excitation fluorescence microscopy has millimeter scale penetration depth in biological
specimens [1, 2]. However, the refractive index inhomogeneity in these samples often distorts the
wavefront of the excitation light resulting in broader focal point spread function and lower two-photon
excitation efficiency. With an adaptive optics system, this wavefront distortion can be compensated and
image resolution and signal level can be preserved at greater imaging depth.
The adaptive optics approach was first developed in fields such as astronomy [3] and ophthalmology [4].
In microscopy, significant progress has been made in confocal systems [5, 6], and also adaptive optics has
been applied to nonlinear microscopies such as two-photon excitation microscopy [7, 8], harmonic
generation microscopy [9, 10], and CARS microscopy [11]. The different implementations of adaptive
optics to nonlinear microscopy can be roughly classified into two approaches: feedback via two-photon
excited fluorescence light or feedback via reflected excitation light. Most fluorescence light feedback
systems are based on maximizing signal strength, for example, by varying the shape of the deformable
mirror via genetic algorithms [12, 13], or image-based algorithms [14]. In addition, the choice of
performance parameters for feedback control using genetic algorithm (e.g., brightness, contrast, and
resolution) have been investigated [15]. As an alternative approach, a differential aberration imaging
technique has been demonstrated by rejecting out-of-focus fluorescence background signals by using a
defocused image [16]. For feedback control using reflected excitation light, most implementations utilize
'This chapter was modified from Ref. 36 with the permission of the publisher.
the coherent nature of the reflected light and incorporate a wavefront sensor component. A major
challenge in applying wavefront sensing in nonlinear microscopes with inherent 3D resolution is to select
the reflected light signal preferentially from the focal plane of interest but not throughout the optical path
of the excitation light. In one approach, Rueckel and co-workers used coherence-gated wavefront sensing
method [17, 18]. This approach utilizes the low temporal coherence properties of femtosecond laser
pulses. The reflected light signal from the focal plane is selected interferometically via a modified
Michelson interferometer and the wavefront distortion is quantified using a shear interferometer.
In this work, we demonstrate an alternative method for adaptive correction in two-photon microscopy via
reflected light feedback utilizing confocal principle for depth selection. This approach simplifies the
optics in the wavefront detection light path and reduces the potential of inducing additional aberrations.
Further, we utilize a Shack-Hartmann wavefront sensor that allows the wavefront distortion to be
monitored by acquiring a single image improving the bandwidth of the adaptive optics feedback loop.
2.2. Methods and Experimental Setup
This two-photon microscope with adaptive optics compensation was constructed by modifying an
existing system as previously described [2]. The major additional components consist of a large dynamic
range deformable mirror, a Shack-Hartmann wavefront sensor, and a confocal light path for reflected light
detection.
2.2.1. A large dynamic range deformable mirror
The deformable mirror (Mirao52d, Imagine Optic, Orsay, France) includes a silver-coated reflective
membrane coupled 52 miniature voice-coil type actuators. The diameter of its pupil is 15.0 mm and it has
200 Hz bandwidth. The maximum wavefront amplitude that can be generated is ±50 gm with ±1 V
24
voltage range. An important feature of this deformable mirror is its large dynamic range. It can generate
mimic Zernike modes up to 4th order and partially up to 6th order.
2.2.2. The detection of the wavefront distortion with a Shack-Hartmann wavefront sensor
The Shack-Hartmann wavefront sensor (HASO32, Imagine Optic, Orsay, France) consists of a lenslet
array and a CCD sensor. It has 32x32 lenslets within 3.6x3.6 mm2 rectangular aperture. Its bandwidth is
20 Hz. With an incident plane wave, an array of foci produces a rectilinear grid defined by the lenslet
dimension. However, if the incoming wavefront is distorted, the foci are displaced from the grid location.
By measuring the distance of the displacement using a CCD camera, the slope of local wavefront can be
calculated and the whole wavefront can be constructed. The advantage of the Shack-Hartmann sensor is
its ability to rapidly quantify wavefront aberration with a single image.
2.2.3. The application confocal detection to select reflected light signal from the focal plane
Only the aberration information of light from the focal region of the objective is relevant for optimizing
two-photon excitation. However, the reflected light signal is not only produced at the focal plane, but also
from out-of-focal regions. The signal from out-of-focal regions needs to be eliminated, and we
accomplished it by setting up a confocal detection light path. The confocal pinhole passes the signal
originating from the focal region of the objective while blocking the out-of-focus light at the confocal
pinhole. The depth selectivity, axial resolution, should be optimized by selecting a small pinhole [19, 20].
However, this requirement must be traded-off against the fact that the pinhole also acts as a spatial filter.
A very small pinhole will filter out all higher spatial frequency information of the distorted wavefront
needed for adaptive correction. These two conflicting requirements must be balanced in our system. The
optimization of the pinhole size will be discussed in the next section.
25
2.2.4. Overall Instrument Configuration
F --- Detector
Scanner
mirror
@
DO
IX I
Dichroic
Mirror
Zn
f4
Wavefront
Camera
C)
1
0
Reflec ted
Beam from
Samp le
Beam Spl itterI
Sample
f4
50/50
-T
I
P-
/4
Deformable
Mirror
C
Fig 2.1. System configuration. The components used are Ti-Sapphire (Ti-Sa) laser (Tsunami, Spectra-Physics,
Mountain View, CA), plano-convex lens LI (focal length 50 mm, KPX082AR. 16, Newport, Irvine, CA), L2 (f 200
mm, KPX106AR.16, Newport), L3 (f 100 mm, KPX094AR.16, Newport), L5 (f 200 mm, KPX106AR.16,
Newport), L6 (f 400 mm, KPX1 15AR.16, Newport), L7 (f 50 mm, KPX082AR.16, Newport), L8 (f 20 mm, 444232,
Zeiss, Thornwood, NY), L10 (f 125 mm, KPX097AR. 16, Newport), L 1 (f 200 mm, KPX106AR. 16, Newport), L 13
(f 100 mm, KPX094AR. 16, Newport), silver coated mirrors (1 OD20ER.2, Newport), beam splitter (1OBC 17MB.2,
Newport), deformable mirror (Mirao52d, Imagine Optic, Orsay, France), scanning mirrors (6350, Cambridge
Technology, Lexington, MA), short-pass dichroic mirror (650dcxxr, Chroma Technology, Rockingham, VT),
objective lens (Fluar, 1.3 NA, Zeiss), photomultiplier detector (R7400P, Hamamatsu, Bridgewater, NJ), pinhole
(P150S, Thorlabs, Newton, NJ), and wavefront camera (HAS032, Imagine Optic).
Fig 2.1 describes the general experimental setup. The laser source was a Ti-Sapphire (Ti-Sa) laser
(Tsunami, Spectra-Physics, Mountain View, CA) pumped by a frequency doubled Nd:YVO4 laser
(Millennia V, Spectra-Physics, Mountain View, CA). The excitation light was reflected by the deformable
26
mirror (Mirao52d, Imagine Optic, Orsay, France), positioned at a conjugate plane of the back aperture of
the microscope objective, towards the excitation optical path of a typical two-photon laser scanning
microscopy [2]. The emission fluorescence signal passing a short-pass dichroic mirror and a short pass
barrier filter (650dcxxr and E700SP, Chroma Technology, Rockingham, VT) was incident upon a
photomultiplier detector (R7400P, Hamamatsu, Bridgewater, NJ) and the associated single photon
counting circuitry. The scattered light signal was reflected by the dichroic scanner, descanned by passing
through the scanning mirrors, and directed toward the wavefront camera (HASO32, Imagine Optic,
Orsay, France). The wavefront camera was also positioned at a conjugate plane of the microscope
objective back aperture. Before the wavefront camera, a confocal pinhole (P150S, Thorlabs, Newton, NJ)
was positioned at a plane conjugated to the microscope focal plane selecting depth-resolved reflected light
signal.
As described above, a small pinhole is necessary for good axial resolution. However, the pinhole also
should be sufficiently large to pass the spatial frequency information contained in the distorted wavefront.
We set the minimum pinhole size based on the highest spatial frequency that the deformable mirror can
generate because higher order distortion cannot be corrected in any case. The distances between the two
actuators on the deformable mirror is 2.5 mm corresponding to a minimum spatial period of 5 mm.
Fourier transform of this spatial period gives a maximum spatial frequency that needs to pass through the
wavefront detection light path. We determined that a confocal aperture with diameter no less than 125 gm
placed at plane 12 of Fig 2.1 would transmit all the relevant aberration information. However, we chose to
use a 150 pm diameter pinhole to provide some additional margin of error. It is necessary to examine
what axial resolution is afforded by this 150 gm diameter pinhole. Accounting for the magnification of
the intermediate lenses, the 150 gm pinhole at plane 12 corresponds to an effective aperture of 37.5 pm
radius at plane 9 of Figure 1. At plane 9, the axial resolution is related to the aperture size by equation (1)
[21]:
27
d,(3dB)plane =
a
(1)
M sino
00)
where a is the radius of the aperture (37.5 gm), sinO0 is NA/n = 1.3/1.518 = 0.856 for an oil immersion
objective lens (Fluar, 1.3 NA, Zeiss, Thornwood, NY), NA is the numerical aperture of the objective lens,
n is the index of refraction of medium, and M is the magnification. Since the focal length of lens 10 is 125
mm, the effective magnification M is 30.3. This results in an axial resolution of about 2.0 pm. This is
resolution more than sufficient because the back aperture of the objective is underfilled corresponding to
an effective axial resolution of 4.8 pm, to be discussed later in section 3.2.
By measuring the wavefront distortion, the wavefront camera provided feedback signal to the deformable
mirror. The deformable mirror generated a pre-distortion to the plane wave that countered the distortion
induced by the sample. The pre-distortion from the deformable mirror and the distortion from the sample
were cancelled out to produce a more ideal point spread function at the focal point. It should be noticed
that the light that was detected by the wavefront camera passed the deformable mirror twice while the
actual incoming excitation light to a sample passed the deformable mirror once. Under the assumption the
specimen is highly scattering, this is the appropriate optical configuration since the detected light also got
aberrated twice by the sample in reflected light mode. Therefore, with close loop control, the adaptive
optics compensation worked with the pre-distorted excitation light before the sample producing a flat
wavefront at the wavefront camera after the feedback system reaches steady state.
Fig 2.2 and Fig 2.3 show representative wavefronts and aberration coefficients before and after adaptive
optics compensation. The wavefront sensor provides the deformable mirror with the feedback signal
satisfying 4 fold of Nyquist-Shannon sampling criterion. Therefore, the adaptive optics system can
correct the wavefront up to the maximum capability of the deformable mirror, which is under 10 nm RMS
error according to its specification.
28
(b)
(a)
0.14
0.14
-3.0
-1.8
-0.6
0.6 X Axis
0
-0.14
1.8
-2.4 -1.2
0
1.2
Y Axis (mm)
-3.0
-1.8
-0.6
0.6 X Axis
0
(mm)
-0.14
2.4 3.0
1.8
-2.4 -1.2
1.
0
241.
.2
Y Axis (mm)
(mm)
3.0
0.04
-0.0
0.
-0.08
0.
2.4
-0.12
-0.14
Fig 2.2. Wavefront change after the AO compensation. The sample is a mouse heart (described in 3.4) and imaged at
20 pm depth. (a) Distorted wavefront without the compensation. (b) Wavefront after the AO compensation.
(a)
Astigmatism at 0*
-0.0411
Astigmatism at 450
(b)
Astigmatism at 0*
-0.0120
0.0198
Astigmatism at 450
0.0005
Coma at 00
0.0051
Coma at 00
-0.0047
Coma at 900
-0.0087
Coma at 90*
-0.0012
-0.0048
3th order spherical aberration
-0.0072
3th order spherical aberration
0.02
--
0.02
--
0.00
0.00
-0.02
-0.02
-0.04
-0.04
pm
4
7
6
5
3rd order aberrations
(Zernike coefficients)
Pm
8
-
4
7
6
5
3rd order aberrations
(Zernike coefficients)
8
Fig 2.3. 3rd order aberration coefficients change after the AO compensation. (a) Uncompensated coefficients
corresponding to Fig 2.2(a). X-axis represents Zernike coefficients; 4 is Z2,2, 5 is Z2,-2, 6 is Z3,1, 7 is Z3,-1, and 8
is Z4,0. The error of the wavefront measurement is less than 2 rm. (b) Compensated coefficients.
It should be also noted that our method holds only when the sample is highly scattering, which is often,
but not always, the case in biological samples. For a more reflective sample, the measured wavefront is
dependent on whether the aberrations generated by the sample are odd (asymmetric) or even (symmetric),
which was extensively discussed in the references [22, 23, 24]. For a reflective sample with the odd
aberrations, this system configuration gives incorrect wavefront measurement.
29
In our system configuration, there are the losses of excitation laser power and reflected light signal due to
the use of the 50/50 beam splitter. The losses can be avoided by using a polarizing beam splitter followed
by a quarter-wave plate [8]. This configuration reflects 100% of the excitation laser power instead of 50%
in the absence of multiple scattering.
2.3. Experiment and Result
To thoroughly quantify the performance of our adaptive optics system, we have devised a series of
experiments using artificial and biological specimens. Two artificial specimens with large aberration were
prepared and used to evaluate our adaptive optics system in terms of minimizing signal loss and
optimizing image resolution loss. Further, the performance of this adaptive optics system was evaluated in
three typical biological tissue specimens: mouse tongue muscle, heart muscle, and brain slice.
2.3.1. Signal loss due to aberrations as a function of imaging depth
This experiment was designed to measure the signal loss due to aberration as a function of imaging depth
and the compensatory performance of the adaptive optics system. A 20x air objective lens (Fluar, 0.75
NA, Zeiss, Thornwood, NY) was intentionally used to image into aqueous fluorescein samples with
varying thickness, 50, 100, and 150 pm [Fig 2.4]. A mirror (1OD20ER.2, Newport, Irvine, CA) was
placed at the bottom of the specimen to enhance reflected light signal. A femtosecond pulsed laser
(Tsunami, Spectra-Physics, Mountain View, CA) was used to provide 780 nm excitation light. The twophoton excitation focal volume was placed just above the mirror surface within 1-2 pm. The refractive
index mismatch between the air objective and the aqueous specimen resulted in the generation of
significant aberration, especially spherical aberration [25, 26]. As the focus went deeper, the signal loss
became higher. The signal loss was measured with a normal two-photon excitation fluorescence
microscope with and without the adaptive optics compensation and the two signal losses were compared.
30
Air -Fluorescein
Cover slip
Fluorescein
Spacer
Mirror surface
Fig 2.4. Signal loss experiment sample
1.00
0.97
I
1.00
0.99
.1.9
0.89
0.8
0.6
0.4
z
0.36
0.2 V
0
0
100
50
150
Depth (gm)
Fig 2.5. Signal loss improvement. The ranges of error bars are smaller than their symbols.
Fig 2.5 shows the result of the fluorescein emission signal loss experiment. The uncompensated signal is
the blue line and the compensated signal is the pink line. The uncompensated signal decreased with
increasing focus depth. However, with adaptive optics compensation, the emission signal remained
almost constant. The signal improvement was 1% at 50 Pm, 7% at 100 pm, and 147% at 150
sm,
sm. At
150
the maximum Zernike coefficient of the uncompensated aberration among the 3rd order aberrations
31
was 0.39 pm, but it became 0.10 pm with the adaptive optics compensation. The Zernike coefficient for
spherical aberration (Z4,0) was reduced from 0.036 pm to 0.004 pm. The maximum aberration was
astigmatism instead of the spherical aberration and it may be caused either by slight misalignment of the
lenses or by slight bending of the dichroic mirror that is common in some microscope dichroic mirror
holders [27, 28]. In addition, the excitation laser source was linearly polarized in front of the sample
instead of circular polarization and it is known that scattering depends on polarization direction.
Therefore, linearly polarized excitation light may generate erroneous astigmatism [18].
2.3.2. Point spread function degradation due to aberrations as a function of imaging depth
Oil -+ Air
Cover slip
Air
1
Spacer
d=50pm
Fluorescence
Mirror surface
beads
Fig 2.6. PSF degradation experiment sample
In this experiment, we tried to quantify point spread function degradation due to aberrations as a function
of imaging depth and the compensatory performance of the adaptive optics system. A 40x oil immersion
objective lens (Fluar, 1.3 NA, Zeiss, Thornwood, NY) was used to image 0.1 pm diameter fluorescence
beads (F8803, Invitrogen, Eugene, OR) deposited on a mirror though an air gap of 50 pm [Fig 2.6]. The
mirror was used to enhance reflected light signal. The excitation wavelength was 780 nm. Spherical
aberration was again generated due to index mismatch between the oil objective and the air specimen.
32
The point spread function was measured with a normal two-photon excitation fluorescence microscope
with and without the adaptive optics system.
Fig 2.7 shows the effect of aberration on PSF laterally and axially. The blue bar shows
the
uncompensated resolution and the pink bar shows the compensated resolution. Clearly, when the air gap
thickness is zero, no compensation is necessary. At 50 pm air gap, lateral resolution was improved by
12% and axial resolution was improved by 38%. Lateral resolution is proportional to 1/NA and axial
resolution is proportional to l/NA2, so it is a reasonable result that axial resolution was affected more
than lateral resolution with the adaptive optics compensation. The Zernike coefficient of the
uncompensated maximum aberration had been 0.44 pm, but it became 0.09 pm with compensation. The
Zernike coefficient for spherical aberration (Z4,0) became from 0.14 pm to 0.02 gm. Here the reason that
the spherical aberration was not the maximum value among the 3rd order aberrations may be again due to
reasons cited in the previous section.
(a)
(b)
15
03
0.86 0.86
11.34
0.91
E
{j5
0.5
0
Depth (pm)
50
0
Depth (pm)
50
Fig 2.7. Resolution (full-width at half-maximum: FWHM) change after AO compensation. (a) Lateral resolution
change. (b) Axial Resolution change.
Further, we should note that the lateral and axial resolutions were larger than the values of diffraction
limited resolutions. It was because the objective lens was underfilled by the excitation light. As we
mentioned with the system configuration, the deformable mirror gives same compensation to the input
beam and the outgoing beam from the sample. The two beams should be located exactly at the same area
on the deformable mirror, so their beam sizes should be same and the objective was just filled with the
33
excitation light. However, because of the Gaussian characteristic of the laser beam, the objective lens was
practically underfilled and the lateral and axial resolutions became larger. According to the experimental
result, the effective NA of the objective was about 0.55 in our experimental setup although its theoretical
NA is 1.3, which is valid only when immersion oil is present under the objective lens until a cover slip,
and a sample is then just below the cover slip in a refractive index of 1.5. This current limitation may be
removed by utilizing a top-hat wavefront shaper or modifying the adaptive optics feedback algorithm to
ignore information from the peripheral region of the wavefront sensor.
2.3.3. Mouse tongue muscle imaging using AO compensated two-photon microscopy
The mouse tongue musculature was visualized based on both endogenous fluorescence and second
harmonic generation. The whole tongue excised from a female C57BL/6 mouse (10 weeks old) was fixed
in phosphate buffered saline (PBS, pH 7.4) containing 2.5% glutaraldehyde for about a day. The fixed
tongue tissue was then immersed in PBS for 3 hours, rinsed with the same buffer excessively and finally
embedded in paraffin without a cover slip. The wavelength of the excitation light was set to 780 nm. The
objective lens was a 40x oil immersion objective lens (Fluar, 1.3 NA, Zeiss, Thornwood, NY). The
emission signal was filtered by a green filter (535/40, Chroma Technology, Rockingham, VT). The field
size was 120 gmx 120
sim with 256x256 pixels
and the dwell time was 40 ps. As representative images,
Fig 2.8 shows mouse tongue muscles images at 80 pm depth with and without the adaptive optics
compensation. The left image is the uncompensated image and the right one is the compensated image
and they were processed with background rejection. The threshold for each image was set to three times
the intensity level measured in regions outside objects of interest (e.g. blood vessels in the mouse heart or
the neurons in the mouse brain).
34
16
14
12
10
8
6
4
2
0
20pm
20pm
Fig 2.8. Mouse tongue images at 80 pm depth without / with AO compensation
In principle, the bandwidths of the wavefront camera and the deformable mirror are about 20 Hz and 200
Hz respectively. However, the current feedback algorithm takes significant CPU resource and it leads to a
compensation time of 4-5 seconds. Therefore, the algorithm is the bottleneck for the bandwidth of the
whole compensation process and it is far from practical for pixel-by-pixel correction over the whole
image. Instead, with the assumption that the optical paths are similar over the scanned area of the image,
adaptive optics compensation was performed only at the center pixel of the image and the rest of the
image was acquired with the same deformable mirror shape setting [29]. In future studies, it would be
interesting to determine the optimal number of pixels that should be corrected per image given the tradeoff between imaging speed and tissue heterogeneity. It would also be interesting to improve imaging
speed by developing faster, more efficient feedback algorithm or by utilizing higher performance
computer hardware.
35
(a)
(b)
50
mD
40
4)
0--
4)
0
0 A
4)
(
1
(d)
AT1___
30
20
10
19~Iii'h
1o-
0
4)
40
60
80
-10
-20
Depth (um)
10
0.8
0.6
0
C
5
0
0.4
0
0.2
EU
0
C.
23
0
40
60
80
-0.2
-5
-0.4
-0.6
-10
-0.8
Depth (um)
-1
Fig 2.9. Signal improvement after AO compensation. (a) ~ (d) show histograms for number of
pixels according to
their intensity; x-axis represents intensity of the pixels. For example, 10 means the pixels
that have 0-10% intensity
of the maximum in the whole image, and 100 means 90-100% intensity pixels. Y-axis represents
the number of
pixels in the intensity range. The blue bars show the number of pixels before the compensation,
and red bars show
the result after the compensation. Each histogram was normalized to itself. (a) shows the distribution
at 20 pm
depth, (b) at 40 gm, (c) at 60 pm, and (d) at 80 gm imaging depth. (e) Percentage improvement according
to imaging
depth. Red line shows the improvement with background rejection (only fluorescent area was calculated),
and blue
line shows the improvement only with 90-100% intensity pixels. (f) Increment of mean photon count.
Red and blue
lines show same data sets as (e). (g) Improvement based on each pixel. Any pixel with more than
2 fold
36
improvement, that comes from mismatch of the uncompensated and compensated image, is saturated to red color.
The background is set to -1 for visualization purpose.
The compensated image shows higher signal strength than the uncompensated one. To compare the signal
intensity of two images, intensity distributions for all pixels were calculated. Fig 2.9(a) ~ (d) show the
histograms for the intensity distributions before and after the compensation. It was shown that the number
of brighter pixels increased, and the number of darker pixels decreased for all imaging depths after the
compensation, which means the in-focus fluorescence signals were improved while the out-of-focus
fluorescence signals were suppressed. To find the improvement trend according to imaging depth, mean
photon counts were calculated with background rejection. Fig 2.9(e) shows the percentage improvement
from the mean photon counts according to imaging depth. The percentage improvement shows that at
greater imaging depth, the improvement becomes larger, so the improvement for 90-100% intensity
pixels was 38.1% at 80 pm depth. Fig 2.9(f) shows the increment of mean photon count. In term of the
increment of photon count, the result at 40 pm depth shows the greatest improvement than at greater
depths. At shallow depth, there is not much to compensate and at greater depths the overall signal is
significantly lower due to scattering attenuation of both excitation and emission photons [30]. Fig 2.9(g)
shows the percentage improvement based on each pixel at 80pm depth calculated from Fig 2.8.
Comparing the degree of improvement on the center vs. the edge, no major difference is observed. From
all over the region where the fluorescence signal is generated, the improvement seems quite uniform. This
may be because the aberration from the optical system is much larger than the aberration from the sample.
2.3.4. Mouse heart muscle imaging using AO compensated two-photon microscopy
As a second example, we imaged mouse heart muscular structures with nuclei based on endogenous
fluorescence, second harmonic generation and exogenous labeling. Through tail-vein injection (in
conjunction with an overdose of anesthetics), nuclei are labeled with Hoechst while the extracellular
matrix around the cells were labeled with Texas-Red maleimide. After staining, the heart was excised and
37
was fixed in 4% paraformaldehyde and embedded in paraffin without a cover slip after some histological
processing. The wavelength of the excitation light was 780 nm. The objective lens was a 40x oil
immersion objective lens (Fluar, 1.3 NA, Zeiss, Thornwood, NY). The emission signal was filtered by a
green filter (535/40, Chroma Technology, Rockingham, VT). The field size was 120 pmx120 pim with
256x256 pixels and the dwell time was 40 gs. Fig 2.10 shows representative images of mouse heart at 80
pm depth with and without the adaptive optics compensation. The left image is the uncompensated image
and the right one is the compensated image. As in the mouse tongue experiment, adaptive correction was
only performed at the center pixel of the image and the same deformable mirror shape was held constant
throughout the whole image.
100
80
60
40
20
0
20pm
20pm
Fig 2.10. Mouse heart images at 80 im depth without / with AO compensation
To compare the signal intensity of two images, the intensity distributions for all pixels were calculated
again. Fig 2.11(a) ~ (d) show the histograms for the intensity distributions before and after the
compensation. The same trend was shown as the mouse tongue result; the number of brighter pixels
increased, and the number of darker pixels became less for all imaging depths after the compensation. To
38
find the improvement trend according to imaging depth, mean photon counts were calculated with
background rejection. Fig 2.11(e) shows the percentage improvement from the mean photon counts
according to imaging depth. The percentage improvement shows that as the imaging plane went deeper,
the improvement became larger in general. The maximum improvement of the 90~100% intensity pixels
was 123% at 80 pm depth.
In terms of the increment of photon count, the result at 20 gm depth showed the greatest increase due to
the generally higher signal level. At greater depth, scattering of excitation and emission photon again
dominated and the improvement due to using adaptive optics became less significant. In terms of
improvement on the center compared to edge, it seemed to be uniform similar to the mouse tongue result.
(a)
(b)
(e) 180
4M1
-
--
4MD
160
-
140
(c)
---10
10-- 20--
0
10
0
3M0 10
20
30
40
100
10
100
20
30
40
100
(d)
(C)
120
80
-- I
60
40
20
MW10 --
MM
0
-- ,----10
20
30
40
100
20
10
20
30
40
40
60
80
Depth (um)
100
Fig 2.11. Signal improvement after AO compensation. (a) ~ (d) show histograms for number of pixels according to
their intensity; x-axis represents the intensity of pixels same as Fig 2.9 except 100 (100 means 40-100% intensity
pixels in the image). Y-axis represents the number of pixels in the intensity range. The blue bars show the number of
pixels before the compensation, and red bars show the result after the compensation. Each histogram was
normalized to itself. (a) shows the distribution at 20
sm depth, (b)
at 40 pm, (c) at 60
sm, and (d)
at 80 gm imaging
depth. (e) Percentage improvement according to imaging depth. Red line shows the improvement with background
rejection (only fluorescent area was calculated), and blue line shows the improvement only with 90-100% intensity
pixels.
39
2.3.5. Neuronal imaging in mouse brain slices using AO compensated two-photon microscopy
Both muscle tissues have high scattering coefficients that appear to be the major limiting factor in twophoton imaging depth. It is interesting to also study tissue specimen that has significantly lower scattering
and allows deeper imaging. For this propose, we chose mouse brain slices with neurons expressing GFP
protein driven by thy-i promoter. Adult thyl-GFP-S mice were perfused transcardially with 4%
paraformaldehyde in PBS. Brains were removed and postfixed overnight in 4% paraformaldehyde in
PBS, then coronally sectioned at 200 ptm using a vibratome. The wavelength of the excitation light was
890 nm, and a 40x water immersion objective lens (Achroplan IR, 0.8 NA, Zeiss, Thornwood, NY) was
used with No. 1/
2
cover slip. The emission signal was filtered by a green filter (535/40, Chroma
Technology, Rockingham, VT). The field size was 120 gmx120 [Lm with 256x256 pixels and the dwell
time was 40 ps.
Fig 2.12 shows representative mouse brain images at 50 pm depth with and without the adaptive optics
compensation. The mouse brain specimen contained sparsely distributed gfp expressing neurons. The left
image is the uncompensated image and the right one is the compensated image. The adaptive optics
compensation was performed at the center of the whole view and scanning was done with the same
deformable mirror shape. The compensated image showed higher signal than the uncompensated one. To
compare the signal level of two images, the intensity distributions for all pixels are shown in Fig 2.13. Fig
2.13(a) and (b) show the histograms for the intensity distributions before and after the compensation.
Since the structure imaged is sparse, the histogram is dominated by the zero intensity pixels. Therefore,
two additional figures below (a) and (b) are added to show histogram distributions excluding the lowest
intensity range.
40
35
30
-
25
20
15
10
5
0
20pm
20pm
Fig 2.12. Mouse brain images at 50 pm depth without / with AO compensation
Again, the number of brighter pixels increased and the number of darker pixels decreased after the
adaptive optics compensation. For the improvement trend according to imaging depth, mean photon
counts were calculated with background rejection. Fig 2.13(c) shows the percentage improvement from
the mean photon counts according to imaging depth. We see a general trend of increasing percentage
improvement as a function of imaging depth similar to the other two tissue cases. Since the scattering
coefficient in the brain is much less than that of muscles, the achievable imaging depth is deeper with
equivalent excitation power. However, the improvement at the maximum achievable depth between with
and without adaptive optics compensation tissues was similar in all cases ranging up to about 2 fold in
terms of the maximum signal strength. In a previous report, fluorescently labeled blood plasma in wildtype zebrafish was imaged [18]. The aberrations introduced by the specimen were mainly astigmatism
(254nm) at 200pm imaging depth and fluorescence signal at the center of a blood vessel was improved
almost 2 fold by the wavefront correction with the coherence-gated wavefront sensing method. In our
measurement, the main aberration was also astigmatism (-125nm) at 150 m imaging depth and the signal
improvement was about the same, which was comparable with the previous result.
41
(a)
4M
(b)
-
-
(c) 160
--
140
12A
310 '1
100
0
7r
60
400
800j
300
600
200
400
-
200
100
0
40
ht-
0
-
20
20
-
100
50
0
i
ca
SS
So
r- C
O0
150
Depth (um)
Fig 2.13. Signal improvement after AO compensation. (a) and (b) show histograms for number of pixels according
to their intensity; x-axis represents intensity of the pixels, and y-axis represents the number of pixels in the intensity
range. The blue bars show the number of pixels before the compensation, and red bars show the result after the
compensation. Each histogram was normalized to itself. (a) shows the distribution at 50 pm depth, and (b) at 150 pm
imaging depth. The bottom figures of (a) and (b) are detailed distributions in the selected range. (c) Percentage
improvement according to imaging depth. Red line shows the improvement with background rejection (only
fluorescent area was calculated), and blue line shows the improvement only with 90~100% intensity pixels.
2.4. Conclusion
Wavefront distortion by tissue specimens with inhomogeneous optical properties can be measured by a
Shack-Hartmann wavefront sensor using a confocal depth selection mechanism. The measured wavefront
distortion can then be corrected by a deformable mirror integrated into the microscope system. In
specimens with large aberration, such as the artificial specimens, significant improvement in signal
strength and resolution can be achieved. In more realistic tissue specimens, the imaging depth is limited to
100-200
sm depth
by the respective scattering coefficients of the respective tissue type. For example, in
general, achievable imaging depth for the brain tissue is approximately three times that of muscles. The
measured aberration for these tissues at this depth is only on the order of a few
42
/1 0 similar to those
reported by other investigators [18, 31]. The corresponding improvement in terms of signal strength is on
the order of 20-70%. However, the improvement on the peak signals (90-100% strength intensity
signals) is up to 2 fold. This means the contrast of images was increased after the adaptive optics
compensation, therefore the images became more vivid, which is one of the most essential roles of the
adaptive optics system in biological imaging.
Our data appear to imply two related conclusions. First, tissue scattering is the primary factor that
determines the maximum imaging depth in tissue imaging by two-photon microscope. While the effect of
scattering is exponential with depth, the accumulated aberration as a function of depth appears to remain
at the same order of magnitude or vary at most linearly. Therefore, scattering effect is always the
dominant factor. Second, adaptive optics control can improve imaging signal at depth. However, the
typical improvement is relatively modest especially for highly scattering specimen because the very
limited imaging depth implies relative little accumulated aberration limiting the improvement that can be
obtained by the use of adaptive optics correction. These conclusions are consistent with most of the
adaptive optics compensated nonlinear optical imaging studies. The only exceptions are the application of
adaptive optical compensation in CARS microscope. One potential explanation may be that CARS signal
is highly dependent on the overlap of the excitation volumes of the pump and probe beams and aberration
may results in spatial mismatch. The correction for aberration may optimize volume overlap and results in
the observed six fold improvement [11].
It may be then important to ask if adaptive optics correction is important for two-photon microscopy. Our
data seems to indicate that for most tissue types with high scattering coefficient and shallow imaging
depth, adaptive correction can improve signal strength. However, one should note that if excitation power
is not limited and photodamage is not an issue, this level of improvement can be equivalently achieved by
less than two fold excitation power increase. In the case that higher excitation power cannot be applied,
adaptive optics may provide some improvement although at the expense of an increase in instrument
complexity and cost.
43
It may also be useful to consider what situations adaptive optics correction would be useful for twophoton imaging in. Clearly, in tissue with relatively low scattering coefficients that allows deep imaging,
adaptive optics is useful to correct for the larger aberration present in this case. This is of course the well
known case with the classic examples of imaging in organs such as the eye [4, 32, 33]. Other potential
application areas are tissues that induced unusually high aberrations. Another classic example is the skin
where the stratified layers have very different indices of refraction and results in significant aberration
even in relatively shallow depth. It would be interesting to examine the possible application of adaptive
optics compensation in skin imaging. In addition, there are several other potentially interesting
applications of adaptive optics in nonlinear microscopy imaging of tissues.
First, tissue scattering, due to a combination of Rayleigh and Mie processes, is lower for light at longer
wavelength region, which corresponds to deeper penetration depth. If the adaptive optics system is
combined with a two-photon microscope operating in 1.2-1.3
im region and imaging near infrared
emitting fluorophores, one may achieve deeper imaging in tissues with corresponding higher aberration
effect. In this case, one may expect adaptive optics compensation to be more important.
Second, if the scattering effect becomes negligible, then the aberration due to the inhomogeneous
refractive index of a sample becomes the major factor for image degradation. One obvious example in
specimen with extensive optical clearing that can dramatically decreases tissue scattering by dehydration
of the specimen and the addition of index matching medium [34, 35]. Optical clearing reduces scattering
by index matching nanometer scale level inhomogeneity in the specimen. With the optical clearing, the
imaging depth can clearly be increased by orders of magnitude. Given the deeper imaging depth,
potentially adaptive optics system may play a more important role provided that this optical clearing do
not similarly eliminate index of refraction inhomogeneity on the optical wavelength scale.
Finally, the adaptive optics system may be important for a number of imaging applications involving
microfluidic devices. In the fabrication of many microfluidic devices, the material type or thickness used
in the fabrication of the device may not match the requirement of the microscope objectives that are
44
typically designed for imaging through 0.17 mm thick glass. In these cases, significant aberrations may be
induced and the use of an adaptive optics system may result in an improvement in image quality in these
devices.
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
W. Denk, J. H. Strickler, and W. W. Webb, "2-photon laser scanning fluorescence microscopy," Science
248, 73-76 (1990).
P. T. C. So, C. Y. Dong, B. R. Masters, and K. M. Berland, "Two-photon excitation fluorescence
microscopy," Annual Review of Biomedical Engineering 2, 399-429 (2000).
F. Rigaut, G. Rousset, P. Kern, J. C. Fontanella, J. P. Gaffard, F. Merkle, and P. Lena, "Adaptive optics on
a 3.6-M telescope - results and performance," Astronomy and Astrophysics 250, 280-290 (1991).
J. Z. Liang, D. R. Williams, and D. T. Miller, "High resolution imaging of the living human retina with
adaptive optics," Investigative Ophthalmology & Visual Science 38, 55-55 (1997).
M. J. Booth, M. A. A. Neil, and T. Wilson, "Aberration correction for confocal imaging in refractive-indexmismatched media," J. Microsc.-Oxf. 192, 90-98 (1998).
M. J. Booth, M. A. A. Neil, R. Juskaitis, and T. Wilson, "Adaptive aberration correction in a confocal
microscope," Proc. Natl. Acad. Sci. U. S. A. 99, 5788-5792 (2002).
M. A. A. Neil, R. Juskaitis, M. J. Booth, T. Wilson, T. Tanaka, and S. Kawata, "Adaptive aberration
correction in a two-photon microscope," J. Microsc.-Oxf. 200, 105-108 (2000).
P. N. Marsh, D. Burns, and J. M. Girkin, "Practical implementation of adaptive optics in multiphoton
microscopy," Opt. Express 11, 1123-1130 (2003).
P. Villoresi, S. Bonora, M. Pascolini, L. Poletto, G. Tondello, C. Vozzi, M. Nisoli, G. Sansone, S. Stagira,
and S. De Silvestri, "Optimization of high-order harmonic generation by adaptive control of a sub-10-fs
pulse wave front," Opt. Lett. 29, 207-209 (2004).
A. Jesacher, A. Thayil, K. Grieve, D. Debarre, T. Watanabe, T. Wilson, S. Srinivas, and M. Booth,
"Adaptive harmonic generation microscopy of mammalian embryos," Opt. Lett. 34, 3154-3156 (2009).
A. J. Wright, S. P. Poland, J. M. Girkin, C. W. Freudiger, C. L. Evans, and X. S. Xie, "Adaptive optics for
enhanced signal in CARS microscopy," Opt. Express 15, 18209-18219 (2007).
0. Albert, L. Sherman, G. Mourou, T. B. Norris, and G. Vdovin, "Smart microscope: an adaptive optics
learning system for aberration correction in multiphoton confocal microscopy," Opt. Lett. 25, 52-54 (2000).
L. Sherman, J. Y. Ye, 0. Albert, and T. B. Norris, "Adaptive correction of depth-induced aberrations in
multiphoton scanning microscopy using a deformable mirror," J. Microsc.-Oxf. 206, 65-71 (2002).
N. Ji, D. E. Milkie, and E. Betzig, "Adaptive optics via pupil segmentation for high-resolution imaging in
biological tissues," Nat. Methods 7, 141-U184.
S. P. Poland, A. J. Wright, and J. M. Girkin, "Evaluation of fitness parameters used in an iterative approach
to aberration correction in optical sectioning microscopy," Appl. Optics 47, 731-736 (2008).
A. Leray and J. Mertz, "Rejection of two-photon fluorescence background in thick tissue by differential
aberration imaging," Opt. Express 14, 10565-10573 (2006).
M. Feierabend, M. Rueckel, and W. Denk, "Coherence-gated wave-front sensing in strongly scattering
samples," Opt. Lett. 29, 2255-2257 (2004).
M. Rueckel, J. A. Mack-Bucher, and W. Denk, "Adaptive wavefront correction in two-photon microscopy
using coherence-gated wavefront sensing," Proc. Natl. Acad. Sci. U. S. A. 103, 17137-17142 (2006).
T. Wilson and A. R. Carlini, "Size of the detector in confocal imaging-systems," Opt. Lett. 12, 227-229
(1987).
D. R. Sandison and W. W. Webb, "Background rejection and signal-to-noise optimization in confocal and
alternative fluorescence microscopes," Appl. Optics 33, 603-615 (1994).
T. R. Corle and G. S. Kino, Confocal scanning optical microscopy and related imaging systems (Academic
Press, San Diego, 1996).
45
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
P. Artal, S. Marcos, R. Navarro, and D. R. Williams, "Odd aberrations and double-pass measurements of
retinal image quality," J. Opt. Soc. Am. A-Opt. Image Sci. Vis. 12, 195-201 (1995).
P. Artal, I. Iglesias, N. Lopezgil, and D. G. Green, "Double-pass measurements of the retinal-image quality
with unequal entrance and exit pupil sizes and the reversibility of the eyes optical-system," J. Opt. Soc.
Am. A-Opt. Image Sci. Vis. 12, 2358-2366 (1995).
M. J. Booth, "Adaptive optics in microscopy," Philos. Trans. R. Soc. A-Math. Phys. Eng. Sci. 365, 28292843 (2007).
S. Hell, G. Reiner, C. Cremer, and E. H. K. Stelzer, "Aberrations in confocal fluorescence microscopy
induced by mismatches in refractive-index," J. Microsc.-Oxf. 169, 391-405 (1993).
P. T. Fwu, P. H. Wang, C. K. Tung, and C. Y. Dong, "Effects of index-mismatch-induced spherical
aberration in pump-probe microscopic image formation," Appl. Optics 44, 4220-4227 (2005).
H. P. Kao and A. S. Verkman, "Tracking of single fluorescent particles in 3 dimensions - use of cylindrical
optics to encode particle position," Biophys. J. 67, 1291-1300 (1994).
T. Ragan, H. D. Huang, P. So, and E. Gratton, "3D particle tracking on a two-photon microscope," J.
Fluoresc. 16, 325-336 (2006).
J. M. Girkin, J. Vijverberg, M. Orazio, S. Poland, and A. J. Wright, "Adaptive optics in confocal and twophoton microscopy of rat brain: a single correction per optical section - art. no. 64420T," in Conference on
Multiphoton Microscopy in the Biomedical Sciences VII, (Spie-Int Soc Optical Engineering, 2007), T4420T4420.
P. Theer and W. Denk, "On the fundamental imaging-depth limit in two-photon microscopy," J. Opt. Soc.
Am. A-Opt. Image Sci. Vis. 23, 3139-3149 (2006).
D. Debarre, E. J. Botcherby, T. Watanabe, S. Srinivas, M. J. Booth, and T. Wilson, "Image-based adaptive
optics for two-photon microscopy," Opt. Lett. 34, 2495-2497 (2009).
N. Doble, G. Yoon, L. Chen, P. Bierden, B. Singer, S. Olivier, and D. R. Williams, "Use of a
microelectromechanical mirror for adaptive optics in the human eye," Opt. Lett. 27, 1537-1539 (2002).
A. Roorda, F. Romero-Borja, W. J. Donnelly, H. Queener, T. J. Hebert, and M. C. W. Campbell, "Adaptive
optics scanning laser ophthalmoscopy," Opt. Express 10, 405-412 (2002).
V. V. Tuchin, Optical clearing of tissues and blood (SPIE Press, 2005).
H. U. Dodt, U. Leischner, A. Schierloh, N. Jahrling, C. P. Mauch, K. Deininger, J. M. Deussing, M. Eder,
W. Zieglgansberger, and K. Becker, "Ultramicroscopy: three-dimensional visualization of neuronal
networks in the whole mouse brain," Nat. Methods 4, 331-336 (2007).
J. W. Cha, J. Ballesta, and P. T. C. So, "Shack-Hartmann wavefront-sensor-based adaptive optics system
for multiphoton microscopy." J. Biomed. Opt. 15, 046022 (2010).
46
Chapter 3
Reassignment
of
Scattered
Emission
Photons
in
Multifocal
Multiphoton Microscopy
3.1. Introduction
Multiphoton excitation fluorescence microscopy has inherent 3D resolution due to the nonlinear
dependence of excitation upon the incident light distribution [1, 2]. Excitation region is localized to a
femtoliter volume at the focal point of a high numerical aperture objective lens. Multiphoton excitation
fluorescence microscopy is routinely used in a variety of tissue imaging applications due to its excellent
imaging depth, high resolution, and lower photo-damage. However, one of the practical limitations of
multiphoton excitation fluorescence microscopy is its imaging speed, typically up to a few frames per
second. While this imaging speed is sufficient in many cases, several classes' applications require higher
imaging speed. These applications include the measurement of dynamic processes including calcium
signaling and action potential propagation [3], high throughput image cytometric study of tissue
physiology [4-6],
and clinical applications where the effects of physiological motion should be
minimized.
There are several methods which have already been introduced for the high-speed multiphoton imaging.
The first approach is based on using high-speed scanners such as polygonal mirrors [7], resonant mirror
scanners [8], or acousto-optical deflectors (AODs) [9-12] instead of galvanometric mirror scanners used
in conventional multiphoton microscopes. The high speed scanners can typically achieve the scanning
speed up to about 30 frames per second in tissues with a comparable imaging depth as conventional
multiphoton microscopy. However, the high speed scanning requires decreasing pixel dwell time resulting
47
in lower image contrast and poorer SNR. This can be partially compensated by increasing the excitation
laser power, but it is not always possible due to the photo-damage on a specimen and excitation saturation
[13, 14].
The second method is two-photon wide-field imaging [15, 16]. With the temporal focusing method, twophoton excitation plane is generated instead of a focus, and it realizes scanningless plane by plane
imaging resulting in the fastest imaging speed. However, its performance is often limited by the lower
axial resolution compared to the conventional multiphoton microscopy and blurring in large depth of
tissue imaging due to the scattering of emission photons.
The third method is multifocal multiphoton microscopy (MMM), which is an intermediate approach
between the previously described two techniques [17-22]. With a lenslet array or diffractive optical
element (DOE) [23, 24], a number of foci are generated simultaneously and scanned together. The size of
the whole scanning region corresponds to the sum of the sub-region scanned by each focus. Therefore, the
imaging speed can be improved proportionally to the number of foci. One practical limitation for the
imaging speed of MMM is available laser power. For typical titanium-sapphire laser with several Watts of
output, approximately one hundred foci can be effectively generated for tissue imaging resulting in
approximately two orders of magnitude improvement in imaging speed.
Similar to two-photon wide-field imaging, the SNR of multifocal multiphoton microscopy is also limited
by the scattering of emission photons in tissues resulting in lower SNR compared to the single focus
scanning multiphoton microscopy. The parallelized excitation foci and simultaneous detection requires
detectors with spatial resolution that can acquire and distinguish signals from all the foci at the same time.
Most multifocal multiphoton microscopes use imaging detectors such as CCD or CMOS cameras.
However, when emission photons generated at one focus are scattered in turbid specimen and arrive at the
locations of detector surface that record signals from other foci, the SNR of the image suffers. This lower
SNR can be compensated by the use of multianode photomultiplier tubes (MAPMT) in descanned
detection geometry where each anode of the MAPMT detects signal for each focus [25]. The larger
48
detection area of the each anode greatly reduces the crosstalk between foci due to the scattering of
emission photons and significantly improves the image SNR.
However, with severe scattering in a turbid specimen, especially in large imaging depth, the scattered
emission photons still arrive at neighbor anodes and result in ghost images, duplicates of an image
acquired by one focus visualized in neighbor sub-images. These scattered emission photons should be
reassigned to their original pixels to remove the ghost images and to increase the SNR of the original
image. We have shown that this can be accomplished by estimating the scattering matrix at one location
of the specimen that characterizes the crosstalk between the anodes of the MAPMT due to emission
photon scattering. By applying the inverse of the scattering matrix to the acquired image, an improved
SNR picture free from ghost images can be produced.
The estimation of the scattering matrix has been done experimentally [25], [Fig 3.1]. The first method of
the estimation is using a tissue phantom with similar optical parameter as real tissue containing sparsely
distributed fluorescence particles. These fluorescence particles are imaged by the MMM. The sparsity of
the particles ensures that the ghost images of each particle can be identified. Through calculating the
relationship between particle intensities in the correct pixel and all its neighbors, the scattering matrix can
be calculated. However, the phantom sample is often a poor model of real tissue specimen on the
microscopic level where there is significant heterogeneity. This may lead to a significant error in image
post-processing. Second, the scattering matrix can be calculated with a real sample by exciting only one
focus at a time in the same MMM system. This method gives the exact coefficients for the scattering
matrix. However, since the scattering matrix is a function of specimen location due to heterogeneity and
depth due to increased scattering, scattering matrix must be measured frequently. This labor intensive
measurement process partly negates the speed advantage of applying MMM imaging.
49
(a)
(b)
Fig 3.1. Two experimental approaches for the estimation of a scattering matrix. (a) Sparsely distributed beads
sample in tissue phantom (2% Intralipid emulsion), and scattering matrix measurement from its MMM image. (b)
Single focus excitation on a real sample and scattering matrix measurement from its MMM image.
Here we demonstrate pure image post-processing approach for MMM to estimate the scattering matrix
without any additional experimental measurement. The proposed approach is based on maximum
likelihood estimation for the quantification of the crosstalk between the different anodes of the MAPMT
with utilizing the physical model of the MMM system. This approach provides the reassignment of the
photons that are originally detected at wrong anodes and generated in the form of the ghost images, to the
actual anode location. Therefore, it provides the improvement in the signal strength and simultaneously
the minimization of the ghost images in the neighbor sub-images. Since the proposed approach uses only
the primary images generated by the MMM, this allows the characterization of the scattering matrix that
is independent of any specimen imaging, location and depth. Mathematical model of the proposed
approach is presented and validated first with simulation data where the results show the convergence of
the method to the actual fluorophores structures. The performance of the proposed method is then
employed for the mouse neuron images where the results are quantitatively analyzed.
50
3.2. Methods
3.2.1. MMM configuration
We have developed two MMM systems; one with 45 gm foci separation, and the other with 85 jm foci
separation. Fig 3.2 shows the schematic of the MMM with 85
sim foci
separation based on a diffractive
optical element (DOE) for multiple foci generation and the MAPMT in descanned detection geometry.
The light source used is Chameleon Ultra II (Coherent, Santa Clara, CA). The excitation laser beam is
expanded and illuminates the 8
x
the excitation wavelengths (8
8 for 780 nm and 4
x
8 or 4
x
4 DOE (customized, Holo/Or, Rehovoth, Israel) depending on
x
4 for 910 nm). The beamlets are shrunk according
to the size of x-y galvanometric mirror scanners (6215H, Cambridge Technology, Lexington, MA), and
overlapped on the scanning mirrors. The beamlets reflected by the scanners are expanded again to fill the
back focal plane of a 20
x
water immersion objective lens with 1.0 NA (W Plan-Apochromat, Zeiss,
Thornwood, NY), and enter the aperture with different entrance angles. The objective lens generates an
array of 8
x
8 excitation foci on the focal plane in a specimen. The image is formed by raster scanning of
the excitation foci across the specimen using the scanning mirrors. The excitation foci are separated each
other by 85 pm, and each focus scans slightly larger than the area of 85 pm
for montage. The whole scanning size covered by the 8
x
x
8 foci is 680 jim
85 pm in the sample plane
x
680 jm with the 85 pm
separation. The emission photons are collected by the same objective lens and descanned by the scanning
mirrors. The descanned emission beamlets become stationary regardless of the scanning. The emission
beamlets are reflected by a dichroic mirror (Chroma Technology, Bellows Falls, VT) toward the MAPMT
and are focused onto the center of each corresponding anode of the MAPMT (H7546B-20, Hamamatsu,
Bridgewater, NJ). An IR blocking filter (BG39, Chroma Technology, Bellows Falls, VT) and a short-pass
filter (ET680sp-2p, Chroma Technology, Bellows Falls, VT) is installed before the MAPMT to block the
excitation light. The other MMM system with 45 pm foci separation has very similar system
configuration, but it uses a different model's Ti-Sapphire (Ti-Sa) laser (Tsunami, Spectra-Physics,
Mountain View, CA) pumped by a continuous wave Nd:YVO 4 laser (Millennia, Spectra-Physics,
51
Mountain View, CA) as a light source, the 8
8 micro-lenses of a square lenslet array (1000-17-S-A,
x
Adaptive Optics, Cambridge, MA), and a 0.95 NA objective lens (XLUMPLFL20XW, Olympus,
Melville, NY). The 45 pm foci separation in a specimen provides total 360 gm
Scan lens
x
360 pm imaging size.
Tube lens
Scanning
mirror
Mirror
Multianode
PMT
rror
Objective LensZN
Foci at
sample
+- DOE
(generating8x8 beamlets)
Fig 3.2. Schematic of MMM based on the MAPMT.
3.2.2. Image reconstruction methodology
Let O(x,y,z) be the fluorophores distribution in the specimen assuming constant irradiance per unit
volume with the specimen. For two-photon point excitation at scan position i', the excitation intensity
distribution at the specimen projected by an objective with a normalized 3D intensity PSF h(i) is
E(i,i') = EOc5(i -
') 0 h(i) = Eoh(-')
. Here E0 is the maximum intensity and the symbol, 0 ,
represents the 3D convolution operator. The fluorescence intensity generated at the specimen due to twophoton process is:
F(i,i')= E02h
2
(
)*
(i)
(1)
The effect of emission photon scattering can be modeled by a more general form of the emission PSF
hm (i), assuming
h,,, ()
=
h(i) in the absence of scattering and h'(i) represents the PSF for the scattered
emission photon.
h,,,()
=
(1-a)h(i)+ah'(i)
(2)
52
Without emission scattering effect, a is zero. In general [25], even in the presence of scattering, the
amplitude of the modification term is generally small, i.e., a < 1. Also, it has been observed that h'(i)
typically has full-width-at-half maximum that is orders of magnitude larger than that of h(i) [25].
The intensity distribution at the image plane in epi-detection geometry is:
I(x, y, i')
=
[E2h 2 (i
- R'). 0(i)]
0 hm (i)z=O
(3)
For the MMM system, let N x N equally spaced foci be arranged on a rectilinear grid with reference
positions {101,.....2,N}, where c0,, = {mA, nA,O), m and n are the foci indices, and A is the separation
distance between foci. In order to generate an image using MMM, this grid of foci must be scanned to
cover each A x A sub-regions for the corresponding axial plane. The signal for an MMM system at
location (x, y) of an imaging sensor (CCD or CMOS) at any scan location (i, j, k) can be written as,
N,N
Im(x,y,i,j,k)=
N,N
I(x,y,i n, +X'i,
,k)=
m,n
Z{[E h 2
_n
i )'
O(
+ X'k )]0hm(i,k)}jzO
(4)
m,n
For a given z-plane, k, using an imager (such as a CCD or CMOS camera with sensor size M x M) that
integrates the signal for all the lateral scan steps, the final single plane image can be written as:
M,M
I
(x,y,k)=
M,MN,N
I
1,]
{[E h2 (i _in
(x,y,i, j, k)
Q,
i',j )-o(i+
'k )]®h,(, k)}|zgo
(5)
m,n
It is clear that in this case, the final image is "blurred" by the emission PSF and SNR is degraded. Instead
of using an imager such as a CCD, we can also use a multianode PMT. By doing so there are two main
differences [25]; (i) The image is descanned in the multianode PMT to ensure that the foci are center at
each anode of the PMT, and (ii) The signal at each anode is integrated at each scanning step.
53
NN
IMD (x, y,0
k)
{ [E2h2 (
=
_
i1,'). O(i +
-
"k
)] S h, (i, k)}
z=,x=x+iA M,y=y+jA/M
m,n
NN
= (Jdx"dy" dz'"E h 2 (x"-mA -
iA / m, y"-nA - jA / M, z")
m,n
-O(x"',
y" , zI"+kA')hn (x
x,
-
y
-
yz
-
z,
k)
zOxx+iA/Myy+jA/M
N,N
dx "dy" dz"E2h 2 (x"-m
-iA / M, y"-nA - jA / M, z")
m,n
O(x", y", z +kA')hm (x + iA / M
-
x",
y + jA / M -
',-z",
k)
(6)
Therefore, within these limited domains, since h'(i) is broad and smooth on the length scale of A , the
integrand of the integral may be replaced by a set of constant values C(ab),(mn),k that are related to the
radially integrated PSF (Ho) of the scattered light. Here (a,b) are the anode index and (m,n) are the foci
index. The C matrix is depth dependent, specified by k for the given z-plane. Considering that the
emission PSF can be further separated into a part taking into account of scattering, Eqn. (6) can now be
rewritten as,
IMD(a,b, i, j,k) = (1- a(k))E H f dx"dy"dz'h 2 (x",y", z') -O(x"+aA + iA /M,y"+bA + jA/M, z'+kA')
N,N
+ a(k)EI
C(a,>m,n),k
dx"dy"dz'h 2(x", y", z')- O(x"+mA + iA / M, y"+nA + jA /M, z'+kA')
m,n
(7)
Then equation (7) can be further simplified as:
N,N
IMD (a,b, i, j,k) = (1 - a(k))HOI(a,b, i, j, k) + a(k) I
C(ab),(mn),kI(m, n, i, j, k)
mn
(8)
54
The detection process consists of measured photon count by each anode (a,b) at each scan location (i, j,
k), i.e. N(a,b,ij,k). N(a,b,ij,k) should follow Poisson statistics with a mean given by Im(a,b,i,j,k).
2.2.1 Maximum Likelihood Estimation
The log-likelihood function of this readout process, described in eqn. (8), can be written as.
is =
I[N(a,b, i, j,k) lnIm(a,b, i, j, k) -IM(a,b, i, j,k)]
(9)
a,b,i,j,k
The log-likelihood function utilizes the physical modeling of the MMM for the maximum likelihood
estimation of specimen fluorophores distribution. Our approach provides maximization of log-likelihood
function, thus provides the quantification of scattering matrix that is used for the reassignment for the
image photons to the correct spatial location, which are originally recorded as ghost image photons by the
neighboring anodes of MAPMT. Therefore, photon reassignment can be casted as a blind deconvolution
problem that seek to recover the image I ,(a,b,i,j,k)
and the mixing kernel, C(ab),(mn),k given
N(a, b, ij, k,a)for a emission scattering condition.
The original fluorophores distribution at the corresponding axial depth recorded by the MMM system can
be considered as the 'first guess' for the iteration process and the maximization of log-likelihood function
can be performed using numerical optimization method which iteratively improves the strength of the
actual signal by reassignment of ghost-image photons to correct spatial location. We adopted Newton's
method for maximizing the log-likelihood function and the iteration step for maximum likelihood
estimation of fluorophores distribution can be written as:
(i)
Maximize the likelihood function with respect to I(a, b, i, j, k) to get a better estimation for this
variable:
55
EVlS(I, HO, C, a,k)
aI(a,b,i,j,k) JI(a,bjij,k)=P()(a,bji,j,k)
(kl)(a,b, i, j,k)= I ()(, b, i, j, k).
(10)
a'2l,(I,Ho,C,a,k)
a12(a,b, i, j,
k) J I(a,bjj,,k)=j(k)(a,b,i,j,k)
(ii) Maximization of Likelihood function with respect to C to get a better estimation for this variable:
Ns,(I, HO, C, a,k)
aC(a,b),(m,n),k
_ (k)
5(k+1)
(a,b),(m,n),k -- (a,b),(m,n),k
c
(k)
(11)
(ab),(m,n),k
(a,b),(mn),k
(iii) Maximization of Likelihood function with respect to a to get a better estimation for this variable:
al, (I,HO, C, a,k)
(k+1)
_
{2l,(I,
,
(k)
a k a=(k)
HO, C, a,k)
0
8a
Here
j(k)
(),
and
2
j(k)
(12)
a=i(ik)
are the estimated fluorophores distribution, scattering matrix coefficients,
emission scattering coefficient respectively at
new estimate of
C(k+1),
5(k+1), and
i(k+1)
kth
iteration step. The maximization process updates the
for iterative step k. The log-likelihood function is evaluated at
each iteration step for maximum likelihood estimation of fluorophores distribution. The accuracy of this
estimation partly depends on the initial estimates chosen and on the constrained parameters, i.e., positivity
of the fluorophores. Fig 3.3 provides the summary of the proposed photon reassignment algorithm for
MMM system.
56
MMM image
IMD(a, ,ij,k)
Cleaner Image
(first guess)
f() (a, b, i,j, k)
Scattering coefficients
(first guess)
ab),
)(t)(k)
.k
Emission scattering
coefficient (first guess)
Number of
photons
N(a, b, ,, k)
Log-likelihoodfunction
is(t), e(t), a(t(k),N)
Cleaner image
(updated)
1(t+'(ab,,k)
Scattering coefficients
(updated)
t(t+0(),(n~n)k
Emission scattering
coefficient (first guess)
a(+ 1)(k)
Log-likelihoodfunction
(updated)
Is (14+1), &+), a(t+')(k), N)I
No
Convergence j
Yes
.top
S s
Fig 3.3. Summary of the photon reassignment process for MMM
3.3. RESULTS
3.3.1. Simulation results
3.3.1.1. 4 Spots in a 2 by 2 MMM image
To see the feasibility of our approach, we started from the simplest case using simulations. We simulated
a replica of 4 beads image in a 2 by 2 MMM [Fig 3.4 (a)-(b)]. First, Fig 3.4 (a) showed the original image
of 4 beads without ghost images, which would be the target of our simulation result. Then, for the MVM
image, one sub-image contained one primary bead image with three ghost bead images, and there was no
overlap between the beads for simplicity, as shown in Fig 3.4 (b). Any scattering-simulated sub-image
57
contained 100% of the intensity of the primary bead, 20% of the intensity of the adjoining beads, and 14%
of the intensity of the diagonal bead. We added Poisson noise in the simulated image by using the
'imnoise' function of MATLAB (MathWorks, Natick, MA). The proposed method requires the first guess
of the cleaner image P(1)(ab,i,jk), scattering matrix coefficients Cb),(mn),, and emission scattering
coefficient
. We chose the actual MMM image as the first guess for the cleaner image. In order to
keep the proposed method adaptive, and to automatically generate the first guess for scattering matrix
coefficients and emission scattering coefficient we have used the following steps:
(i)
Identify the spatial position of the pixel (and its MAPMT anode) that contains the maximum intensity
in the entire MMM image.
(ii) Observe the intensities at the pixels corresponding to same spatial location of all the neighboring
anodes identified in step (i).
(iii) Generate the intensity ratios of the maximum value pixel to neighboring anode pixels.
(iv) Repeat the steps (i)-(iii) for a few more values next to the maximum intensity, if possible, and take
the mean of the all these values.
(v) More coefficients of the scattering matrix for the other anodes are calculated in the same manner
described in the previous steps. All these calculated coefficients are used for the first guess of the
scattering matrix. In order to conserve the photon count, the sum of each row of scattering matrix
(representing one anode under observation and its contribution in all other anodes of MAPMT) is
normalized to 1.
(vi) The mean value observed in step (iv) is also considered as the first guess of emission scattering
coefficient i.e. a^(1 .
58
Original
aMMM
Processed
200
100
(a)
10
(b)
300
200
100
(d)
(e)
()
Fig 3.4. (a) Original 4 beads image in a 2 by 2 MMM. (b) A scattering-affected image of (a); one sub-image
contains one primary bead image and three ghost bead images with the specified proportions. (c) The processed
image of (b). (d) Original alphabet image in a 2 by 2 MMM. (e) A scattering-affected image of (d) in the same
fashion. (f) The processed image of (e).
We process the simulated MMM image by initializing of the parameters as discussed above. The iteration
process, as shown in Fig 3.3, started to maximize the log-likelihood function and this also performed
reassigning the scattered emission photons of the ghost images to the original spatial locations. During
iterative process we defined two constraints; first total number of photons of the MMM images is
constant before and after the processing, and second the positivity of the pixel values (as the fluorescence
signal cannot be negative). The reconstruction process took around 10 iterations and Fig 3.4 (c) shows the
processed image. It can be visually observed that all the ghost images are completely suppressed in the
final reconstructed image. It can be clearly observed that the intensity of processed beads image is
increased, and this represents the increase in the photon count of the beads location came from the ghost
image locations. More quantified results for the improvement of the signal strength and signal-to-ghost
image ratio (SGR) will be presented later in this paper.
59
3.3.1.2. Alphabets in a 2 by 2 MMM image
As a next simulation, an alphabet image was examined with our method. Compared to the simulated
beads image, the alphabet image has certain shapes and overlaps that provide more complexity. Again, an
original image without ghost images was generated [Fig 3.4 (d)], which would be our target. Then, ghost
images were imposed on all sub-images in the way that one sub-image in a 2 by 2 MMM image contains
one primary alphabet image with the ghost images of the other three alphabets [Fig 3.4 (e)]. The one
scattering-affected sub-image contains 100% of the intensity from a primary sub-image, 20% of the
intensity from the other neighbor sub-images, and 14% of the intensity of the diagonal image. The final
processed image, shown in Fig 3.4 (f) showed cleaner reconstruction of alphabets similar to the original.
Clearly, the effects of ghost images are significantly suppressed from the MMM images by using the
proposed approach. To start the iteration process the first guess of the scattering matrix and scattering
coefficient was calculated same as for the simulated bead image. Again, as discussed before, the image
intensity of the processed image is higher than the original simulated image because of the reassignment
of the ghost image photons to the original spatial locations from where they are generated due to emission
scattering.
Quantitative analysis of the reconstruction results is performed to analyze the performance of the
proposed method. Fig 3.5 (a) shows the intensity comparison of the MMM and processed image of
simulated beads. The sum of the intensity of the simulated bead and its ghost images, in MMM image, is
compared with the intensity of the same bead of the processed image. The error bar, in the MMM, are
showing the range of Poisson noise, added in the simulated MMM image used for processing using the
proposed method. Clearly, reconstruction intensities at different anode locations are within the Poisson
noise range showing the accuracy of the photon reassignment process from ghost images to correct spatial
locations. Next, comparison of elements of the scattering matrix corresponding to different anodes is
performed. The scattering matrix elements for (1,1) anode should be [0.61 0.14 0.14 0.1] based on the
preset parameters and the acquired emission scattering coefficient with normalization, and the same ratio
60
follows for other anodes depending on their spatial locations. Our target is to compare the recovered
scattering matrix elements, based on the proposed approach, with the original one used for simulation. Fig
3.5 (b) shows the original scattering matrix elements used for simulation for both beads and alphabets
images. Fig 3.5 (c) and (d) show the reconstructed scattering matrix elements for the beads images and
alphabets images respectively.
(a)
(b)g
30
30
S20
o
MMMM
0.5
rErcs
0.5
o
(1,2)
0.5
0.
(1) (1,2
(2,1)
(2,2)
(2,1)
(2,2)
(d)
(C)
0.5
0)
(1,2)
(1,1)
(1,2)
(2,1)
(2,2)
0.5
0O.
05.
(2,1)
Fig 3.5. (a) Intensity comparison of the primary beads in the simulated MMM image (blue, main anode + ghosts,
subjected to Poisson noise) and their processed images (red). (b-d) Scattering coefficients distribution at 2by2
MMM anodes. (b) Original distribution used in simulation, (c) estimated distribution of the processed bead image,
and (d) the alphabet image. One sub-image contains a primary coefficient from the corresponding image and three
scattering coefficients showing the leakage of emission photons to the neighbor anodes due to scattering.
3.3.2. Experimental results
3.3.2.1 Fluorescent beads image in 6 by 6 MMM with 85 pm foci separation
61
We tested our proposed approach with fluorescent beads image taken from 6 by 6 MMM system (with
foci separation 85 pm) as described in Sec. 2.1. The sample was prepared with 10 and 15 ptm diameter
fluorescent latex microspheres (F8836 and F21010, Molecular Probes, Eugene, OR) immobilized in 3D
by 2 % agarose gel. Fat emulsion (Microlipid, Nestle, Vevey, Switzerland) was added as a scatterer in 2
% concentration to mimic the scattering characteristic of a tissue specimen.
The left column of Fig 3.6 shows the acquired fluorescent beads MMM images at four different depths (0
pm, 40 pm, 85 gm, and 135 pm). As expected, the scattering effect increases with increase in imaging
depth and thus the effect of ghost images are more prominent for the larger imaging depth. To suppress
the effect of the ghost images and to reassign the photons of the ghost images to their original spatial
locations, we used our proposed method. For the 6 by 6 MMM image, total 36
x
36 = 1296 coefficients
are necessary for the scattering matrix C. Apparently, it is required to separately estimate the first guess of
the scattering matrix and scattering coefficient for each depth as the effect of scattering is different. To
start the photon reassignment iteration process, we have used the same criteria as explain in section 3.1.1
to evaluate the first guess of the scattering matrix and scattering coefficient. The iteration process took
around 30 iterations and the right column of Fig 3.6 shows the processed images at their corresponding
depths. Clearly, most of the strong ghost images were removed successfully, and only the primary bead
images were left, with better signal strength. In order to understand the performance of the proposed
method on the signal levels of the imaging bead and its effect on the ghost image, we analyzed some of
the relevant imaging parameters of the original and the processed MMM image. Fig 3.7 (a) shows the
strength of the signal of the bead at its original spatial location before and after processing. Clearly, the
strength of the signal is higher for the processed images, particularly at the larger imaging depth. It is due
to the fact that the processing provides the reassignment of the photons from the ghost image locations to
the original beads location and thus increases the strength of the signal. This improvement in the signal is
expected to be higher for the larger imaging depth as the ghost images are strong corresponding to the
higher scattering effect. Fig 3.7 (b) shows the effect of the photon reassignment by calculating the ratio of
62
the signal at original bead location to the signal detected at its strongest ghost image location, called
signal-to-ghost ratio (SGR). We calculated SGR for original and the processed images at each imaging
depth. Evidently, the SGR is supposed to be the highest for the smallest imaging depth because of the
least scattering effect. The SGR of the original image, which is significantly lower for the large imaging
depth because of severe scattering, is substantially recovered for the processed image.
Original
Processed
1250
=L
p250
2W0
200
150
150
50I
IW
E0
1000
20
1
00
0
800
300
6
200
100
200
200
1200
E
0100
100
1'40
10
4.'
200
200
015
o100
0
3000
Ir200
Fig 3.6. Left column: fluorescent beads images with the 6 by 6 MMM system at three imaging depth. Right column:
Corresponding processed images
63
120o MOriginal
100
U Processed
250
200
~80 0
.. 600
09
2 150
400
100
2
3Original
Wroce
" Opm
I 1 L40pm
501
85pm
135pm
0Opm
(a)
40pm
85pm
135pm
(b)
Fig 3.7. (a) Bead's signal comparison at different depths for original and processed image. (b) Signal-to- Ghost ratio
(SGR) of the imaging beads at different depths.
Elements of scattering matrix at the image bead location anode and adjacent nearest neighbors are
analyzed for different imaging depth, as shown in Fig 3.8 (a). As the imaging depth increases, due to
increase in the emission scattering the scattering matrix elements become flatter and the ratio of main
anode to neighboring anodes decreases as previously investigated in the Ref [25]. This effect is further
elaborated by plotting the values of scattering coefficient acquired at the different imaging depths, as
shown in Fig 3.8 (b). As the imaging depth increases, the scattering coefficient expectedly increases
showing the increased scattering effect.
0.1
-- Depth:
0pm
.
0.08
0
-Depth: 85pm
-Depth: 135pm
VE
0
0
1
--Fitted,.
--- Depth: 40pum
0 .8
0)0.06
0 .4
~0.04
U)
0.2
ci,
-2
-1
0
1
Anode position
2
3
'050
Depth (pm)
(a)
100
ISO
(b)
Fig 3.8. (a) Plot of scattering coefficient for an anode and its distribution at neighboring anodes corresponding to
different imaging depths of fluorescence beads image, (b) Plot of emission scattering coefficient as function of
depth.
64
3.3.2.2. Mouse brain imaging in 4 by 4 MMM with 85 pm foci separation
We tested our approach with mouse brain image taken in the 4 by 4 MMM system. Thyl-GFP transgenic
mice [26] had surgery for cranial windows that were bilaterally implanted over the visual cortices
between 6-8 weeks of age. These mice express green fluorescent protein (GFP) in a sparse pseudorandom subset of neocortical neurons. Imaging was performed on adult mice (>3 months) previously
implanted with cranial windows. Mice were anesthetized with isoflurane (3% for induction and 1.5%
during imaging). Anesthesia was monitored by breathing rate and foot pinch reflex. The head was
positioned in a custom made stereotaxic restraint affixed to a stage. Cell body and dendritic arbors of
inhibitory neurons labeled with GFP in layers 2/3 of visual cortex were imaged. The excitation
wavelength was 910 nm, , the laser power per focus was about 42 mW, the dwell time was 40
Rim pixel, and the image size was 340 jm
x
340 [im with 4
x
sts with 0.5
4 foci..
The left column of Fig 3.9 shows the MMM images of the mouse brain at 100 im and 190 pim imaging
depths. Ghost images can be visually observed in at both depths, particularly they are more prominent for
the larger imaging depth. The proposed approach was used to suppress the effect of the ghost images and
to reassign the photons of the ghost images to their original spatial locations. For the 4 by 4 MMM image,
total 16
x
16 = 256 coefficients are necessary for the scattering matrix C. We have used the same criteria
as explained before to evaluate the first guess of the scattering matrix and scattering coefficient.
Processed images are shown in Fig 3.9, the right column at the corresponding depths. Most of the strong
ghost images were clearly removed. With the expectation of the similar behavior of imaging parameters
as shown in Fig 3.7, we also analyzed the behavior of elements of scattering matrix and scattering
coefficient at different depth and Fig 3.10 show the plot of their values as a function of depth. Similar to
the result of the bead sample in section 3.2.1, the scattering matrix becomes flatter and the scattering
coefficient increases as the imaging depth increases.
65
I140
Original
Processed
200
120
1240
E
100
150
~80
60
100
40
50
20
10
10
0
00
0
300
800
E
'200
600
400
100
200
0
0
Fig 3.9. A mouse brain image with the 4 by 4 MMM system. Left acquired images at 100 pm and 190 im imaging
depths, and Right are the corresponding processed images.
0.18-
1
-'-Depth: 100pm
-Depth: 180pm
0.8 -Fitted
0.6
6
c0.2f
0.4
0.2
. 42
Anode position
2
50
100
Depth (pm)
(b)
(a)
150
200
Fig 3.10. (a) Plot of scattering coefficient for an anode and its distribution at neighboring anodes corresponding to
different imaging depths of mouse brain, (b) Plot of emission scattering coefficient as function of depth.
66
3.3.2.3 Fluorescent beads and Mouse brain imaging in MMM with 45 gm foci separation
We further tested our proposed approach with a fluorescent beads and Mouse brain image taken from
MMM system with foci separation 45 pm. The reduced separation of the foci, for the same scattering
conditions, will increase the crosstalk between the foci and it results as the strong ghost images. The
beads sample was prepared with 4 gm diameter fluorescent latex microspheres (F8858, Molecular Probes,
Eugene, OR) immobilized in 3D by 2 % agarose gel. Intralipid emulsion (Liposyn III, Abbott
Laboratories, North Chicago, IL) was added as a scatterer in 2 % concentration to mimic the scattering
characteristic of a tissue specimen. For the mouse brain imaging, a Thy I-GFP transgenic mouse [26] was
deeply anesthetized with 2.5% Avertin (0.025 ml/g i.p.) and transcardially perfused with PBS, followed
by 4% paraformaldehyde. Its brain was dissected and placed overnight in cold 4% paraformaldehyde. 300
pm thick coronal sections were sectioned with a vibrotome, then mounted and coverslipped on
microscope slides using adhesive silicone isolators (JTR20-A2-1.0, Grace Bio-Labs, Bend, OR).
Fig 3.11 (a) shows the fluorescence beads image taken from the 6 by 6 MMM with foci separation of 45
ptm at 150 pm imaging depth and (b) is its corresponding processed image. Clearly, due to the smaller
foci separation the scattered emission photons were scattered into the adjacent neighboring anodes of the
MAPMT and resulting in the formation of ghost images. We used our proposed approach to suppress the
ghost images in such a severe scattering case. From the processed image shown in Fig 3.11 (b), we can
visually observe that the ghost images are significantly suppressed, and the remaining dim beads are some
signals from the beads situated at neighboring axial planes. However, even with the same scattering in
sample but because of different foci separation, the scattering coefficients were 0.88 for 85pm at 135pm
imaging depth, and 0.98 for 45ptm imaging 150pm depth. Finally, Fig 3.11 (c) shows the mouse brain
imaging taken at 90 pm imaging depth using the 8 by 8 MMM system with foci separation 45 gm, and
Fig 3.11 (d) is its corresponding processed image. The same criteria, as discussed before, were used to
estimate the first guess of the scattering matrix and scattering coefficient.
67
3000
400
300
2000
200
1000
1001
(a)
(b)
2000
300
1500
200
1000
100
500
(c)
(d)
Fig 3.11. MMM images (with foci separation 45pm) and its processing. (a) Fluorescent beads image with the 6 by 6
MMM system, and (b) is the corresponding processed image. (c) Image of GFP expression neurons in mouse brain
acquired using 8 by 8 MMM system at 90pm depth, and (b) is the corresponding processed image.
3.4. CONCLUSIONS
MMM has an advantage of imaging speed improvement realizing high throughput in vivo imaging.
However, MMM images suffer from ghost images due to scattering of a turbid specimen especially when
the imaging depth is large. The ghost images have been processed by acquiring the scattering coefficients
experimentally. However, the experimental estimation requires frequent measurement with different
specimen and even with different locations and imaging depths. In this paper, we have proposed to
estimate the scattering coefficients with the photon reassignment process established based on the
maximum likelihood (ML) estimation. This image post-processing method increases the SNR and SBR of
the final processed image by reassigning the scattered photons to the original spatial locations, and also
avoids experimental measurements resulting in saving time and minimizing possible errors. We have
68
shown the feasibility of our approach with simulated results and confirmed that the algorithm can
estimate the scattering coefficients and reassign the ghost images. Further we examined it with
fluorescent bead samples in the turbid medium, and in vivo mouse brain images. The processed images
showed that the scattered emission photons were reassigned to their original spatial locations resulting in
higher signal at the original location. The SGR was improved by up to a factor of 10. The algorithm also
confirmed that the shorter foci separation generated more crosstalk, and the scattering coefficient has
increased as a function of imaging depth. To minimize the crosstalk between foci and the resulting ghost
images inherently, it is recommended to adopt a large field of view objective lens and maximize the foci
separation. However, in the case that only small field of view objective lenses are available or a region of
interest is small, our proposed approach will be a good solution for image post processing and image
recovery. Fig 3.12 shows the plot of log-likelihood function versus number of iterations for the processing
of mouse brain imaging as shown in Fig 3.9. Convergence of the log-likelihood function starts at around
15 iterations, which provides the final processed image. The total computational time taken for processing
two imaging planes was 40 seconds by using the normal desktop computer with i5 processor.
0X106
-2
0
~0
0
.-
6-
-8
-10
5
10
15
Number of iterations
20
Fig 3.12. Plot of log-likelihood function for number of iterations.
References
1.
Denk, W., J.H. Strickler, and W.W. Webb, 2-PHOTON LASER SCANNING FLUORESCENCE MICROSCOPY.
Science, 1990. 248(4951): p. 73-76.
69
2.
5.
So, P.T.C., et al., Two-photon excitation fluorescence microscopy. Annual Review of Biomedical
Engineering, 2000. 2: p. 399-429.
Svoboda, K., et al., In vivo dendritic calcium dynamics in neocortical pyramidal neurons. Nature, 1997.
385(6612): p. 161-165.
Lee, W.C.A., et al., A dynamic zone defines interneuron remodeling in the adult neocortex. Proceedings of
the National Academy of Sciences, 2008. 105(50): p. 19968-19973.
Ragan, T., et al., Two-photon tissue cytometry. Methods in cell biology, 2004. 75: p. 23-39.
6.
Kim, K.H., et al., Three-dimensional tissue cytometer based on high-speed multiphoton microscopy.
3.
4.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Cytometry Part A, 2007. 71(12): p. 991-1002.
Kim, K.H., C. Buehler, and P.T.C. So, High-speed, two-photon scanning microscope. Applied optics, 1999.
38(28): p. 6004-6009.
Fan, G., et al., Video-rate scanning two-photon excitation fluorescence microscopy and ratio imaging with
cameleons. Biophysical journal, 1999. 76(5): p. 2412-2420.
lyer, V., B.E. Losavio, and P. Saggau, Compensation of spatial and temporal dispersion for acousto-optic
multiphoton laser-scanning microscopy. Journal of Biomedical Optics, 2003. 8: p. 460.
Reddy, G.D. and P. Saggau, Fast three-dimensional laser scanning scheme using acousto-optic deflectors.
Journal of Biomedical Optics, 2005. 10: p. 064038.
Zeng, S., et al., Simultaneous compensation for spatial and temporal dispersion of acousto-optical
deflectors for two-dimensional scanning with a single prism. Optics Letters, 2006. 31(8): p. 1091-1093.
Katona, G., et al., Fast two-photon in vivo imaging with three-dimensional random-access scanning in
large tissue volumes. Nature methods, 2012. 9(2): p. 201-208.
Cianci, G.C., J. Wu, and K.M. Berland, Saturation modified point spread functions in two-photon
microscopy. Microscopy research and technique, 2004. 64(2): p. 135-141.
Zipfel, W.R., R.M. Williams, and W.W. Webb, Nonlinear magic: multiphoton microscopy in the biosciences.
Nature biotechnology, 2003. 21(11): p. 1369-1377.
Oron, D., E. Tal, and Y. Silberberg, Scanningless depth-resolved microscopy. Optics Express, 2005. 13(5): p.
1468-1476.
16.
Zhu, G., et al. Simultaneous spatial and temporalfocusing offemtosecond pulses. 2005: Optical Society of
17.
America.
Bewersdorf, J., R. Pick, and S.W. Hell, Multifocalmultiphoton microscopy. Optics Letters, 1998. 23(9): p.
655-657.
18.
Buist, A.H., et al., Real time two-photon absorption microscopy using multi point excitation. Journal of
19.
20.
21.
22.
23.
24.
25.
26.
Microscopy-Oxford, 1998. 192: p. 217-226.
Nielsen, T., et al., High efficiency beam splitterfor multifocal multiphoton microscopy. Journal of
microscopy, 2001. 201(3): p. 368-376.
Kurtz, R., et al., Application of multiline two-photon microscopy to functional in vivo imaging. Journal of
neuroscience methods, 2006. 151(2): p. 276-286.
Jureller, J.E., H.Y. Kim, and N.F. Scherer, Stochastic scanning multiphoton multifocal microscopy. Optics
Express, 2006. 14(8): p. 3406-3414.
Ami r, W., et al., Simultaneous imaging of multiple focal planes using a two-photon scanning microscope.
Optics Letters, 2007. 32(12): p. 1731-1733.
Sacconi, L., et al., Multiphoton multifocal microscopy exploiting a diffractive optical element. Optics
Letters, 2003. 28(20): p. 1918-1920.
Watson, B.O., V. Nikolenko, and R. Yuste, Two-photon imaging with diffractive optical elements. Frontiers
in neural circuits, 2009. 3.
Kim, K.H., et al., Multifocal multiphoton microscopy based on multianode photomultiplier tubes. Optics
Express, 2007. 15(18): p. 11658-11678.
Feng, G., et al., Imaging neuronal subsets in transgenic mice expressing multiple spectral variants of GFP.
Neuron, 2000. 28(1): p. 41-51.
70
Chapter 4
Non-descanned Multifocal Multiphoton Microscopy with
Multianode Photomultiplier Tube
4.1. Introduction
Multiphoton excitation fluorescence microscopy has inherent 3D resolution due to the nonlinear
dependence of excitation efficiency on the incident light flux distribution [1, 2]. Multiphoton excitation
can be localized to a femtoliter region at the focal point of a high numerical aperture objective. This
power microscope modality has become one of the common solutions for non-invasive, deep imaging in
many turbid biological specimens. Improving multiphoton microscope imaging speed is important for
many research areas including studies of fast intra- and inter-cellular signaling events involving voltage or
calcium transients and studies involving animal models where motion artifacts and anesthetic use should
be limited. Several methods have been developed to improve imaging speed beyond conventional point
scanning approach based on galvanometric mirror scanners. One approach is based on the use higher
speed scanners such as polygonal mirrors [3], resonant mirror scanners [4], or acousto-optical deflectors
[5-8]. These high speed scanners can typically achieve frame rate up to about 1 kHz in tissues with a
comparable imaging depth as conventional multiphoton microscopy. However, the higher speed scanning
requires a correspondingly decreased pixel dwell time resulting in poorer signal-to-noise ratio (SNR).
This tradeoff can be partially compensated by increasing the excitation laser power but laser power is
ultimately limited by specimen photodamage and excitation saturation [9, 10]. Another approach is twophoton wide-field imaging based on temporal focusing [11, 12]. In this case, two-photon excitation is
localized to a plane, instead of a point, by controlling the spectrally dispersing the light pulse away from
71
the focal plane. However, wide-field two-photon imaging is often limited by the lower axial resolution
and the smaller field-of-view (FOV) due to the need for much higher peak power laser pulses. Another
popular approach to improve imaging speed is multifocal multiphoton microscopy (MMM) [13, 14]. With
a lenslet array or a diffractive optical element (DOE) [15, 16], multiple foci are generated and scanned
simultaneously. Within the limit of available laser power, about 100 foci can be effectively generated
with a standard Ti-Sapphire oscillator resulting in approximately two orders of magnitude improvement
in imaging speed proportionally. For MMM system, simultaneous acquisition of data from many foci
often require the use of an area detector, such as a CCD or a CMOS sensor [13, 14]. The spatial
registration is achieved by telecentric mapping of the image plane into the image sensor plane. In the
presence of a turbid specimen, the emission photons can be scattered resulting in image blurring. More
important, as scattered emission photons are scattered away from the correct location, they contribute to
an increased background and degrading image SNR. To overcome this limit of emission light scattering,
MMM with a descanned detection configuration using a multianode photomultiplier tube (MAPMT) has
been developed [17, 18]. As in most confocal microscopes, the descanned geometry guides the emission
photons from each focus backward along the same optical path shared with the excitation beams. Since
the motion of the scanning mirror is much slower than the speed of light, the emission light rays after the
scanning mirror becomes stationary independent of scanner motion and can be separated from the
excitation beams with a dichroic mirror. The unscattered ballistic emission photons generated by each
focus are focused at the cathode which is corresponding to the center of each anode of the MAPMT. This
approach is described as "descanned" detection. The scattered emission photons will not be focused at the
center but many can still be collected effectively by the appropriate anode. Very intuitively, the larger
area projected by each anode at the specimen plane will result in greater collection efficiency of these
scattered emission photons. At the limit of infinitely large anode, the collection efficiency of scattered
emission photons will be comparable to that of a point focusing multiphoton system. Since the infinitely
large anode limit cannot be completely realized, if the sample is highly scattering and imaging depth is
72
large, the scattered emission photons will reach neighbor anodes resulting in ghost images in the neighbor
sub-images. We have shown that a number of post-processing algorithms can be developed to reassign the
scattered photons into the neighboring pixels and mostly eliminating these ghost images [17]. In any case,
to minimize the deleterious effect of the scattering of emission photons it is best to image using a high
numerical aperture (NA), low magnification objective so that the foci can be physically separated as far as
possible over the FOV on the specimen plane. Today, the most advanced objectives for multiphoton
imaging typically has NA on the order of 1.0 and FOV of about 1 mm. This large FOV presents
significant challenge in microscope intermediate optics design. The edge foci are generated from rays that
subtend large angles with the optical axis and are far from paraxial and significant aberration can result
from the use of intermediate optics with low f-number. This challenge is compounded by the fact that the
excitation ray beams must be first de-magnified to fit onto the millimeter size galvanometric scan mirrors,
their small size and light weight are important for fast scanning. The excitation beams must be magnified
again to backfill the several centimeter diameter back aperture of these large FOV objectives to ensure
diffraction limited focusing. These beam size control requirements are difficult to satisfy with using
commercially available high f-number lenses that have short focal distances that are not compatible with
the placement with components in the optical path with substantial physical sizes such as the
galvanometric scanning mirror system. The need for some low f-number lenses results in aberrated point
spread function (PSF), especially at the edge and corner foci, resulting in resolution and signal strength
degradation at the edge of the image. Another major problem of the descanned MMM design is the loss of
the emission signal in the descanned detection configuration that not only has many more optical surfaces
that resulting in attenuating both un-scattered and scattered emission photons. The much longer optical
path further results in losing a majority of scattered emission photons that are often scattered at large
angles relative to the optical axis. Signal attenuation in the descanned geometry can become quite severe
[17]. This problem is, of course, well known in point scanning multiphoton microscope and the preferred
method to overcome this limitation is using a non-descanned configuration [19, 20]. In this work, we
73
propose a novel non-descanned MMM with a MAPMT detector that preserve the SNR immunity to
scattered emission photon scattering but allows scanning over millimeter scale FOV with minimal
aberration and improve emission photon collection by up to four times.
4.2. Method
4.2.1. Non-descanned MMM with an imaging detector
,
\CCD
Tube Lens
Dichroic r
=1na
Scanning
mirrors
--
Multianude
descann
o for
MMM
\
Dichroick
or descanned MMM
Objectiv. Lens
sample
4-
DOE
Fig 4.1. The schematic of older generations of MMM. Two generations of MMM are implemented here including a
non-descanned MMM acquiring signal with a camera and a descanned version acquiring signal with a MAPMT.
Fig 4.1 shows the system configuration of a non-descanned MMM with an imaging detector similar to
previous publications [13, 14]. The light source is a Chameleon Ultra II (Coherent, Santa Clara, CA). The
excitation laser beam is split into a number of beamlets with a DOE (customized, 8
x
x
8 for 780 nm and 4
4 for 910 nm, Holo/Or, Rehovoth, Israel). The multiple beamlets are sent to x-y galvanometric mirror
74
scanners (6215H, Cambridge Technology, Lexington, MA), and scanned simultaneously. The scanned
beamlets are expanded to overfill the back aperture of a 20x water immersion objective lens (W PlanApochromat, 1.0 NA, Zeiss, Thornwood, NY) with different entrance angles. The objective lens generates
an array of excitation foci on the focal plane in a specimen. The emission photons are collected by the
same objective lens. In the very early design of MMM the emission photons pass through the dichroic
mirror after the objective lens according to the separation wavelength, and arrive at an imaging detector,
such as a CCD camera. The camera acquires all the emission photons until scanning is done. As discussed,
this configuration is very susceptible to SNR loss due to scattering of emission photons.
4.2.2. Descanned MMM with a MAPMT
Fig 4.1 also shows the descanned detection configuration with a MAPMT. For descanning, the dichroic
mirror between the objective lens and lens L 4 is replaced by a normal mirror, and the dichroic mirror is
located between the scanning mirrors and lens L 2. The emission photons from the sample travel backward
through the optical path shared with the excitation beam. After the emission rays travel through the
scanning mirrors they become stationary independent of scanner position; this process is called "descanning" and is well known in the design of confocal microscopes [21-24]. De-scanning is possible
because the transit time of the excitation photon, the fluorescence lifetime, and the arrival time of the
emission photons in combination is much faster than the motion of the scanner and the scanner is virtually
motionless relative to the light. The symmetry of the light path between excitation and emission light rays
ensures that the emission rays directions are constant independent of scanner motion. The stationary
emission rays from the foci are reflected by the dichroic mirror (Chroma Technology, Bellows Falls, VT)
and focused at cathode locations that correspond to the centers of each of the anodes of the MAPMT
(H7546B-20, Hamamatsu, Bridgewater, NJ). An IR blocking filter (BG39, Chroma Technology, Bellows
Falls, VT) and a short-pass filter (ET680sp-2p, Chroma Technology, Bellows Falls, VT) are installed
before the MAPMT to block the excitation light. As long as the anode surface corresponds to a
75
sufficiently large area on the specimen plane, we can ensure that most of the scattered emission photons
are collected in the designated channels. Therefore, the use of de-scanning and MAPMT detector can
suppress the scattering effect of emission photon and resulting in higher SNR. However, as described
above, there is significant loss of the emission signal due to having many optical surfaces and the long
descanning path with limited size of optics (e.g. scanning mirrors). Important, significant aberration of
edge foci is observed with using high NA, large FOV objectives.
4.2.3. Non-descanned MMM with MAPMT
The use of a high NA, large FOV objective allow excitation to be spaced far apart in the specimen plane
minimizing crosstalk between the emission signals from neighboring foci. Unfortunately, the aberrations
from the intermediate optics prohibit fully utilizing the FOV of the objective lens in the descanned
configuration. In Fig 4.1, after the DOE, the size of the multiple excitation beamlets must be decreased
with lens L1 (f 300 mm, singlet, KPX232AR.16, Newport, Irvine, CA) and lens L2 (f 75 mm, doublet,
AC508-075-B-ML, Thorlabs, Newton, NJ) in a 4-f geometry to fit onto small, 5 mm size scanning
mirrors (6215H, Cambridge Technology, Lexington, MA) to allow fast mechanical scanning. The lens L 3
(f 35 mm, doublet, AC254-035-B-ML, Thorlabs, Newton, NJ) and lens L4 (f 175 mm, singlet,
KPX1 96AR. 16, Newport, Irvine, CA) in another 4-f configuration expand the beamlets to slightly overfill
the 2 cm diameter back aperture of the objective lens ensuring diffraction limited focusing. Given the
finite physical size of the scanning mirror assembly, lenses L2 and L 3 must have sufficiently long focal
lengths and relative poor (low) f-numbers, and generate large aberrations especially for edge and corner
foci. Consequently, the final FOV becomes very limited (only around 300 pm
available size of 700 pm
x
x
300 pm from the
700 gm). Large size scanning mirrors are available to avoid the aberrations
from the reduction and the expansion processes, but scanning with them is slow. In addition, the use of
larger mirror will also result in larger physical separation between the x-axis scanning mirror and y-axis
scanning mirror resulting in greater non-uniformity of excitation over the FOV [25].
76
Fig 4.2 shows the schematic of the non-descanned MMM with MAPMT. The scanning mirrors are
positioned before the DOE. In this configuration the excitation beam can be expanded gradually with two
4-f beam expanders corresponding to lens pairs of L (Eyepiece E-PL 10x/20, Zeiss, Thornwood, NY)
and L2 (f 100 mm, doublet, AC508-100-B-ML, Thorlabs, Newton, NJ), and L3 (f 200 mm, doublet,
AC508-200-B-ML, Thorlabs, Newton, NJ) and L4 (f 300 mm, singlet, KPX232AR.16, Newport,
Irvine,
CA). This arrangement improves the overall f-numbers of the intermediate optics and minimizes the
aberrations. After the lens L 2 , the scanned laser beam enters the DOE with various angles of incidence
(AOI). Within a small range of AOI (in our design ±0.4
0),
the performance of the DOE was consistent.
The dichroic mirror is located at the mirror position between lens L4 and the objective lens so that the
emission photons reach the MAPMT with a shorter path length (500 mm versus 872 mm of the descanned
configuration in Fig 4.1). The same IR blocking filter (BG39, Chroma Technology, Bellows Falls, VT)
and the short-pass filter (ET680sp-2p, Chroma Technology, Bellows Falls, VT) are installed before the
MAPMT to block the excitation light. Fig 4.3 shows the f-numbers of our two MMM systems optimized
for each descanned and non-descanned case using commercially available lenses.
MAI'MTWWRE~
Tube Les
Scanwng
r
nurror
nurror
Objective.,
DOE
Foci at
sample
iT
*
@ ***
::::
Fig 4.2. The schematic of MMM system with non-descanned detection.
77
U Descan
U Non-descan
10
8
6
4
2
0
Li
13
L2
L4
Fig 4.3. The f-numbers of Lens LI, L2 , L3, and L4 in the descanned and non-descanned MMM.
1 mm
The four lenses in each case were optimized to utilize the full FOV of the 20x objective lens (FOV:
have
diameter at a sample plane), which provides about 700 pm x 700 pm scanning area. The lenses that
if the f
lower f-numbers generate dominant aberrations for the final PSF at the sample plane. Generally,
number is smaller than 10, a doublet or more complex compound lens should be used, instead of a singlet,
lenses
to minimize aberration [26, 27]. The descanned MMM design contains two low f-number doublet
2 in the
before and after the scanning mirrors (L 2 and L3), but their f-numbers improves from about
descanned configuration to over 5 in the non-descanned configuration. The f-number of the lens LI is
the f
slightly decreased, but it is a theta lens that can be normally used as a scan lens and easily afford
number 5.9.
4.2.4. ZEMAX simulation for the corner and edge foci of the non-descanned MMM
in less
As discussed above, the non-descanned MMM geometry can avoid low f-numbers optics resulting
aberrations. With the optical system design software, ZEMAX (Radiant ZEMAX LLC, Bellevue, WA)
the aberration coefficients were calculated and spot diagrams were simulated.
became smaller
With the improved aberration coefficients, the RMS radius of the simulated corner focus
the MAPMT
by a factor of 3. The two bottom rows show the actual focus images taken by a camera at
improved. The
location of Fig 4.2. It is visually shown that the second corner focus and edge focus were
was up
middle plots show the normalized PSF profiles of the displayed actual foci, and the improvement
but they
to two-fold. For more effective comparison, the first corner foci should have been compared,
78
could not be imaged due to beam clipping at the location of lens L of Fig 4.1 in the
descanned MMM
3
since there was not an available commercial lens that had the designed focal length
with a sufficient
diameter to avoid the beam clipping for the four corner beamlets. It should be noted
that the actual
resolutions of the displayed foci can be different from the measurement by the camera
since the images
taken by the camera is accumulated intensity signals in 3D rather than 2D PSF measurement
for lateral
resolution.
Zemike coefficients
Descanned geometrv
Z5 (astigmatism 2,-2)
2
Z6 (astigmatism 2,2)
1.5 1
Z7 (coma 3,-1)
1
Z8 (coma 3,1)
0
0.5
Z9 (trefoil 3,-3)
Non-descanned geometry
0 Descan U Non-descan
-.
--
-
-
-
-
Z10 (trefoil 3,3)
-
Z8
ZII (spherical)
-0.5
Integrated coefficient
-6
Z9
-
-
Z10
11
Integ
-
-1
-
Spot diagram of the
corner focus
(scale bar, left: 40
pim, right: 20 pm)
WM
I
S
U
RMS radius (pm)
W5
X
*E
2
8.5
Actual second corner
foci
(red: non-descan PSF
profile,
blue: descan PSF
profile)
FWHM
4.5
1
6.6
2 improvement
improvement
4.1
Actual edge foci
(red: non-descan PSF
profile,
blue: descan PSF
profile)
FWHM
3.4
Fig 4.4. The ZEMAX simulation of the excitation foci in the descanned and non-descanned geometry, and their
actual images. The scale bars in the actual images are 5 pm long.
79
4.2.5. Detection strategy: image shifting
The non-descanned MMM is expected to achieve a large FOV with improved PSF. However, in the nondescanned geometry, the emission photons from the n
x
n excitation foci are no longer stationary at the
cathode surface of the MAPMT. Since there are dead spaces between anodes in the MAPMT, the direct
telecentric mapping of the emission photons onto the MAPMT results in a loss of information when the
foci move onto the dead space during scanning (Fig 4.5 (a)). To avoid the dead spaces of the MAPMT,
we demonstrate a detection scheme that involves shifting the specimen and acquiring multiple exposures.
Consider a portion of the image (a sub-image) corresponding to the surface area of four neighboring
anodes (Fig 4.5(b)). By moving a specimen four times, each quadrant of the sub-image can be collected
by the center area of each anode avoiding the dead space (Fig 4.5 (c) and (d)). In addition, when the
emission focus is close to the edge of the anode of the MAPMT in one acquisition, another acquisition
can collect the same emission photons close to the center of its anode providing the information for the
emission photon reassignment to minimize the crosstalk between the anodes. In terms of the number of
shift, the minimally required sample shifts to cover the dead spaces is three times (Fig 4.5 (e)), but a fourstep movement is preferable to better utilize the center area of each anode where there is less aberration
and to ensure more uniform SNR across the final image when the individual captures are synthesized. In
our implementation, the specimens were translated by a custom-built manual stage. While our stage is
currently slower (a few seconds), the time required for producing the 42.5 gm shift can be accomplished
with a commercial high performance stage within a couple of milliseconds. We kept a constant pixel
dwell time of 40 gs for both single focus scanning experiment and the descanned MMM experiment. The
total data acquisition time was 4/n 2 shorter than single focus scanning assuming that n
x
n foci are used.
With this approach, most the image acquired 4 times more photons with the exceptions of the small dead
spaces where signal either does not increase or increases only twice (Fig 4.5 (d-e)). The four times higher
signals result in two times higher SNR according to Poisson noise statistics. The SNR dependences in
different regions of the image can be seen from imaging a 300 pM fluorescein solution (Fig 4.5 (f)).
80
However, it should be noted that we can in principle keep SNR same and shorter
acquisition time by 4
and maintain a speed gain of n2 for n x n foci.
(a)
Anode Matrix and Guide Mark (Unit: mm)
GUIDE MARK
(b)
Dead space (-24%)
03
4
.
(c)
One anod
+ Scanning-o
0
10.21
W763 84
6
1046
Anode Pattern
(d)
2.5
(e)
(f)
2
1.4
21.5
0.5
0
50
100
150
Photon countof a pixel
200
Fig 4.5. The detection scheme of the sample shift. (a) The anode matrix of the MAPMT with
0.3mm dead spaces
[28]. (b) The scan areas in a specimen (orange) and the areas covered by the anodes (blue).
(c) One scan area
covered by 4 time sample shift. (d) Overlapped scan areas with 4 time and (e) 3 time movement
in the 4 x 4 MMM
case. Red: 4 time, yellow: 3 time, green: 2 time, and violet: 1 time overlap. (f) SNR improvement
of each case.
4.3. Result
4.3.1. Fluorescent beads images with 8
x
8 MMM
As a demonstration of the capability of this non-descanned MMM, we started with imaging
a fluorescent
bead sample. The sample was prepared with 4 pm diameter fluorescent polystyrene
microspheres (F8859,
Molecular Probes, Eugene, OR) immobilized in 3D by 2 % agarose gel (UltraPure Low Melting
Point
Agarose, Invitrogen, Carlsbad, CA). Fat emulsion (Microlipid, Nestle, Vevey, Switzerland) was added
to
the sample in 2 % concentration to mimic the scattering characteristic of typical tissue specimens
[29, 30].
The excitation wavelength was 780 nm, and the laser power per focus was adjusted according
to the
81
imaging depth to maximize signal with no excitation satuarion (2.5 mW and 5 mW for 40 im and 80 jm
depth respectively in the non-descanned MMM case, and 15 mW for the single focus scanning case). The
dwell time was 40 is for both cases. The scanning size of one focus in the non-descanned MMM was 85
pm
x
85 jm with 0.4 gm resolution and the whole FOV was 680 gm
x
680 gm with imaged 8
x
8 foci.
As described in section 2.5, the sample was images four times with the x and y shift by 42.5 jm, and the
four images were integrated to one final image. The single focus image was scanned with the same
resolution (0.4 jm) over a FOV of 410 gm
x
410 pm (1024
x
1024 pixels). Cropped images acquired
with either methods were displayed in the Fig 4.6 for the better presentation of the 4 Km beads. The
acquired raw images are typically not uniform (the center area is brighter than the edge area) because
while the x and y scanning mirrors were closely spaced, they cannot be both at the eye-point of the
scanning system. Further, there were aberrations due to the large FOV. In addition, the emission signal of
the non-descanned MMM is focused and scanned on the cathod surface of the MAPMT, so the acquired
image contains the physical pattern of the cathode. To avoid the non-uniformity in the raw images, they
were normalized with a 20 time averaged (minimal shot noise) image of a 300 pM uniform fluorescein
solution measured as calibration.
Fig 4.6 showed that the non-descanned MMM could successfully image the beads sample. The measured
full-width at half-maximum (FWHM) of the bead for the non-descanned MMM and the single focus
scanning image were same 4.1 jm at 40 jm depth, and 4.2 and 4.1 jm at 80 jm depth showing the
comparable imaging capability of the non-descanned MMM to the single focus scanned image with 16
times improved imaging speed (excluding the four time manual movement of the sample). For further
verification we investigated the capability of large fleid of view imaging and compared the emission
signal loss between the descanned and non-descanned MMM.
82
Depth
Non-descanned MMM
Single focus
40
80
Fig 4.6. Fluorescent bead images in the non-descanned MMM and the standard single focus scanning microscope at
40, and 80 pm imaging depths. The images are 410 im
x
410 pm with 0.4 pm resolution.
4.3.2. Field-of-view comparison
To compare the FOV of the descanned and non-descanned MMM, the same sample was imaged in both
systems. The sample was a mouse kidney section labeled with three fluorescent dyes (FluoCells@
Prepared Slide #3,
F24630, Molecular Probes, Eugene, Oregon). Alexa Fluor@ 488 wheat germ
agglutinin was used to label elements of the glomeruli and convoluted tubules. The filamentous actin
prevalent in glomeruli and the brush border were stained with red-fluorescent Alexa Fluor(& 568
phalloidin. Finally, the nuclei were counterstained with the blue-fluorescent DNA stain DAPI. The
excitation wavelength was 780 nm, the laser power per focus was about 6 mW, and the dwell time was 40
83
ps. The scanning size of one focus was 85 gm
680 pm
x
680 pm with 8
x
x
85 pm with 0.4 pm resolution, and the whole FOV was
8 foci. The excitation wavelength for the single focus scaning was same 780
nm, the laser power was about 10 mW, and the dwell time was 40 ps. The scan size was same 680 pm
x
680 pm with 0.5 pm resolution. All the emission signals were collected without spectral separation for all
three fluorophores.
700
600
00
500
500
500
00
500
00400
300
300
200
300
200200
100
g100
100
0
"0
0
Fig 4.7. Mouse kidney image. (a) Raw image of the descanned MMM, image size: 680 pm
x
680 pm with 8
x
8 foci.
(b) Raw image of the non-descanned MMM image with the same FOV with the same number of foci. (c) Raw image
of the single focus scanning with the same FOV. (d-f) Normalized images of (a-c) with (g-i). (g-i) 20 time averaged
fluorescein solution image of each system. The periodic pattern in (h) is the actual shape of the faceplate in front of
the cathod in the MAPMT.
84
Fig 4.7 shows the mouse kidney images taken in the descanned MMM, non-descanned MMM, and the
standard single focus scanning microscope. Fig 4.7 (a-c) show the raw images acquired from the three
systems. For uniform signal intensity over the full FOV, the three raw mouse kidney images were
normalized with the fluorescein solution images (Fig 4.7 (g-i)) acquired in each system with the same
imaging condition as described in section 3.1. The final processed non-descanned MMM image (Fig 4.7
(d)) showed that the majority of the FOV of the objective lens can be readily utilized whereas the
descanned MMM (Fig 4.7 (e)) had a much more limited FOV. The center area of the both images showed
similar SNR but signal attenuation in the radial direction was much more severe in the descanned MMM.
Quantitative comparison was shown in Fig 4.8. From around 300 gm radial location, the descanned
MMM could not get any signal, but the non-descanned MMM could maintain the emission signals up to
the full field size. This signal attenuation of the descanned MMM is due to increasing aberration at the
image edge and the subsequent reduction in two-photon excitation efficiency. The non-descanned MMM
showed about half signal level compared to the single focus image. This lower signal is mainly due to the
non-uniformity of the DOE. The power of the corner beamlet is about 70 % of the center beamlet, which
is within the specification (target uniformity of all beamlets: ±15 %). Therefore, inherently the twophoton excitation of the corner focus has only half of the efficiency of the center focus. Due to the nonuniformity of the DOE and the aberrations from the intermediate optics the signal of the non-descanned
MMM was lower than the single focus image. However, the non-descanned MMM can image the whole
FOV with 16 times faster acquisition time (5 seconds of the non-descanned MMM excluding the manual
sample movement time versus 77 seconds of the single focus scanning with the same number of pixels
and same pixel size). In addition, it should be noted that the speed improvement can be eventually 64
times maintaining comparable SNR as descrived in section 2.5.
85
1
-single
focus
-- descan
-- non-descan
b 0.8
.c 0.6
-D
(D
N
0
Z 0.2Radius 0
100
200
300
400 (pm)
Fig 4.8. Normalized intensity profile comparison of Fig 4.7 (g-i). The distance from the center to the corner of the
680 gim
x
680 pm image is 481 lim.
4.3.3. Signal attenuation comparison
We proceed to compare the signal collection efficiency between the descanned and non-descanned MMM
in turbid medium. Fat emulsion (Microlipid, Nestle, Vevey, Switzerland) was added to the 300 pM
fluorescein solution at 2% concentration same as section 3.1. Images were acquired as a function of
imaging depths at every 10 pm. Fig 4.9 shows the signal attenuation in the non-descanned (red) and
descanned (blue) MMM. The power of the excitation laser was adjusted to keep the same powers under
the objective lens in the both systems. To avoid the non-uniformity of the excitation over the scanning
area described above, only the signal at the center area was analyzed in this measurement. With the same
excitation laser power and the dwell time, the signal of the non-descanned MMM was higher than that of
the descanned MIMM. The improvement was shown even at the surface since the forward emission
photons were also scattered backwards and collected more in the non-descanned MMM with the shorter
optical path and less optical components. The attenuation of the emission signal of the descanned MMM
was much faster than the non-descanned MMM. For example, at 30 Pm depth, the signal collection
efficiency of the non-descanned MMM was three times higher than the descanned MMM. Up to about
150 gm depth, the signal improvement was observed about two-fold.
86
700
non-descan
-descan
600
500
C:
o400
U
C
2300
0
0.200
324%(Max)
316%
100
0
0
230%
100
190A
200
Depth (pm)
300
400
Fig 4.9. Fluorescence emission signal attenuation in the descanned and non-descanned
MMM. Data points are
connected by lines for eye guidance.
4.3.4. Mouse brain images with 4 x 4 MMM
We further demonstrate the use of non-descanned MMM for thick biological specimen.
A Thyl-GFP
transgenic mouse [31] was deeply anesthetized with 2.5% Avertin (0.025 ml/g i.p.)
and transcardially
perfused with PBS, followed by 4% paraformaldehyde. Its brain was dissected and postfixed
overnight in
cold 4% paraformaldehyde. 300 pm thick coronal sections were sectioned with
a vibratome, then
mounted and coverslipped on microscope slides using adhesive
isolators. To excite the GFP efficiently,
the excitation laser wavelength was adjusted to 910 nm. In the previous results the laser
power at 780 nm
was enough to generate 8
x
8 excitation foci, but the available laser power at 910 nm from the Ti-
Sapphire laser was about one third of the power at 780m. Therefore, in this case 4
with the same foci separation, and the total imaging size was 340 pm
x
x
4 foci were generated
340 pm. The excitation laser
power was approximately 35 mW per each excitation focus. Fig 4.10 shows the mouse
brain images taken
in the descanned MMM, non-descanned MMM, and the standard single
focus scanning microscope at 40,
100, and 160 pm imaging depths. For uniform signal intensity, all the images were normalized
with the
fluorescein solution images acquired in each system with the same imaging condition as described
above.
The imaging size was 340 pm x 340 gm with 4 x 4 foci (previously 680 pm x 680 Pm with 8
x
8 foci).
For the smaller FOV, the descanned MMM did not show significant signal loss at the image edge
due to
87
the aberrations as in mouse kidney images. However, the non-descanned MMM could collect more
emission signals as confirmed in section 3.3, and show more structural details at 160 pm depth. In the
comparison between the non-descanned MMM images and the single focus scanned images, the single
focus scanned images showed higher SNR as shown in section 3.2. However, the acquisition of the nondescanned MMM is 4 times faster than the single focus scanning reducing the 1-2 hour imaging time of
some neurobiology experiments [32, 33] to 15~30, and possibly 4-8 minutes with further improvement
described in section 2.5.
Depth
Single focus
Non-descanned MMM
Descanned MMM
40
100
160
I
I
I.
I-I
Fig 4.10. Mouse brain images at 40, 100, and 160 pm depths. All images are 340 pm
size. The left column images are descanned MMM images with 4
descanned MMM images with 4
x
x
x
340 pm with 0.4 pm pixel
4 foci. The middle column images are non-
4 foci and the 4 time sample movement. The right column images are the
standard single focus scanned images.
88
For quantitative comparison of signal attenuation between the descanned and non-descanned
MMM, the
intensity trend of the neuronal cell bodies is plotted as a function of the imaging depth
in Fig 4.11. For
accurate comparison, only the brightest center parts of the cell bodies were counted.
With the same
excitation laser power and the dwell time, the signal of the non-descanned MMM was higher
than that of
the descanned MMM as previously found in section 3.3. The signal improvement was about twoto fourfold over the imaging depths.
600
-
282%
500
non-descan
descan
400
0
co 300-
2M
+.J
0
100
%
50
100
Depth (prm)
150
200
Fig 4.11. The fluorescence emission signal attenuation of the neuronal cell bodies in the descanned and
nondescanned MMM. The percentage numbers show the signal improvement, and the error bars show ± one standard
deviation at each data point. The data points are connected by lines for eye guidance.
4.4. Discussion
The descanned MMM can achieve better SNR compared to the traditional MMM by suppressing the SNR
degradation due to the scattering of emission photons using the MAPMT in the descanned detection
configuration. However, it still produces image artifacts, appearing as ghost sub-images, due to channel
crosstalk and significant emission signal loss due to the long detection path length. The non-descanned
MMM can minimize these ghost images with the maximized foci separation by fully utilizing the FOV of
the objective lens with minimal aberrations for corner and edge foci. In addition, the non-descanned
MMM can minimize the detection path length so that the signal collection efficiency can be improved up
89
to four-fold. However, the emission signals are no longer stationary on the detector due to scanning, and a
new detection scheme is necessary. We introduced a detection scheme of shifting a specimen and imaging
four times. With four time acquisition of the emission signals by locating the specimen at the
corresponding quadrant of one image, the non-stationary signals could be collected with avoiding the
dead space of the MAPMT. Further, maximum imaging speed improvement equal to the number of foci
can be achieved with high performance scanners and stage to obtain comparable SNR with single focus
scanning.
In addition to the sample shift and multiple imaging, another potential approach is the use of multiple
detectors. Instead of moving and imaging a specimen several times, the use of multiple MAPMTs with
split emission signals can achieve imaging with one time scanning and comparable SNR by covering
different locations of a specimen and avoiding the signal loss due to the dead space. However,
implementing multiple MAPMTs requires substantial more complex electronics, e.g. 256 acquisition
channels for four 8
x
8 anode MAPMTs. Another possible option would be the use of a custom parabolic
light collector array that has minimal dead space between elements at the entrance and can funnel the
emission photons reaching each element to the center area of a corresponding MAPMT anode. The
custom parabolic light collector (CPC) array may avoid either the sample movement or the complex data
acquisition electronics system with the use of multiple MAPMT detectors, and may allow the single shot
imaging with one MAPMT. However, the CPC should be fabricated with high precision for the inlet size
of the CPC exactly matching to the anode matrix size of the MAPMT. The trade-off in the use of CPC is
the more pronounced crosstalk between anodes requiring a more effective image post processing
algorithm. Another detection method is to use a fast camera and take an image at every scanning point.
The locations of focused emission photons from the multiple foci at each pixel can be easily predicted
with imaging a non-scattering sample (e.g. a fluorescein solution) in advance, and then the scattered
emission photons from a general specimen can be also collected by integrating the specific area of the
camera image for each focus to minimize the crosstalk between neighbors, which realizes a virtual
90
MAPMT in the non-descanned detection configuration. We have substantially investigated this method,
but it turned out that the accumulated readout noise during integrating a large number of pixels for each
focus significantly degraded SNR and made this method inapplicable.
In any case, it is true that the SNR of the standard single focus scanning two-photon microscope can often
achieve better SNR than the MMM. However, many applications with bright specimens have fairly good
SNR but minimizing the acquisition time is critical. In this case, MMM can be a more desirable imaging
solution, and the non-descanned MMM can perform better with higher SNR and less crosstalk with a
large FOV compared with the previous version of descanned MMM. If the applications deal with relative
transparent specimens [34-36] or shallower depths where scattering is not critical, other approaches such
as two-photon wide-field microscopy [11, 12, 37] or light sheet microscopy [38, 39] may provide faster
imaging speed with comparable SNR.
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
Denk, W., J.H. Strickler, and W.W. Webb, 2-PHOTON LASER SCANNING FLUORESCENCE
MICROSCOPY Science, 1990. 248(4951): p. 73-76.
So, P.T.C., et al., Two-photon excitation fluorescence microscopy. Annual Review of Biomedical
Engineering, 2000. 2: p. 399-429.
Kim, K.H., C. Buehler, and P.T.C. So, High-speed, two-photon scanningmicroscope. Applied optics, 1999.
38(28): p. 6004-6009.
Fan, G., et al., Video-rate scanning two-photon excitationfluorescence microscopy and ratio imaging with
cameleons. Biophysical journal, 1999. 76(5): p. 2412-2420.
Iyer, V., B.E. Losavio, and P. Saggau, Compensation of spatial and temporal dispersionfor acousto-optic
multiphoton laser-scanningmicroscopy. Journal of Biomedical Optics, 2003. 8: p. 460.
Reddy, G.D. and P. Saggau, Fast three-dimensionallaser scanning scheme using acousto-optic deflectors.
Journal of Biomedical Optics, 2005. 10: p. 064038.
Zeng, S., et al., Simultaneous compensation for spatial and temporal dispersion of acousto-optical
deflectorsfor two-dimensionalscanning with a single prism. Optics Letters, 2006. 31(8): p. 1091-1093.
Katona, G., et al., Fast two-photon in vivo imaging with three-dimensional random-access scanning in
large tissue volumes. Nature methods, 2012. 9(2): p. 201-208.
Zipfel, W.R., R.M. Williams, and W.W. Webb, Nonlinear magic: multiphoton microscopy in the
biosciences. Nature biotechnology, 2003. 21(11): p. 1369-1377.
Cianci, G.C., J. Wu, and K.M. Berland, Saturation modified point spread functions in two-photon
microscopy. Microscopy research and technique, 2004. 64(2): p. 135-141.
Oron, D., E. Tal, and Y. Silberberg, Scanningless depth-resolved microscopy. Optics Express, 2005. 13(5):
p. 1468-1476.
Zhu, G., et al. Simultaneous spatial and temporalfocusing offemtosecond pulses. 2005: Optical Society of
America.
Bewersdorf, J., R. Pick, and S.W. Hell, Multifocal multiphoton microscopy. Optics Letters, 1998. 23(9): p.
655-657.
91
14.
15.
Buist, A.H., et al., Real time two-photon absorption microscopy using multi point excitation. Journal of
Microscopy-Oxford, 1998. 192: p. 217-226.
Sacconi, L., et al., Multiphoton multifocal microscopy exploiting a diffractive optical element. Optics
Letters, 2003. 28(20): p. 1918-1920.
16.
17.
Watson, B.O., V. Nikolenko, and R. Yuste, Two-photon imaging with diffractive optical elements. Frontiers
in neural circuits, 2009. 3.
Kim, K.H., et al., Multifocal multiphoton microscopy based on multianode photomultiplier tubes. Optics
Express, 2007. 15(18): p. 11658-11678.
18.
Martini, J., V. Andresen, and D. Anselmetti, Scattering suppression and confocal detection in multifocal
19.
Le Grand, Y., et al., Non-descanned versus descannedepifluorescence collection in two-photon microscopy:
Experiments and Monte Carlo simulations. Optics Communications, 2008. 281(21): p. 5480-5486.
Zinter, J.P. and M.J. Levene, Maximizing fluorescence collection efficiency in multiphoton microscopy.
multiphoton microscopy. Journal of biomedical optics, 2007. 12(3): p. 034010-034010-6.
20.
21.
Optics Express, 2011. 19(16): p. 15348.
Minsky, M., Microscopy apparatus. 1961, US Patent 3013467.
22.
Sheppard, C. and A. Choudhury, Image formation in the scanning microscope. Journal of Modern Optics,
23.
Brakenhoff, G., P. Blom, and P. Barends, Confocal scanning light microscopy with high aperture
immersion lenses. Journal of microscopy, 1979. 117(2): p. 219-232.
24.
Carlsson, K. and N. Aslund, Confocal imagingfor 3-D digitalmicroscopy. Applied optics, 1987. 26(16): p.
3232-3238.
25.
26.
Pawley, J., Handbook of biological confocal microscopy. 2006: Springer.
Fischer, R.E., et al., Optical system design. 2000.
27.
28.
Lens Selection Guide, in THE NE WPORT RESOURCE.
Multianode PhotomultiplierTube Assembly H7546A, H7546B, HAMAMATSU.
29.
Dunn, A.K., et al., Influence of optical properties on two-photon fluorescence imaging in turbid samples.
1977. 24(10): p. 1051-1073.
Applied optics, 2000. 39(7): p. 1194-1201.
30.
31.
Dong, C.-Y., K. Koenig, and P. So, Characterizingpoint spreadfunctions of two-photon fluorescence
microscopy in turbidmedium. Journal of biomedical optics, 2003. 8(3): p. 450-459.
Feng, G., et al., Imaging neuronalsubsets in transgenicmice expressing multiple spectral variants of GFP.
Neuron, 2000. 28(1): p. 41-5 1.
32.
Chen, J.L., et al., Structural basisfor the role of inhibition in facilitating adult brain plasticity. Nature
neuroscience, 2011. 14(5): p. 587-594.
33.
34.
35.
36.
Chen, J.L., et al., Clustered dynamics of inhibitory synapses and dendritic spines in the adult neocortex.
Neuron, 2012. 74(2): p. 361-373.
Tuchin, V.V., Optical clearing of tissues and blood. Vol. 154. 2006: SPIE Press Bellingham.
Dodt, H.-U., et al., Ultramicroscopy: three-dimensional visualization of neuronal networks in the whole
mouse brain. Nature methods, 2007. 4(4): p. 331-336.
Chung, K. and K. Deisseroth, CLARITYfor mapping the nervous system. Nature methods, 2013. 10(6): p.
508-513.
37.
Cheng, L.-C., et al., Spatiotemporalfocusing-based widefield multiphoton microscopy for fast optical
sectioning. Optics Express, 2012. 20(8): p. 8939-8948.
38.
Fuchs, E., et al., Thin laser light sheet microscopefor microbial oceanography. Optics Express, 2002.
39.
Keller, P.J., et al., Reconstruction of zebrafish early embryonic development by scanned light sheet
10(2): p. 145.
microscopy. Science, 2008. 322(5904): p. 1065-1069.
92
Chapter 5
Spectral-Resolved Multifocal Multiphoton Microscopy with
Multianode Photomultiplier Tube
5.1. Introduction
Multiphoton excitation fluorescence microscopy has inherent 3D resolution due to the nonlinear
dependence of excitation efficiency on the incident light flux [1-3]. Multiphoton excitation can be
localized to a femtoliter region at the focal point of a high numerical aperture objective lens reducing
specimen photodamage and photobleaching. Multiphoton microscopy further can excite common
fluorophores with infrared radiation enhancing excitation light penetration depth in turbid specimens
where scattering is reduced. Equally importantly, unlike confocal microscopy, no confocal pinhole is
required for depth resolution allowing scattered emission photons to be collected effectively greatly
enhancing image signal to noise ratio (SNR) in turbid specimens. Consequently, multiphoton excitation
fluorescence microscopy has become the preferred tool for deep imaging of highly scattering tissue
specimens [4]. In many applications, high speed imaging is critical. For example, high speed imaging is
useful for some studies of neuronal plasticity that may require high resolution mapping the whole
dendritic arbor, covering almost 1 mm3 in volume, while minimizing the period of anesthesia to reduce
animal stress [5-10]. One of the methods to improve the imaging speed is multifocal multiphoton
microscopy (MMM) [11, 12]. With a lenslet array or diffractive optical element (DOE) [13, 14], a number
of foci are generated simultaneously in a specimen and scanned together. Each excitation focus scans a
fraction of the whole imaging area; the final image is synthesized by montaging the data from each focus.
Within the limit of the available laser power, the number of foci can be maximized and the imaging speed
93
can be improved in proportion to the number of foci. The early MMM used imaging detectors, such as
CCD cameras, to record emission signals from multiple foci simultaneously. However, the turbidity of
typical biological specimens scattered emission photons to neighbor pixels resulting in a blurred image
with degraded SNR. This difficulty was recently partly circumvented by introducing descanned MMM
using a detection scheme based on using a multianode photomultiplier tube (MAPMT). With this
approach, the scattered emission photons are much more effectively collected by the large area anodes
that corresponds to tens of micron size areas on the specimen plane providing much better immunity to
SNR degradation due to emission photon scattering [15].
Scan lens
Tube lens
jMirror
S~mnjm~
Multianode
PM
Dchroic
i(or
Objective Lens
sample::;
~~
DOE
Fig 5.1. The Schematic of MMM.
Fig 5.1 shows the schematic of a MMM system. The excitation laser light is divided to several beamlets
by the DOE, and they are delivered to the scanning mirrors with different angles by two lenses in the 4-f
location. All the beamlets are scanned by the scanning mirrors together, and expanded to slightly overfill
the back aperture of the objective lens. The beamlets enter the objective lens with different incident
angles, are focused in the sample, and generate the multiple excitation foci distributed into a square 2D
array. The emission signals from the foci are collected by the same objective lens, travel along the same
beam path as the excitation light, and are descanned by the same scanning mirrors. As the arrival time of
94
the emission photons is much faster than the dwell time for one pixel, the emission photons from each
focus are descanned and become stationary. The emission photons are separated by the dichroic mirror
from the excitation laser light, and focused at the cathode locations that correspond to the center of each
anode of the MAPMT. With the large area of the each anode, much of the scattered emission photons can
be effectively collected in the designated channels. The deleterious effect of emission photon scattering
can be effectively suppressed.
Many applications require not only high imaging speed but also the simultaneous detection of multiple
fluorophores to delineate structural relationships between different cellular and tissue constituents. Other
applications would require the imaging of multiple spectral ranges to assay tissue metabolic and
biochemical states using ratiometric fluorescent sensors. For example, multicolor imaging is critical in the
study of brain function. Recently, this approach has been used to study synapse formation and
dissociation in the context of dendritic remodeling in a mammalian brain [16-18]. An essential feature of
the mammalian brain is its capacity to adapt in response to the environment. This experience-dependent
plasticity is thought to involve the remodeling of synaptic connections between neurons. By monitoring
inhibitory and excitatory synapses of dendritic spines it has been investigated how synapse dynamics
accompanies dendritic arbor rearrangements. To simultaneously monitor the multiple fluorophores
marking dendrites, excitatory synapses and inhibitory synapses, spectral detection is essential. At the
same time, the imaging speed remains critical in order to image the large dendrite arbor at high resolution
so that individual synapses can be reliably resolved. Therefore, the addition of spectral detection
capability into a MAPMT based MMM is important.
High speed imaging and spectral detection can, in principle, be accomplished by spectrally separating the
emission light using dichroic mirrors and directing the different spectral components to multiple
MAPMTs. However, MAPMTs are relatively bulky and a system with multiple detectors with their
associated optics will have a very long detection optical path resulting in substantial loss of scattered
emission photons [15]. Today, several groups have investigated various types of the spectral-resolved
95
MMM. A MMM system with temporal and spectral resolution has been developed with a specially
designed streak camera [19-21]. The same group later has developed a spectral-resolved MMM with a
CCD camera with an one-axis descanned configuration [22]. In addition, a spectrometer-integrated NIMM
using an optical multiplexer has been developed [23-25]. However, these previous investigation require
specimen scanning or the use of a pulse multiplexing that limits image speed improvement. Here we
introduce a spectral-resolved MMM by implementing the spectral detection in a descanned MMM system
using a MAPMT. Previous designs of MAPMT based MMMs with descanned configuration utilized 2D
array of the excitation foci mapping to the physical anode matrix of the MAPMT. Instead of producing a
2D array of excitation foci, spectral-resolved MAPMT based MMM generates a linear array of excitation
foci. The spectral information can be acquired in the orthogonal axis of the MAPMT simultaneously. This
approach doesn't require extending of the detection light path that often suffers from significant signal
loss.
5.2. Method
5.2.1. System configuration
LL
Pnsm
MAPMT
L7
L6
L
sacipl
INL-
L
DoE
Fig 5.2. The configuration of the spectral-resolved MMM.
96
Fig 5.2 shows the spectral-resolved MMM. The excitation light path is same as the previously described
[15]. The femtosecond light source used was a Chameleon Ultra II (Coherent, Santa Clara, CA). The
excitation laser beam was split into 4
x
1 beamlets with a DOE (customized, Holo/Or, Rehovoth, Israel).
The beamlets were sent to the scanning mirrors (6215H, Cambridge Technology, Lexington, MA)
through the lens L1 (f 300 mm, singlet, KPX232AR.16, Newport, Irvine, CA) and lens L2 (f 75 mm,
doublet, AC508-075-B-ML, Thorlabs, Newton, NJ) in a 4-f configuration and scanned together. The
beamlets were expanded by the lens L 3 (f 35 mm, doublet, AC254-035-B-ML, Thorlabs, Newton, NJ) and
lens L4 (f 175 mm, singlet, KPX196AR.16, Newport, Irvine, CA) to slightly overfill the back aperture of
the objective lens (W Plan-Apochromat, 20
x,
1.0 NA, Zeiss, Thornwood, NY). The 4
were generated in a sample with 85 ptm separations, and each focus scanned 85 pm
total imaging size was 340 pm
x
x
x
1 excitation foci
340 pm area, so the
340 gm. The emission photons were collected by the same objective lens,
delivered along the same optical path as the excitation beam path, and descanned by the scanning mirrors.
The stationary emission light after descanning was separated from the excitation laser light by the
dichroic mirror (Chroma Technology, Bellows Falls, VT), relayed by lens L5 (f 62.9 mm, singlet,
KPX085AR.14, Newport, Irvine, CA) and lens L6 (f 150 mm, doublet, AC508-150-A-ML, Thorlabs,
Newton, NJ), and spectrally separated by a prism (PS853, Thorlabs, Newton, NJ) that anti-reflection
coated (Chroma Technology, Bellows Falls, VT). For the spectral decomposition, a grating is an
alternative choice. However, the grating generally suffers from significant light loss and its diffraction
efficiency can varied substantially across the spectral range. In contrast, a prism has greater efficiency and
more uniform sensitivity over a broader spectral range. However, prisms are less dispersive than gratings.
Therefore, we have utilized two prisms to achieve our designed spectral dispersion power. The spatially
separated and spectrally dispersed emission light was focused onto a MAPMT (H7546B-20, Hamamatsu,
Bridgewater, NJ) through lens L7 (f 100 mm, doublet, AC508-100-A-ML, Thorlabs, Newton, NJ). The
double prisms and the lens L 7 were designed to collect the range of the emission spectra of typical
fluorescent proteins (from teal to red fluorescent proteins). A BG39 (Chroma Technology, Bellows Falls,
97
VT) and a shortpass filter (ET680sp-2p, Chroma Technology, Bellows Falls, VT) were installed in front
of the MAPMT to block the excitation IR and pass only the emission photons in the designed spectral
range. Since the MAPMT has 8
x
8 channels, only 4 rows were used for the spatial separated foci while
all 8 columns were used for 8 channels in spectral resolved detection. However, while we choose to use
only 4 excitation foci, a different DOE can be fabricated to generate 8 foci to utilize the full 8
x
8
channels of the MAPMT. Further, within the available laser power, the use of multiple MAPMTs can
improve the imaging speed more, e.g. placing two 8
x
8 MAPMTs side-by-side allowing 16 foci scanning
in parallel with 8 simultaneous spectral channels. A single-photon avalanche diode (SPAD) array can be
another choice with its large number of pixels and higher quantum efficiency for further imaging speed
improvement and more spectral channels [26-28]. However, it should be noted that the SPAD array has
typically a low fill-factor (around 10%) resulting in the loss of emission photons. This limitation may be
overcome in the future with the use of a matching micro-lenslet array that takes the photon falling within
each lenslet to the corresponding active area of each pixel.
5.2.2. Spectral decomposition
Spectral decomposition is routinely done by linear unmixing on a pixel by pixel basis.
i
_S11
S12
1J2
S21
S22
[JmJ
.smi
sm2
where [I] is an n
x
"'
..
Sin
-
11-
S2n
12
smn
.In]
j
1
1 matrix representing the concentration of n fluorophore species that is to be
determined, [S] is an m
x
n matrix representing the known spectra of the n species distributed in m
detected spectral channels, and [J] is an m
x
1 matrix representing the measured spectrally mixed signals
of all the fluorophores collected in the mth spectral channel. If [S] is known, retrieving [I] can be simply
done by matrix inversion. [S] can be readily estimated in two ways. First, imaging each fluorophore with
our system can provide the required spectral distribution information (Fig 5.3 (a)). This method gives
perfect estimation, but it requires multiple calibration measurements performed in advance. The second
98
method is to estimate [S] with the known spectrum of each fluorophore. For the typical organic dyes or
fluorescent proteins, their emission spectra are well known. With these known spectra, the required
spectral distribution of each fluorophore can be estimated using the known wavelength dependent
detection efficiency of our microscope taking into account transmittance of optics (e.g. dichroic
and
barrier filters) and the quantum efficiency of the MAPMT (Fig 5.3 (b)). In any case, the estimated spectral
distribution is normalized to make the sum of the coefficients one so that the total photon count
is
preserved after unmixing.
ChI
Ch2
Ch3
Ch4
Ch6
Ch5
7
C8
(%)
10
(b)
0 400
Chl
4-5
Ch2
500
Ch3
515
500
Ch4
530
600 (nm)
Ch5
545
Ch6
560
Ch7
575
Ch
540
605 -
Fig 5.3. Spectral distribution estimation by (a) imaging the single-color samples with 8 channels or (b) based on the
known emission spectrum of each specimen. The graph in (b) shows the total detection efficiency as a function of
wavelength. The spectral calibration is discussed in section 3.1.
5.3. Result
5.3.1. Detection channel calibration
For spectral calibration of the eight spectral channels, several known samples have been used: Quinine
Sulfate (emission peak (Em): 460 nm, R-14782, Molecular Probes, Eugene, OR), Fluorescein (Em: 513
nm, R-14782, Molecular Probes, Eugene, OR), yellow-green beads (Em: 515 nm, F8859, Molecular
99
Probes, Eugene, OR), orange beads (Em: 560 nm, F8833, Molecular Probes, Eugene, OR), red beads (Em:
605 nm, F8842, Molecular Probes, Eugene, OR), and several second harmonic generation signals from a
collagen sample (475 nm to 537.5 nm with an increment of 12.5 nm from the Ti-Sa laser that generates
the maximum output wavelength of 1075 nm). With these calibration samples, the detector was installed
to collect from 485 nm to 605 nm with every 15 nm wavelength window.
5.3.2. Three color fluorescent beads
We first evaluated the performance of the spectral-resolved MMM with a three color fluorescent bead
sample. Three different fluorescent polystyrene microspheres (F8859: yellow-green, F8833: orange, and
F8842: red, Molecular Probes, Eugene, OR) were mixed and immobilized in 3D by 2 % agarose gel
(UltraPure Low Melting Point Agarose, Invitrogen, Carlsbad, CA). Different sizes of beads (4, 10, and 15
pm diameter) were intentionally chosen for each color bead so that the correct species assignment based
on spectral unmixing can be readily verified based on bead size. The excitation wavelength was 910 nm,
the laser power was about 5mW per focus, and the dwell time was 40 ps per 0.5 pm pixel resolution. Fig
5.4 shows the three bead image. The acquired raw images for each wavelength channel are shown in (a).
Individual raw image is not directly corresponding to different types of beads due to the typically board
fluorescent spectra of fluorophores and the resulting spectral overlap. Since the spectrum of each bead is
distributed differently onto the eight spectral channels, the identity of each beads can be determined based
on the spectral unmixing algorithm as described in section 2.2. Fig 5.4 (b) shows the spectral distribution
of the three color bead over the eight channels. The fractional intensities of each spectrum on each
spectral channel are normalized such that the area under the each curve is unity preserving the original
photon count in the spectral unmixing process. From the acquired coefficients, the eight raw images were
processed with matrix calculation for the three decomposed images shown in Fig 5.4 (c). The final
processed image showed the spectral-resolved MMM could successfully image the three color bead
sample with spectral resolution at four times faster speed than single focus scanning. The full-width at
100
half-maximum (FWHM) of the three different beads expectedly showed good agreement with their
specified sizes (4:1, 10.1, and 15.2 pm for each 4, 10, and 15 pm bead) demonstrating the spatial
resolution of this system. For over hundreds of beads imaged, there was no mis-assignment in the spectral
unmixing process.
(b)
0.5
0.4
0.3
0.2
0.1
01
1
2
3
4
5
6
7
8
Fig 5.4. Three color fluorescent bead image. (a) Eight channel images in pseudo colors for display purpose. The raw
image of each channel contains only combined intensity information. (b) The spectrum of each bead on the eight
channels. (c) Processed image in pseudo colors. The image size is 340 pm
x
340 jm.
5.3.3. Mouse kidney image
We further tested the system with a fixed mouse kidney sample labeled with three fluorescent dyes
(FluoCells@ Prepared Slide #3, F24630, Molecular Probes, Eugene, Oregon). Red-fluorescent Alexa
Fluor@ 568 phalloidin was used to label the filamentous actin. Green-fluorescent Alexa Fluor@ 488
wheat germ agglutinin was used to label glycoproteins or glycolipids in the membrane. Finally, the nuclei
were counterstained with the blue-fluorescent DAPI. The excitation wavelength was 780 nm, the laser
101
power was about 11 mW per focus, and the dwell time was 40 ps per 0.5 Pm pixel resolution. The
acquired raw images are typically not uniform (the center area is brighter than the edge area) because
while the x and y scanning mirrors are closely spaced, they cannot be both at the eye-point of the
scanning system. In addition, the four foci are under different aberration condition, so the center two subimages are brighter than the top and bottom sub-images. To avoid the non-uniformity in the
raw images,
they were normalized with rhodamine solution images. A 1 mM rhodamine 6G solution was imaged with
each spectral channel by moving the MAPMT to record the same peak intensity signal onto each channel,
and the acquired raw images were normalized with the rhodamine solution images. Similar to the three
color bead sample described in section 3.2, the three emission signals of the mouse kidney sample were
recorded in the eight channels. For the fluorescent bead image, the unmixing is very straight forward
since the beads are sparsely distributed beads spatially with almost no mixed contribution from more than
one species in a single pixel. However, in this mouse kidney sample the three fluorophores are distributed
within the complex tissue structures with significant spatial overlaps resulting in significant spectral
overlap within each pixel. We first estimate of the spectrum of each fluorescent dye based on their known
emission spectrum (Molecular Probes, Eugene, Oregon) and the wavelength dependent detection
efficiency of this MMM system as described in section 2.2. The three colors were decomposed with the
estimated spectral distribution, and showed in one image. Fig 5.5 shows that the system could
successfully image the sample with details in the three different fluorophores. The acquired image
matched well with images of similarly labeled samples acquired with other spectral resolved methods [29].
The distribution of three unmixed species corresponded well to the expected morphology of the
underlying tissue components. The blue arrow in Fig 5.5 points to a representative nucleus labeled with
DAPI showing the expected ellipsoidal geometry. The green arrows in Fig 5.5 point to tufts of capillaries
forming the glomeruli in the kidney with the endothelial cell plasma membrane labeled with wheat germ
agglutinin. The red arrow in Fig 5.5 points to filamentary structures of actin cytoskeleton of cells
contained in the glomeruli and the brush border of proximal convoluted tubules. It should be noted that
102
the image size was 290 pm
x
340 gm rather than a square shape due to the mismatch of the laser
wavelength and the DOE specification. The DOE installed in the system was designed to generate
4
beamlets with 0.8
0
x
4
separation angle at 910 nm wavelength. However, in this case 780 nm wavelength
was used for efficient excitation. Therefore, the excitation laser was slightly less diffracted than
the
designed angle resulting in the shortening of one spatial dimension by 50 pm.
Fig 5.5. A mouse kidney image labeled with three colors. The red arrow indicates representative actin labeled with
phalloidin, the green arrows show glomeruli labeled with Alexa 488, and the blue arrow for nuclei.
5.3.4. Nerve sample image
The capabilities of the spectral-resolved MMM are further demonstrated by imaging ex vivo samples of
rat sciatic nerves that had been fully transected, entubulated inside a porous collagen scaffold, and
harvested from the sacrificed animal one week later [30]. The nerve tissue was gently fixed by 4%
paraformaldehyde, embedded in OCT medium, sectioned, stained using a primary antibody against a
103
smooth muscle actin (aSMA; A5228, Sigma Aldrich, St. Louis, MO) that is detected by an Alexa Fluor
488-conjugated anti-mouse secondary antibody, and counterstained by DAPI (binds nucleic acids), and
TRITC-conjugated phalloidin (binds to F-actin) [31]. The laser excitation wavelength was 780 nm, power
per focus was about 18 mW, and the dwell time was 40 ps per 0.5 gm pixel. The image field of view was
290 x 340 pm, same as the mouse kidney image shown in section 3.3. Non-uniform excitation over the
image is corrected by normalizing the acquired images with the rhodamine solution image at each channel
as described in the mouse kidney image. The signal contribution of the three fluorescent sources (DAPI,
Alexa Fluor 488, TRITC) were estimated based on the estimated emission spectra over the eight channels
as described in section 3.3. Fig 5.6 shows the nerve sample image. Each color is analogous to the
fluorescent emission of one fluorescent source (red: TRITC, green: Alexa488, blue: DAPI). The
instrument can successfully distinguish the spatial distribution of the fluorophores in the nerve sample in
agreement with the published morphology of injured nerve samples [30]. The image consists of three
parts: the nerve tissue (left), the inner lumen of the scaffold (right) and a cell-rich "capsule" that forms
between the nerve and the scaffold (center). Phalloidin stains dimly sheath-like actin structures around the
Schwann cells that surround axons, and tiny blood vessels in the nerve fascicle. A continuous line that
stains strongly for phalloidin highlights the nerve perineurium, a thin layer of contractile cells around the
fascicle. The perineurium is surrounded by epineurium, a collagenous connective tissue that stains dimly
for DAPI (corresponds to the weak collagen autofluorescence) and acSMA (non-specific antibody
binding). The bright circular blue feature at the center of the capsule is the autofluorescence emission of
the stitch added by the surgeon to keep the nerve and the scaffold together. The stitch is surrounded by
contractile cells that stain for phalloidin. In the capsule there are also several large blood vessels that stain
brightly for both actin and aSMA. The scaffold stains dimly for collagen (dim DAPI signal) and also
contains some weak acSMA signal (non-specific antibody binding). A number of contractile cells are
bound on the scaffold. These contractile cells stain always strongly for actin and sometime for aSMA.
104
Fig 5.6. A nerve image surrounded by a scaffold labeled with three colors.
5.3.5. Mouse brain image
Lastly, we have performed in vivo imaging of neuron dendrite morphology and inhibitory synapse
distribution in a rodent brain with the spectral-resolved MMM. To obtain mice that express fluorescent
labels in a sparse subset of layer II/III pyramidal neurons in visual cortex, we in utero electroporated cre
recombinase dependent expression plasmids for eYFP (for neuronal morphology) and Teal-Gephyrin (for
inhibitory synapses) in timed pregnant C57BL6 mice as previously described in the reference [17].
Between 6-8 weeks of age, mice were anesthetized with isoflurane (3% for induction and 2% during
surgery) and a 5mm glass coverslip was placed following craniotomy over the visual cortex. Imaging
was performed on adult mice (>3months) anesthetized with isoflurane (3% for induction, 1-2% for
imaging). The head was positioned in a custom made stereotaxic restraint affixed to a stage. Individual
eYFP labeled dendrites and Teal-Gephyrin labeled synapses in layers 2/3 of visual cortex of an
anesthetized adult mouse were imaged in the spectral-resolved MMM. As a comparison, the same animal
was imaged using a single focus scanning two-photon microscope with two spectral channels. The single
105
focus scanning two-photon microscope was previously described in Ref [17]. Specifically, the excitation
light path is practically identical to the spectral-resolved MMM except only a single focus is generated.
The major difference of the single focus system is that the spectral-resolved detection is performed in a
non-descanned configuration. The emission photons are collected by the same objective lens (W PlanApochromat, 20
x,
1.0 NA, Zeiss, Thornwood, NY), and separated from the excitation beam path by a
short-pass dichroic mirror (750dcspxr, Chroma Technology, Bellows Falls, VT). The emission photons
are separated again by another dichroic mirror (520dcxr, Chroma Technology, Bellows Falls, VT) at 520
nm, which is the middle of the eYFP and Teal-Gephyrin emission peaks, and mapped onto two PMTs
(R7400U-01 and R7400U-20, Hamamatsu, Bridgewater, NJ) locating at the conjugate plane of the back
focal plane of the objective with the tube lens of the microscope (AXIOSKOP 2 FS, Zeiss, Thornwood,
NY) and another lens (KPX079AR. 14, Newport, Irvine, CA) in 4-f location. There are a BG39 (Newport,
Irvine, CA) and a bandpass filter (hq485/70m-2p for Teal-Gephyrin and et560/80m-2p for eYFP, Chroma
Technology, Bellows Falls, VT) in front of the two PMTs to block the excitation IR and get the photons
in the designed spectral range. For these in vivo experiments, the excitation wavelength was 910 nm, the
laser power was about 50 mW per focus, the dwell time was 40
image size was 340 pm
x
sis with
0.5 pm pixel resolution, and the
340 pm for both the spectral-resolved MMM and the single focus scanning
microscope. For flat field correction, the acquired images were normalized with the rhodamine solution
images acquired in each imaging system as described in section 3.3. For calibration measurements to
obtain the true spectra for Teal and eYFP, HEK293 cells were singly transfected with plasmids
pFueYFPW (for eYFP expression) or pFUTealW (for Teal expression) using calcium phosphate mediated
gene transfer [32]. Transfected cells with each fluorescent labels were imaged in both systems. Imaging
the cells labeled with the only single color provided the spectral distribution information of each label for
the spectral decomposition. After the spectral unmixing the images were processed with background
rejection and 2D B 3-Spline filter as a low pass filter as done in the previous publications [10, 17] since the
dot-shape synapses are only about one micron size occupying 3 x3 pixels and the maximum photon count
106
is only around 30, so the raw image is easily degraded by photon shot noise. While simple
low pass
filtering after unmixing provides fairly satisfactory results, more advanced unmixing algorithms
including
Poisson noise removal algorithm based on variance stabilization or maximum likelihood estimation
may
further improve our ability to identify small synapses [33-35].
Fig 5.7. A mouse brain image labeled with two colors. (a) Spectral-resolved MMM image, (b) single focus scanning
image and the enlarged images of the specified areas.
Fig 5.7 (a) shows the neuronal image taken by the spectral-resolved MMM and (b) is the image by the
single focus scanning two-photon microscope at 60 pm depth. The bottom images are the enlarged images
107
of the specified area for details. The red line-shape structures are dendrites labeled with Teal-Gephyrin,
and the green dots are synapses labeled with eYFP. The displayed red and green colors are pseudo-colors
for better contrast. The spectral-resolved MMM acquired the image 4 times faster than the single focus
scanning reducing the acquisition time of a general 1-2 hour brain imaging session [10, 17] to 15-30
minutes. The speed improvement can be maximized further by optimizing the DOE for more foci as
described in the method section.
Fig 5.8. A mouse brain image labeled with two colors. (a) Spectral-resolved MMM image at 184 pm depth, and (b)
at 156 pm depth. (c) The processed image of (a) with the ghost image removal algorithm, and (d) the processed
image of (b). (c) Single focus scanning image at 156 pm depth.
It is known that the SNR of the MMM is lower than that of the single focus scanning two-photon
microscope resulting in limited imaging depth [15], which is same for the spectral-resolved MMM.
However, the typical imaging depth of 100-300 pm in the brain plasticity research can be achieved with
MMM with MAPMT in the descanned configuration. Fig 5.8 shows the spectral-resolved MMM images
108
of the same animal used in Fig 5.7 at larger imaging depths. The raw images (Fig 5.8 (a) and (b)) showed
image artifacts, appearing as ghost sub-images, due to scattering of the specimen from the large imaging
depth, so they were processed with ghost image removal algorithm previously described in Ref [15]. The
spectral-resolved MMM images at the larger imaging depths showed clear neuronal cell bodies and
dendritic morphology (Fig 5.8 (c) and (d)) that is compared with single focus scanning system Fig 5.8 (e).
5.4. Discussion and Conclusion
MMM has successfully achieved the imaging speed improvement with the parallelized excitation. For
applications that require spectral resolved imaging, we have developed a spectral-resolved MMM system
that provide both high throughput and high content images. This spectral-resolved MMM is designed
based on using one dimension of the MAPMT for resolving the signals from a multiple foci linear array
while using the other dimension for resolving the spectral content of each foci. Within the available laser
power the number of foci can be maximized resulting in maximum imaging speed improvement. This
system has been demonstrated to successfully perform spectral resolved imaging deep in mouse brain in
vivo with comparable SNR as single focus system but much higher speed.
This spectral-resolved MMM based on MAPMT is an interesting addition to other previous developed
spectral-resolved MMMs. The use of the MAPMT with single-photon counting circuitry can achieve
much higher frame rate compared with the system using imaging cameras [22], especially ones with long
dead-time streak cameras [19-21]. However, streak camera has the advantage of providing lifetimeresolved imaging capability. In addition, multiple readouts (equal to the number of lines in an image) are
required for these camera systems resulting in very undesirable accumulation of camera readout noise
substantially degrading image SNR. Moreover, MMMs using cameras as detectors are vulnerable to
scattering of emission photons in turbid specimens resulting in further degradation of SNR and limited
imaging depth [15]. The pulse multiplexing MMMs have the advantage of mostly immune to emission
photon scattering similar to single focus scanning systems [23, 24]. The pulse multiplexing system further
109
has the advantage that spectral detection can be readily accomplished with a grating spectrometer-camera
system. A very promising system with two multiplexed beams has been demonstrated [25]. However,
these pulse multiplexing MMMs require custom laser sources that is not broadly available. More
importantly, the maximum number of time multiplexed beams generated by these laser sources are
currently limited to less than ten resulting in slower imaging system compared with more conventional
MMMs that can readily utilizes tens to hundreds of foci with a standard Ti-Sapphire oscillator.
One major disadvantage of MMM system based on MAPMT is the crosstalk between the foci resulting in
image artifacts appearing as ghost images in the neighbor sub-images. Therefore, image post processing
to unmix signals from the different foci may be required in highly turbid specimen or deep image [15].
More advanced cross talk removal algorithms based on a maximum likelihood approach is being
developed [36].
The current eight channel detection allows us to image maximum eight different fluorophores, but
practically to avoid severe spectral overlap, 3~4 color imaging would be a reasonable choice. To image
more fluorophores at the same time, multiple MAPMTs or a SPAD array can be used. However, since
different fluorophores have different excitation spectrum, simultaneous excitation with multiple excitation
wavelengths may be required. Since DOE is wavelength dependent, the use of multiple excitation
wavelengths can significantly complicate the design of this type of MMM systems.
One exciting future direction is the incorporation of simultaneous spectral and lifetime resolved imaging
capability [19-21, 37]. For MMM systems based on MAPMT, high speed electronics for high speed
fluorescence lifetime measurement can be readily implemented at the output anodes of 4x4 or 8x8
MAPMTs. A low cost, highly parallelized time-correlated single photon counting system based on high
speed field programmable gate array electronics is under development.
For specimens with bright emission signals, spectral decomposition for fluorophores with sufficiently
well separated spectra is often straight forward. However, for low level emission signals, the success of
spectral decomposition is often limited by the inherent Poisson noise in the image data resulting in
110
significant uncertainty during
decomposition
and
potential
spectral unmixing
misinterpretation
[38].
of the
This error
image
data.
results in improper
Therefore,
further
spectral
spectral
decomposition strategy such as image processing with the application of Poisson noise removal
algorithms will be required [33-35].
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
Denk, W., J.H. Strickler, and W.W. Webb, 2-PHOTON LASER SCANNING FLUORESCENCE
MICROSCOPY Science, 1990. 248(4951): p. 73-76.
So, P.T.C., et al., Two-photon excitation fluorescence microscopy. Annual Review of Biomedical
Engineering, 2000. 2: p. 399-429.
Zipfel, W.R., R.M. Williams, and W.W. Webb, Nonlinear magic: multiphoton microscopy in the
biosciences. Nature biotechnology, 2003. 21(11): p. 1369-1377.
Helmchen, F. and W. Denk, Deep tissue two-photon microscopy. Nature methods, 2005. 2(12): p. 932-940.
Denk, W., et al., Anatomical andfunctional imaging of neurons using 2-photon laser scanning microscopy.
Journal of neuroscience methods, 1994. 54(2): p. 151-162.
Yuste, R. and W. Denk, Dendritic spines as basicfunctional units of neuronal integration.Nature, 1995.
375(6533): p. 682-684.
Svoboda, K., et al., In vivo dendritic calcium dynamics in neocorticalpyramidal neurons. Nature, 1997.
385(6612): p. 161-165.
Lee, W.C.A., et al., Dynamic remodeling of dendritic arbors in GABAergic interneurons of adult visual
cortex. Plos Biology, 2006. 4(2): p. 271-280.
Lee, W.C.A., et al., A dynamic zone defines interneuronremodeling in the adult neocortex. Proceedings of
the National Academy of Sciences, 2008. 105(50): p. 19968-19973.
Chen, J.L., et al., Structural basis for the role of inhibition in facilitating adult brain plasticity. Nature
neuroscience, 2011. 14(5): p. 587-594.
Bewersdorf, J., R. Pick, and S.W. Hell, Multifocal multiphoton microscopy. Optics Letters, 1998. 23(9): p.
655-657.
Buist, A.H., et al., Real time two-photon absorption microscopy using multi point excitation. Journal of
Microscopy-Oxford, 1998. 192: p. 217-226.
Sacconi, L., et al., Multiphoton multifocal microscopy exploiting a diffractive optical element. Optics
Letters, 2003. 28(20): p. 1918-1920.
Watson, B.O., V. Nikolenko, and R. Yuste, Two-photon imaging with diffractive optical elements. Frontiers
in neural circuits, 2009. 3.
Kim, K.H., et al., Multifocal multiphoton microscopy based on multianode photomultiplier tubes. Optics
Express, 2007. 15(18): p. 11658-11678.
Gray, N.W., et al., Rapid redistributionof synaptic PSD-95 in the neocortex in vivo. PLoS biology, 2006.
4(11): p. e370.
Chen, J.L., et al., Clustered dynamics of inhibitory synapses and dendritic spines in the adult neocortex.
Neuron, 2012. 74(2): p. 361-373.
van Versendaal, D., et al., Elimination of inhibitory synapses is a major component of adult ocular
dominanceplasticity. Neuron, 2012. 74(2): p. 374-383.
Qu, J., et al., Temporally and spectrally resolved sampling imaging with a specially designed streak
camera. Optics letters, 2006. 31(3): p. 368-370.
Liu, L., et al., Simultaneous time-and spectrum-resolved multifocal multiphoton microscopy. Applied
Physics B, 2006. 84(3): p. 379-383.
Liu, L., et al. Development of a Simultaneously Time-and Spectrum-Resolved Multifocal Multiphoton
Microscopy System Using a Streak Camera. in Biophotonics, Nanophotonics and Metamaterials, 2006
Metamaterials2006. InternationalSymposium on. 2006: IEEE.
111
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
Shao, Y., et al., High-speed spectrally resolved multifocal multiphoton microscopy. Applied Physics B,
2010. 99(4): p. 633-637.
Amir, W., et al., Simultaneous imaging of multiple focal planes using a two-photon scanning microscope.
Optics Letters, 2007. 32(12): p. 1731-1733.
Chandler, E., et al., High-resolution mosaic imaging with multifocal, multiphoton photon-counting
microscopy. Applied optics, 2009. 48(11): p. 2067-2077.
Chandler, E.V., C.G. Durfee, and J.A. Squier, Integrated spectrometer design with application to
multiphoton microscopy. Optics Express, 2011. 19(1): p. 118-127.
Cova, S., et al., 20-ps timing resolution with single-photon avalanche diodes. Review of scientific
instruments, 1989. 60(6): p. 1104-1110.
Dautet, H., et al., Photon counting techniques with silicon avalanchephotodiodes. Applied optics, 1993.
32(21): p. 3894-3900.
Cova, S., et al., Avalanche photodiodes and quenching circuitsfor single-photon detection. Applied optics,
1996. 35(12): p. 1956-1976.
MolecularProbes. FluoCells@ PreparedSlide #3 (Mouse Kidney Section with Alexa Fluor®488 WGA,
Alexa
Fluor®
568
Phalloidin,
DAPI)
Available
from:
http://www.lifetechnologies.com/order/catalog/product/F24630?ICID=search-f24630.
Soller, E.C., et al., Common features of optimal collagen scaffolds that disrupt wound contraction and
enhance regenerationboth in peripheralnerves and in skin. Biomaterials, 2012. 33(19): p. 4783-4791.
Buydash, M., Mechanical Modeling of Tissue Response DuringEarly Stage EntubulatedPeripheralNerve
Regeneration, in DepartmentofMechanicalEngineering.2013, Massachusetts Institute of Technology.
Graham, F. and A. Van der Eb, A new techniquefor the assay of infectivity of human adenovirus 5 DNA.
Virology, 1973. 52(2): p. 456-467.
Nowak, R.D. and E.D. Kolaczyk, A statistical multiscale framework for Poisson inverse problems.
Information Theory, IEEE Transactions on, 2000. 46(5): p. 1811-1825.
Zhang, B., J.M. Fadili, and J.-L. Starck, Wavelets, ridgelets, and curvelets for Poisson noise removal.
Image Processing, IEEE Transactions on, 2008. 17(7): p. 1093-1108.
Spring, B. and R. Clegg, Image analysis for denoisingfull-field frequency-domain fluorescence lifetime
images. Journal of microscopy, 2009. 235(2): p. 221-237.
Cha, J.W., et al. Spectral Decomposition of Multicolor Imaging in Multifocal Multiphoton Microscopy. in
Biomedical Optics. 2010: Optical Society of America.
Pelet, S., M.J. Previte, and P.T. So, Comparing the quantification of Forster resonance energy transfer
measurement accuracies based on intensity, spectral, and lifetime imaging. Journal of biomedical optics,
2006. 11(3): p. 034017-034017-11.
Pawley, J., Handbook of biologicalconfocal microscopy. 2006: Springer.
112
Chapter 6
Conclusion and Outlook
This thesis has presented several significant improvements in multiphoton microscope design with a
particular emphasis in improving the performance of multifocal multiphoton microscopes. First, adaptive
optics (AO) compensation was introduced to improve signal-to-noise ratio (SNR) and resolution degraded
due to the wavefront distortion by tissue specimens with inhomogeneous optical properties. The ShackHartmann wavefront sensor with a confocal depth selection mechanism could measure the wavefront
distortion, and the deformable mirror could compensate the aberrations in the feedback loop. The AO
compensation could achieve up to two fold signal improvement for peak intensities while suppressing the
background signals improving image contrast. However, it has been also found that tissue scattering was
the primary factor that limits the maximum imaging depth in tissue imaging by two-photon microscope.
Given the limited imaging depth, the importance of AO compensation in many systems is relatively
limited.
Secondly, a novel emission photon reassignment algorithm has been developed to overcome signal
crosstalk between signals from different foci in MAPMT based MMMs. This photon reassignment
algorithm is based on optical model based maximum likelihood optimization with minimal a prior
measurements. This image post-processing method could significantly increase the SNR of the final
processed image by reassigning the scattered photons to the original spatial locations and minimizing
image artifacts.
Further, a new non-descanned MAPMT based MMM was developed to minimize the crosstalk between
foci. This MMM could minimize crosstalk artifacts, appearing as ghost images, through maximizing foci
separation by fully utilizing the large field-of-view (FOV), high numerical aperture (NA) objective lenses.
113
One complication in designing MMM systems with these large FOV objectives is that the edge and
corner foci are susceptive to optical aberration if the intermediate optics of the microscope system is not
optimized. However, the optimization of intermediate optics is found to be incompatible with the use of
small, high speed galvanometric scanners and high NA objectives with large back aperture in the
descanned configuration. Implementing MMM system with a novel non-descanned geometry could
minimize the optical aberration of the edge and corner foci allowing full use of the large FOV objectives
resulting in maximized foci separation. In addition, the non-descanned MMM further minimized the
detection path length so that the signal collection efficiency could be improved up to four-fold. To detect
the non-stationary emission signals and avoid the dead space of the MAPMT, a detection scheme that
involves four times shifting of the image field was successfully demonstrated.
Finally, a spectral resolved MMM based on MAPMT was introduced to simultaneously improve imaging
speed and enable spectral resolved detection. This system is configured by using one dimension of the
MAPMT to detect signal from a linear array of excitation foci while using the other dimension for the
spectral-resolved detection. We have demonstrated a system that generates four excitation foci along a
line providing four time speed improvement. The signals from the four foci were distributed into eight
spectral channels allowing simultaneous 3-4 color imaging in practice. In future implementation, clearly
more foci can be generated with a different diffractive optical element (DOE) providing higher speed. In
addition, imaging faster with more foci and/or imaging with high spectral resolution can be readily
accomplished by implementing multiple MAPMTs together or by using single-photon avalanche diode
(SPAD) array with more pixel elements. Simple spectral unmixing algorithm is used in this work but
more advanced algorithm incorporating ideas such as Poisson noise removal algorithm is under
development that promises to allow unmixing of images with more overlapped spectra or images with
very low SNR.
In any case, it is true that the standard single focus scanning two-photon microscope will often have better
SNR than MMM systems. It is also true that single focus scanning two-photon microscope will often have
114
better imaging depth than MMM systems. However, we have shown that MMM systems offer significant
improvement in imaging speed that single focus system cannot rival (roughly equal to the number of foci
generated) with only a minimal loss in SNR. Clearly, all imaging methodologies have their strengths and
weakness. The appropriate imaging method should be selected to target a particular biomedical
application. For example, there
are many applications where the specimens contain
sufficient
fluorophores to provide sufficient SNR, and the use of MMM can greatly reduce image acquisition time
and provide the opportunity to study system dynamics on the millisecond to second time scale. As another
example, if imaging with both high speed and high spectral resolution is required, spectral-resolved
MMM may be an excellent system for this type of tissue imaging applications.
115