High Throughput 3D Optical Microscopy :
from Image Cytometry to Endomicroscopy
by
Heejin Choi
B.S., Pohang University of Science and Technology (1999)
M.S., Korea Advanced Institute of Science and Technology (2003)
Submitted to the Department of Mechanical Engineering
in Partial Fulfillment of the Requirements for the Degree of
Doctor of Philosophy
MASSACHusrrs iNsrfjE~
OF TECHNOLOGY
at the
MAY 0 8 2014
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
LIBRARIES
February 2014
C 2014 Massachusetts Institute of Technology.
All rights reserved
Signature of A uthor................................................
..................
Department of Mhanical Engineering
Jan 15, 2014
C ertified by ..................................................
...........................................
Peter T.C. So
Professor of Mechanical Engin ing and Biological Engineering
IXesioAervisor
A ccepted by ........................................
.............
..........
David E. Hardt
Chairman of the Departmental Committee for Graduate Students
High Throughput 3D Optical Microscopy :
from Image Cytometry to Endomicroscopy
by
Heejin Choi
Submitted to the Department of Mechanical Engineering on January 15, 2014
in Partial Fulfillment of the Requirements for the Degree of
Doctor of Philosophy in Mechanical Engineering
Abstract
Optical microscopy is an imaging technique that allows morphological mapping of intracellular
structures with submicron resolution. More importantly, optical microscopy is a technique that
can readily provide images with biochemical contrast based on different spectroscopic modalities
such as fluorescence spectrum and lifetime, Raman spectrum, optical polarization and phase.
Although, optical microscopy can provide superior resolution over many other medical imaging
modalities such as MRI, CT or ultrasound, the relatively low throughput limits its range of
biomedical application.
In this thesis, high throughput 3D optical imaging instruments have been developed for
medical and biological applications based on widefield 3D resolved techniques. First, we
developed a high throughput depth resolved widefield image cytometer based on the structured
light illumination and the high speed remote depth scanning. This system improves imaging
throughput by an order of magnitude over the current technology and can potentially be applied
to image cytometry investigation to study cultured cell morphologies with statistical significance
comparable to the flow cytometer. The statistical accuracy of this instrument is verified by
quantitatively measuring the rare cell populations. Hyperspectral imaging is also possible based
on the use of an interferometric full field spectrometer. Second, we developed a depth resolved
widefield two photon endomicroscope for the medical diagnosis based on the temporally focused
widefield two photon microscopy. The developed instrument can parallelize the image
acquisition process over the whole field of view without the need for any scanning mechanism at
the distal end of the optical fiber. The method for delivering high peak power laser pulses
1
through the optical fiber has been proposed to improve the signal to noise ratio which in turn
improves the imaging throughput. A structured light illumination method have also been
proposed and demonstrated for improving depth resolution. In addition, this temporally focused
widefield two photon microscopy has been applied to the high throughput depth resolved
measurement of fluorescence and phosphorescence lifetime with millisecond level frame rate.
Thesis supervisor: Peter. T.C. So
Title : Professor of Mechanical Engineering and Biological Engineering
Thesis committee : Martin L. Culpepper
Title : Professor of Mechanical Engineering
Thesis committee: Scott Manalis
Title : Professor of Mechanical Engineering and Biological Engineering
Thesis committee: George Barbastathis
Title : Professor of Mechanical Engineering
2
Acknowledgement
I would not have been able to successfully finish my PhD without help from numerous people.
First of all, I would like to thank my advisor, Prof. Peter So. He is truly exceptional as an advisor.
From the moment I joined his group and until now, he was always ready to help me whenever
needed and have supported me to get through all the hardships during my PhD. When I first
joined the lab, I didn't know anything about optics. He patiently allowed me to first obtain basic
knowledge before I started doing real research. He is really a role model I want to follow as a
researcher and also as an educator. I also sincerely thank Prof. George Barbastathis, Prof. Scott
Manalis and Prof. Martin Culpepper who were willing to accept the role of thesis committee and
have provided me valuable advises during my PhD. I was fortunate to have a chance to work
with really smart people in SMART center in Singapore. I built the image cytometer system in
SMART center but I didn't have enough time to collect all the data. Dushan took over my project
and collected data for the rare cell counting experiments and performed image processing. Ting
Yuan was kind enough to help me prepare the sample for the rare cell counting experiment. It
started as a favor but he ended up repeating preparing sample several times for the precise
mixing ratio, especially 1 to 105 ratio. I also would like to thank Elijah and Vijay for their help
during my stay in Singapore. I also thank members of So Lab (Jaewon, Yanghyo, Chris, Yunho,
Dimitrios, Sebastian, Poorya, Barry and Yi). Especially, Yanghyo and Chris were willing to
sacrifice their own time for helping me aligning the pulse splitter and generously allowed me to
use the regenerative amplifier first to finish up the endoscope experiment in my last stage of PhD.
I also thank former members of So Lab who helped me to settle down when I first joined the lab.
I thank KGSAME for their friendship. I also thank members of Korean Church who supported
me spiritually when I was going through hard times during my PhD. At last, I would like to
thank my parents who always have supported for me to come to this stage and would be the most
pleased with my completion of PhD degree.
3
This page is intentionally left blank
4
Table of Contents
1. Introduction to 3D optical microscopy..........................................................
17
1.1 Confocal laser scanning microscopy............................................................17
1.2 Two photon laser scanning microscopy.........................................................19
1.3 Temporally focused widefield two photon microscopy.........................................20
1.4 Light sheet microscopy ..........................................................................
21
1.5 Organization of the thesis........................................................................22
2. Three-dimensional image cytometer based on widefield structured light microscopy and
high-speed remote depth scanning...................................................................25
2.1 Introduction .........................................................................................
25
2.2 Design of the overall system.....................................................................27
2.3 Structured light illumination (SLI) for depth-resolved widefield imaging................29
2.4 Axial resolution of the SLI measured with a thin Rhodamine solution. ................... 30
2.5 High speed remote depth scanning..............................................................
32
2.6 Application of the high throughput 3D image cytometer....................................
34
2.7 Rare Cell Detection in 2D cell culture............................................................36
2.7.1 2D cell culture preparation.....................................................................
36
2.7.2 Data acquisition and analysis................................................................
37
2.7.3 Image processing procedure ...................................................................
40
2.8 C onclusions ..........................................................................................
43
3. Depth resolved hyperspectral imaging cytometer based on structured light illumination
and Fourier transform interferometry.............................................................47
3.1 Introduction ..........................................................................................
47
3.2 Overall instrument design of the imaging Fourier transform spectrometer..................49
3.3 Principle of imaging Fourier transform spectrometer based on Sagnac interferometer.....50
3.4 Depth-resolved imaging Fourier transform spectrometry.....................................55
3.5 Experimental validation of spectral background rejection...................................57
3.5.1 Hyperspectral imaging........................................................................57
3.5.2 Spectral background rejection by SLI......................................................58
3.6 C onclusion ...........................................................................................
4. Two photon point scanning endomicroscopy for clinical diagnosis...........................65
5
61
4.1 Review of the endoscopic microscopy.........................................................66
4.1.1 Non-depth resolved Endomicroscopes......................................................66
4.1.2 Depth resolved endomicroscopes...........................................................67
67
4.1.2.1 OCT endoscope ...........................................................................
4.1.2.2 C onfocal endoscope........................................................................68
4.1.2.3 Nonlinear optical (NLO) endoscope......................................................69
4.2 Side-view 3D point scanning two photon endomicroscope.................................72
4.2.1 Design of the two photon endomicroscope.................................................73
4.2.2 Characterization of the DCPCF for the two photon excitation..........................75
4.2.2.1 Single mode excitation beam delivery through the inner core of the DCPCF......75
4.2.2.2 Effect of the pulse dispersion during the beam delivery through DCPCF...........76
4.2.3 Imaging experiments and preliminary results.............................................78
5. Improving femtosecond laser pulse delivery through a hollow core photonic crystal fiber
for temporally focused two-photon endomicroscopy..........................................85
85
5.1 Introduction ........................................................................................
5.2 Improving the two photon excited fluorescence signal using pulse splitter..................86
5.3 Increase the damage threshold by stretching the pulse.......................................91
5.4 Design of a temporally focused widefield two photon endomicroscope (TFEM)...........95
5.5 Experimental validation of the pulse delivery methods.........................................97
99
5.6 C onclusion ...........................................................................................
6. Improvement of axial resolution and contrast in temporally focused widefield twophoton microscopy with structured light illumination.........................................103
6.1 Introduction .........................................................................................
103
6.2 Methods of generating structured light illumination in TFM.................................104
6.3 Depth resolution improvement using SLI in TFM.............................................109
6.3.1 Theoretical estim ation.........................................................................109
6.3.2 Experimental verification.....................................................................110
6.4 Contrast enhancement using SLI in TFM.......................................................113
6.5 C onclusion ...........................................................................................
114
7. 3D-resolved fluorescence and phosphorescence lifetime imaging using temporal focusing
wide-field two-photon excitation ...................................................................
6
119
7.1 Introduction .........................................................................................
119
7.2 Temporal focusing wide-field Two-Photon FLIM and PLIM................................121
7.3 Results and analysis.................................................................................123
7.3.1 Fluorescence lifetime measurement performance and applications ..................... 123
7.3.2 Demonstration of 3D phosphorescence and fluorescence lifetime-resolved imaging in
tissue engineering scaffolds .................................................................
7.4 C onclusion ...........................................................................................
129
134
8. Conclusion...............................................................................................141
8.1 Thesis summ ary ....................................................................................
8.2 Thesis contribution.................................................................................142
7
14 1
This page is intentionallyleft blank
8
List of Figures
Fig. 1.1 Typical configuration of the confocal microscope. A pinhole that is positioned at the
conjugate plane of the focal plane of the objective blocks out-of-focus light for 3D resolved
imaging. ... -- - --- -- -------------- -- ---...........................................................
18
Fig.1.2 Fluorescence signal generated near the focal region of the objective (a) one photon
excitation (b) two photon excitation. ...................................................................
19
Fig.1.3 Typical configuration of the two-photon microscope. The detector is positioned in the
non-descan position which collects all the scattered emission photons from the focal point of the
excitation beam ---......................... -...
.. ........................................................
20
Fig. 1.4 In spatial focusing, the pulse width (t) is fixed while the illumination area(A) is minimal
at focus. In temporal focusing, the illumination area (A) is fixed and the temporal pulse width(t)
is minim al at focus. ................... ............. ......................................................
20
Fig.1.5 Widefield TFM
------.......-----
....................................................
21
Fig. 1.6 Optical system arrangement of LSM. A cylindrical lens used to create a sheet of light that
penetrates the sample. Inset shows the close look of the illumination and selection mechanism in
LSM ...............- ............- ..-.-.-... -........................ ..........................................
22
Fig. 2.1 Schematic diagram of the high throughput 3D imaging cytometry. The current diagram
is for the high throughput imaging mode and the filter wheel is the detection path is used for the
spectral imaging. When the blue box rotate 180 degree and the beam splitter is engaged it
function as a hyperspectral imaging mode. M: mirror, DM: dichroic mirror, DOE: diffractive
optical element, OS 1 : optical shutter for minimizing the photobleaching effect, OS 2 : optical
shutter for generating UI and SI, PBS: polarizing beam cube splitter, FW: filter wheel with three
emission filter, QWP: quarter-wave plate, FW: band pass filter wheel, EmF: emission filter, PA:
piezo actuator, FFP: front focal plane, OBJ: imaging objective lens (Zeiss 20x Water NAL.0),
ROBJ: remote focusing objective lens (Nikon 20x Air NAO.75) .................................. 29
Fig.2.2 Background rejection with HiLo algorithm (a) Uniformly illuminated image (b)
Structurally illuminated image (c) HiLo processed image..........................................30
Fig.2.3 Optical sectioning measured with thin Rhodamine solution. WF: Widefield, HiLo,
Theory.......
--------------.....................
..........................................
31
Fig.2.4. Remote focusing PSF at different focal positions...........................................33
Fig.2.5 3D image stack of mouse kidney obtained with (a) Uniform illumination (b) after HiLo
processing. Green color represents elements of the glomeruli and convoluted tubules labeled
with Alexa Fluor@ 488 wheat germ agglutinin. Red color represents the filamentous actin
prevalent in glomeruli and the brush border labeled with red-fluorescent Alexa Fluor@ 568
phalloid in . ........... ......................... ............................................................
34
Fig.2.6. (a) 3D image of the muntjac skin fibroblast cells (F36925, Invitrogen). The green color
represents filamentous actin labeled with green fluorescent Alexa Fluor 488 phalloidin. The red
color represents mitochondria labeled with an anti-OxPhos Complex V inhibitor protein mouse
9
monoclonal antibody in conjunction with orange fluorescent Alexa Fluor@ 555 goat anti-mouse
IgG (b) TGFs activated fibroblast cells. Green color represents actin filament labeled with
rhodamine phalloidin and blue color represents SMAD protein labeled with Alexa Fluor 488.. .35
Fig.2.7 Multiscale view of mouse kidney. Ix image is stitched with 16x16 FOVs. Area marked
36
with white square is zoomed in successively. ............................................................
Fig.2.8 Representative 3D image of rare cell sample at 1:1 mixing ratio.........................38
Fig.2.9 Representative image of 1:10 4 ratio sample where the nucleus is labeled with m-Cherry
(represented as blue color) and the cytoplasm labeling with Cell Tracker Green (represented as
green color ). The size of single FOV is 420 pim x 350 im and the image shown is part of the
39
1:104 sample where 20x 15 FOVs are stitched together. ...........................................
Fig.2.10 Rare cell subpopulation counting result......................................................39
Fig.2.11 Cell counting pipeline (a) Nuclei counting (b) Rare cell counting.......................43
Fig.3.1. Schematic diagram of the high throughput depth-resolved imaging Fourier transform
spectrometer. Enclosed by dotted line is the Sagnac interferometer. M: mirror, DM: dichroic
mirror, DOE: diffractive optical element, OS: optical shutter, PBS: polarizing beam cube splitter,
BS: 50R/50T plate beam splitter, QWP: quarter-wave plate, EmF: emission filter, PA: piezo
actuator, FFP: front focal plane, OBJ: objective lens (Zeiss 20x Water NAL.0), ROBJ: remote
50
focusing objective lens (Nikon 20x Air NAO.75) .......................................................
Fig. 3.2 (a) Ray tracing from the intermediate image plane to the detector at central position of
the Sagnac interferometer. Color represents the spatial field points. (b) Photograph of the setup
installed in the emission beam path of the image cytometer. (c) OPD as a function of the
rotational angle of the Sagnac interferometer for the spatial field point on the optical axis (the
rays represented in green color in (a) with the following parameters : 8 = 450, t = 2.93mm, n =
1.517 (Refractive index of N-BK7) ....................................................................
52
Fig.3.3. Sampling of the interferogram (a) Fringe pattern of a white light LED at 0 = 00 sampled
by the sCMOS camera. The dotted yellow vertical line represents a zero OPD which shifts to the
left as the interferometer rotates. Pixels in each column see the same OPD and each column see a
linearly varying OPD (b) Sampled interferogram observed by the yellow pixel at AO = 0.005* (2
pixel or 170nm in OPD space) (c) Recovered spectrum of LED after quad band emission filter
53
with minimum detectable wavelength of 340nm. ....................................................
Fig. 3.4. Interferogram of 6pm yellow green fluorescent bead (a), Rhodamine 6G solution (b),
4Rm red fluorescent bead (c) and its the processed spectrum are (d), (e) and (f). Green curve in
spectral graph is the reference value from http://www.lifetechnologies.com/us/en/home/lifescience/cell-analysis/labeling-chemistry/fluorescence-spectraviewer.htm.......................55
Fig. 3.5 (a) Two color image of the muntjac skin fibroblast cells (F36925, Invitrogen). The
green color represents filamentous actin imaged with the 525nm/35nm bandpass filter (FF01520/35, Semrock) and the orange color represents mitochondria imaged with 609nm/54nm band
pass filter (FF01-609/54, Semrock). (b) X plane images of the region marked in red box in (a) at
58
selected w avelengths. ....................................................................................
10
Fig. 3.6 (a) Spectral images from UL, SI and after HiLo processing at X = 521nm and k = 608nm
(b) Spectra for UI and after HiLo processing. The spectrum from the out-of-focus Rhodamine
solution is rejected after the HiLo processing. .......................................................
59
Fig. 3.7 k plane images of the muntjac skin fibroblast cells (F36925, Invitrogen) with Rhodamine
background for UI and after HiLo processing. .......................................................
60
Fig. 3.8 True color images of mouse kidney (F24630, Invitrogen) for UI and after HiLo
processing. Green colored feature represents the elements of the glomeruli and convoluted
tubules labeled with Alexa Fluor 488. Orange colored feature represents the filamentous actin
and the brush border labeled with Alexa fluor 568. ................................................
61
Fig. 4.1 (a) Optical layout of the two photon excitation endomicroscope (b) Cross-section of the
DCPCF for the excitation and emission beam delivery. (c) Fiber resonator. The tip of DCPCF is
resonated by the fiber resonator for the beam scanning in fast (X) axis. (d) MEMS actuator
oscillates the prism for slow axis scanning (Y) and translate the GRIN lens for the depth
scanning (Z) (e) Endomicroscope assembled in the housing. DM : dichroic mirror, EF :
emission filter, PMT : photomultiplier tube, DCPCF : double clad photonic crystal fiber.........73
Fig. 4.2 (a) Far field image of the cross section of DCPCF. Single mode beam in core is shown
as bright triangular shape. Inner clad structure surrounds the core and used for emission beam
delivery. (b) Gaussian intensity profile of single mode beam from the core. (c) The wavefront of
the single mode beam from the core, RMS = 0.01 k................................................75
Fig. 4.3 Experimental setup for characterizing the dispersion of DCPCF. ...........................
76
Fig. 4.4 Temporal pulse width measurement by varying the distance between the grating pair.. .77
Fig. 4.5 Two photon efficiency measured with Fluorescein. The slope of each curve is close to 2,
which demonstrates that the signal is generated by the two photon excitation process. The
intercept of the linearly interpolated curve with the y axis is related to the two photon excitation
efficiency, which is inversely proportional to the pulse width of the excitation beam.............77
Fig. 4.6 Ratio plot of the TPE and the pulse width by varying the distance between the grating
pair. 0cm grating distance means no prechirping.TPE ratio = TPEtAir/ Photon Count DCPCF , Pulse
W idth ratio = FW HM DCPCF/ FW HM Air .......
............. ........................................
78
Fig. 4.7 A stack of fluorescent beads images with a field of view of 50 x 60 microns. Each bead
is of 15 micron in diameter and each frame is 3 micron apart in Z-axis. The images is acquired at
3.5 fram e/sec................................................................................................
78
Fig. 5.1 Far field image of the end surface of HC-800-01 fiber (NKT photonics) (a) clean
surface (b) damaged surface. These images are acquired by illuminating the proximal end of the
fiber with a white light LED and imaging the distal end with a 40x air objective and a CCD
camera. The bright dot shown is the photonic lattice structure where the light from LED
propagate through these structures via total internal reflection (c) intensity pattern of the output
beam from the hollow core fiber. .......................................................................
87
Fig. 5.2 Coupling efficiency plot of HC800-01 fiber. The length of fiber is 80cm..................89
Fig. 5.3 Schematic diagram of the 64x pulse splitter and the timing diagram of the pulse
multiplication at each step. After the pulse splitter, the beam goes through the single prism pulse
11
compressor to compensate the group velocity dispersion induced by HCPCF. The 4x pulse
splitter diagram in the box is adapted from, BE : beam expander, FM: flip mirror, BS 1 ,2 : nonpolarizing beam cube splitter, BS 3 : polarizing beam cube splitter , PSI: pulse splitter with 74 ps
pulse separation, PS 2 : pulse splitter with 37 ps pulse separation, DL : delay line, HWP: half wave
plate, P: prism, CCM: corner cube mirror, FCL: fiber coupling lens, HCPCF: hollow core
photonic crystal fiber .....................................................................................
91
Fig. 5.4 (a) Schematic drawing of single prism pulse compressor. Large tuning range of GDD is
achieved by using a prism made of highly dispersive glass, SF66. GDD is tuned by changing the
distance between the prism and the corner cube mirror. (b) Measured pulse width as a function of
the corner cube mirror position. (blue curve). GDD is calculated using Eq.(5.2) from the
92
measured pulse width. Input pulse width was 130 fsec2. ............................................
Fig. 5.5 Damage threshold of HC-800-02 (NKT photonics) measured without pulse stretching
(150 fsec) and CCM positions at 20 cm (187 fsec ), 30 cm (359 fsec), 40 cm (632 fsec), 50 cm
. . 93
(922 fsec). ...............................................................................................
Fig. 5.6 Pulse width measured after the fiber (HC-800-01) for various amount of negative GDD
from the pulse compressor. CCM at 0cm corresponds to the pulse width measured without the
pulse compressor. The minimal pulse width after the fiber is achieved when CCM is at 20cm
. . 94
position . ...................................................................................................
Fig. 5.7 Damage threshold of the fiber after 64x pulse multiplication without pulse stretching
95
(red) and with pulse stretching of CCM at 20cm position (blue). ...................................
Fig. 5.8 (top) Ray trace of the excitation beam path. Color represents wavelength. (bottom) Ray
trace of the emission beam path. Color represents spatial field. Excitation field of view (FOV) :
30 x 60 um, Working distance of objective : 180um, NA of objective: 0.8, All ray tracing is
97
performed by commercial software ( Zemax, Radiant Zemax) .....................................
Fig. 5.9 (a) 3D modeling of the assembled TFEM. (b) Photo showing the assembled TPE. Size :
97
8m m ( diameter), 35m m (length) .........................................................................
Fig. 5.10 (a) Normalized two photon signal for three cases. 1.No pulse multiplication without
dispersion compensation 2.With 64x pulse multiplication and without dispersion compensation
3.With 64x pulse multiplication and dispersion compensation (b), (c), (d) are 2pm bead images
98
for case 1, 2 and 3 respectively. ........................................................................
Fig. 6.1 Generating SLI is possible through either an interferometer or a grid. The interferometric
setup shown here is much simplified for grid projection where the components in the dotted box
are bypassed and the light goes directly from RDG through the f=200mm and f=100mm lenses
before passing through the grid and onwards. BE: beam expander, RDG: reflective diffraction
grating, NPB: non-polarizing beam splitter, M: mirror, ExTL: excitation tube lens, EmTL:
emission tube lens, DM: dichroic mirror, Obj: objective, BFP: back focal plane, FFP: front focal
105
plane, GP: grid projection, FP: fringe projection ......................................................
Fig. 6.2 (a) The profile of the illumination fringe pattern of FP predicted by Eq. (1). (b) The
profile of the illumination fringe pattern of GP predicted by Eq. (2). X = 780nm, Of=1 3.2 degree,
Og= 27.2 degree, a=2/n (c) The experimentally measured contrast of the coherent illumination
fringe pattern and the incoherent detection fringe pattern of both FP and GP. Standard deviations
109
are small compared to averages and not well visible in the plot. ...................................
12
Fig. 6.3 (a) Contrast decay of fringe pattern of the spatial period of Tg = 0.43, 0.85, 1.71, 3.42
pm as a function of the defocus (distance from focal plane). (b) Normalized intensity of SLI and
TPLSM as a function of the normalized defocus unit ................................................
110
Fig. 6.4 (a) Axial resolution measured with the thin Rhodamine solution. Fringe period of SLI is
1.71 pm (red) TFM HiLo at 0% intralipid (blue) TFM HiLo at 2% intralipid (green) TFM at 0%
intralipid (black) TFM at 2% intralipid (orange) TPLSM in Fig. 6.3(b) convoluted with 2 ptm
thick Rhodamine solution. (b) Averages and standard deviations of FWHM of 10 measurements
for 0% and 2% scattering condition of both TFM and HiLo TFM...................................112
Fig. 6.5 xz sections of the fine glomeruli and convoluted tubules structure in a mouse kidney
sample acquired with TFM without SLI, HiLo processed TFM with fringe period of 3.42pm,
1.71im, 0.85pm, respectively. The thickness of the imaged portion is 14ptm. Intensity increases
from purple to red. The cross sectional intensity plot along the line indicated by the yellow arrow
is also shown on the right side. Further details on the sample can be found in
http://products.invitrogen.com/ivgn/product/F24630..................................................113
Fig. 6.6 Unprocessed images in scattering conditions (a) 0% v/v, (c) 3% v/v, and (e) 5%v/v
Lipofundin-20. Processed images are (b), (d), and (f) for the same scattering conditions,
respectively. Each image represents a 90jm x 70pm. ................................................
114
Fig. 7.1 Temporal focusing wide-field (TFWF) FLIM/PLIM design. (a) Optical sub-system:
temporal focusing widefield multiphoton microscopy. (b) Electronic sub-system: frequency
domain lifetime measurement via heterodyne detection. .............................................
123
Fig. 7.2 Demonstration of accurate measurement of fluorescence lifetime of Rhodamine B
solutions in different solvents by TFWF FLIM. Fluorescence in water was used as a reference.
(a) Tabulated results of estimated lifetime values of r extracted from either phase (Ph) or
modulation (Mod) measurements. Literature values are also included as a reference. (b)
Lifetime resolved data for each pixel from the fluorescein (FL) and rhodamine solution images
shown in polar plot form at. ...............................................................................
124
Fig. 7.3 (a) An intensity scaled mean lifetime image of fixed fibroblasts with vacuoles loaded
with endocytosed conjugated polymer nanoparticles of high two-photon absorption cross section.
Color scale represents pixel lifetime values corresponding to the color bar with units of seconds.
Image brightness represents pixel intensity values. Black regions are ignored in analysis
corresponding to locations with intensity below 500 photons that are mostly outside the
boundary of this cell. (b) Representative polar plot of pixel lifetime values for 10 ms data
acquisition time. (c) Tabulated mean lifetime values and their standard deviations are estimated
from the modulation or the phase data for four different image acquisition rates..................126
Fig. 7.4 Fluorescence lifetime imaging of an ex vivo histological sample by TFWF FLIM. (a)
Intensity-scaled fluorescence lifetime-resolved image of a regenerated ex vivo rat sciatic nerve
after injury stained with FluroMyelin Green. The image field of view is approximately 20x20
pim 2 (a) and 100x100 ptm 2 (c). Scale bar has units of seconds. (b) Polar plot representation of the
pixel lifetime estimated based on the phase data. (c) A larger view of the same sample acquired
using a point scanning two-photon microscope equipped with TCSPC lifetime resolved imaging
system. The scale bar has unit of nanoseconds. The lifetime measurements for both systems are
in excellent agreem ent. ....................................................................................
127
13
Fig. 7.5 Demonstration of fast 3D-resolved lifetime imaging by TFWF FLIM system. (a)
Intensity scaled lifetime images of 15 ptm yellow-green beads and 4 ptm red beads imaged at
different axial plane locations z. Pixel lifetimes were estimated from modulation data. The scale
bar represents 5 im in length. (b) Representative polar plot of pixel lifetime information at z= 15
12 8
pm . ...........................................................................................................
Fig. 7.6 Fluorescence lifetime imaging of human fibroblasts seeded inside a collagen scaffold
double-stained with Calcein AM and Syto 13 by TFWF FLIM. (a) Intensity scaled lifetime
images. (b) Polar plot of fibroblasts labeled only with Calcein AM. (c) Polar plot of fibroblasts
labeled with both Calcein AM and Syto 13. (d) Polar plot of fibroblasts labeled only with Syto
12 9
13 . ............................................................................................................
Fig. 7.7 Fast measurement of partial oxygen pressure by TFWF phosphorescence lifetime
imaging. (a) Polar plot of phosphorescence lifetime of 1 mM Tris(2,2'bipyridyl)dichlororuthenium(II) hexahydrate solutions equilibrated with 0, 4, 8, and 21% PO2 gas
mixtures. Inverse phosphorescence lifetimes ((b), from modulation data; (c), from phase data)
are plotted against 02 concentration demonstrating Stern-Volmer dependence with R2 values of
131
0.99 and 0.95 respectively for linear regression. ......................................................
Fig. 7.8 Uncertainty in estimating the lifetime of 1mM TDRT solutions depends on the image
integration time. (a) TDRT phosphorescence lifetime measurement uncertainties (FWHM) for
images acquired with different integration times are plotted against the average number of
photons contained in these images. (b) Representative polar plots of TDRT solution pixel
phosphorescence lifetime measurements at integration times of 1.5, 3, and 6 sec. ............... 132
Fig. 7.9 Fast 3D-resolved TFWF PLIM in sample consisting of human fibroblasts stained with
Rhodamine DHPE, seeded inside a collagen matrix and treated, in PBS buffer containing 1 mM
TDRT ruthenium-based oxygen sensor. PLIM Images were acquired at 300 kHz modulation
frequency. In sequence from left to right: intensity image, phosphorescence lifetime-resolved
image from phase data, phosphorescence lifetime-resolved image from modulation data. On the
far right is the polar plot of the pixel phosphorescence lifetime-resolved measurements. The pixel
lifetime data in the polar plot distributes between the ruthenium phosphorescence lifetime that
lies close to the universal circle and the rhodamine fluorescence lifetime at the lower right hand
corner corresponding to an effective zero lifetime for 300 kHz modulation frequency. PLIM
Movie sequences corresponding to phase and modulation lifetime measurements throughout a
3D matrix are included (Supplementary material Mov 1, Mov 2). ................................. 133
Fig. 7.10 Representative single plane intensity scaled lifetime-resolved image measured at a
133
m odulation frequency of 300 kH z. ......................................................................
Fig. 7.11 Fast sequential 3D-resolved TFWF FLIM and PLIM imaging of human dermal
fibroblasts seeded in a collagen scaffold. Representative images of the fluorescence component
emitted by Rhodamine DHPE (top), and the phosphorescence component, emitted by TDRT
ruthenium-based oxygen sensor (bottom), of the signal emitted from a single 3D resolved plane.
3D resolved image stacks of these two components can be seen in accompanying movie
134
sequences (M ov 3, M ov 4) ................................................................................
14
List of Tables
Table 6.1. Reduced scattering coefficients..............................................................111
15
This page is intentionallyleft blank
16
Chapter
1
Introduction to 3D optical microscopy
Optical microscopy is an imaging technique that allows morphological mapping of intracellular
structures with submicron resolution. More importantly, optical microscopy can provides images
with biochemical contrast based on different spectroscopic modalities such as fluorescence
spectrum and lifetime, Raman spectrum, optical polarization and phase. Of these, fluorescence
microscopy by its nature has several advantages for biomedical applications since fluorescence
imaging is specific to particular chemical and biological features. Biological structures can often
be identified by their endogenous emission characteristics, but, more importantly, the abundance
of contrast agents developed in the past several decades allow specific labeling of biological
specimens. It is also very important that the Stokes shift between the excitation and emission
wavelengths allows highly sensitive detection. The detection of single protein tagged by a single
fluorophore is routine today. Importantly, the location of single fluorophore can be determined
much below the diffraction limit.
Wide-field fluorescence microscopy is very useful for imaging thin sample. However, for
a thicker sample where the scattering of the light blurs the image, wide-field microscopy can
only provide limited depth information, meaning that it cannot be used for studying 3D
structures of many biological systems. There are now a variety of methods to extract depth
information from biological specimens. In this chapter, we describe some of the 3D fluorescence
imaging techniques that are relevant for the biomedical applications.
1.1 Confocal laser scanning microscopy (CLSM)
In 1961, Minsky filed the first patent on confocal microscopy where conjugated pinholes
were used to reject out of focal plane background while point scanning was used for mapping
volumetric information [1]. Conceptually, this form of 3D microscopy works by placing a spatial
filter (pinhole) in the confocal plane of the objective so that only light emanating from the focal
volume can efficiently pass through the pinhole. Out-of-focus illuminated objects form a
17
defocused spot at the pinhole and consequently only contribute weakly to the signal. This optical
sectioning effect allows imaging in three-dimensions based on the confocal principle. Confocal
microscopes [2-4] may be based on reflected light (similar to white light modalities of wide-field
microscopy) and fluorescence signals. The image contrast in reflected light confocal microscopes
is generated by index of refraction mismatch in the specimen [5] and the image contrast in
fluorescence light confocal microscopy is based on the non-uniform distribution of endogenous
fluorophores.
Confocal microscopy generates images at a single point at a time. Therefore, in order to
map out the 3D fluorophore distribution, raster scanning of either the specimen or the light beam
must be performed. Typically, confocal microscopes acquire a 2D image by scanning the beam,
where the light is deflected by mirrors to focus at different lateral locations. Typical confocal
microscopes use a galvanometer system consisting of two synchronized scanners that move in
tandem to produce an x-y raster scanning pattern. The axial position of the excitation light is
typically accomplished by either piezo-driven or mechanically driven objective positioners. The
emission light is focused down and filtered through a pinhole to accomplish the confocal
detection.
Specimen
Emission pinhole
Computer
aperture
Objective Diclroic
Detector
z-actuator
X-Y Scanner
Microscope
Field 4etr
Fig. 1.1 Typical configuration of the confocal microscope. A pinhole that is positioned at the conjugate
plane of the focal plane of the objective blocks out-of-focus light for 3D resolved imaging.
18
1.2 Two photon laser scanning microscopy (TPLSM)
Denk, Webb and co-workers in 1990 introduced two-photon fluorescence microscopy [6].
Fluorophores can be excited by the simultaneous absorption of two photons, each having half the
energy needed for the excitation transition.
Since the two-photon excitation probability is
significantly less than the one-photon probability, two-photon excitation occurs only at
appreciable rates in regions of high temporal and spatial photon concentration. The high spatial
concentration of photons can be achieved by focusing the laser beam with a high numerical
aperture objective to a diffraction-limited spot. The high temporal concentration of photons is
made possible by the availability of high peak power mode-locked lasers. In general, twophoton excitation allows 3-D biological structures to be imaged with resolution comparable to
confocal microscopes but with a number of significant advantages: (1) Conventional confocal
techniques obtain 3-D resolution by using a detection pinhole to reject out of focal plane
fluorescence. In contrast, two-photon excitation achieves a similar effect by limiting the
excitation region to a sub-micron volume at the focal point. This capability of limiting the
region of excitation instead of the region of detection is critical. Photo-damage of biological
specimens is restricted to the focal point. Since out-of-plane chromophores are not excited, they
are not subject to photobleaching. (2) Two-photon excitation wavelengths are typically redshifted to about twice the one-photon excitation wavelengths. The significantly lower absorption
and scattering coefficients at these longer wavelengths ensure deeper tissue penetration. (3) The
wide separation between the excitation and emission spectra ensures that the excitation light and
any Raman scattering can be rejected without filtering out any of the fluorescence photons. This
sensitivity enhancement improves the detection signal to background ratio.
Fig. 1.2 Fluorescence signal generated near the focal region of the objective (a) one photon excitation (b)
two photon excitation. The figure is adapted from the paper by Zipel et al [7].
19
Skin
*bjective
PiezoZ-stage-
Dichroi4
mirror
Microscopepan
.X-Y
Scanner
MField hperture
plane
Computer
PMT/
Discriminator
Fig. 1.3 Typical configuration of the two-photon microscope. The detector is positioned in the non-descan
position which collects all the scattered emission photons from the focal point of the excitation beam.
1.3 Temporally focused widefield two photon microscopy (TFM)
In TPLSM, the high concentration of photons is generated by spatially focusing the
excitation pulsed laser to a small spot with high NA objective while the pulse width of the beam
is maintained constant during the propagation through the focal region of the objective. In TFM,
the high concentration of photons is generated by temporally focusing the excitation pulsed laser
while the spatial size of the beam is maintained constant. The advantage of TFM over TPLSM is
that it can parallelize the image acquisition process in 2D without beam scanning and achieve
higher imaging rate.
Total TPE signal per plane
FsI
=
TPE(z)dA oc
cra
Spatial Focusing
1
Temporal Focusing
Fig. 1.4 In spatial focusing, the pulse width (t) is fixed while the illumination area(A) is minimal at focus.
In temporal focusing, the illumination area (A) is fixed and the temporal pulse width(t) is minimal at
focus. The figure is adapted from the PhD thesis by Daekeun Kim [8].
20
Figure 1.5 shows one realization of TFM. The pulse width is controlled by first spectrally
dispersing the excitation pulse with grating and recombines them only at the focal plane of the
objective so that the original short input pulse width is only recovered at the focal plane of the
objective and the pulse width is broadened above and below the focal plane. Thus, the depth
resolved widefield images can be obtained.
IUltrafast Optical Pulse
from Ti:Sapphire Laser
Y Z
Grating
A
Dichroic
Y
Pupil plane
sarple
V'
Focal plane
Fig. 1.5 Widefield TFM
1.4 Light Sheet Microscopy (LSM)
High-resolution imaging methods such as confocal and multiphoton microscopy provide
optical sectioned imaging of specimens with relatively small dimensions (<1 mm). However, 3D
imaging of specimens with dimensions of several millimetres is not practical with highly time
consuming point scanning based methods such as CLSM and TPLSM. LSM typically have
slightly lower resolution than CLSM and TPLSM but provide deeper imaging in relative
transparent specimens, such as embryos. However, the most important advantage of LSM is its
high throughput.
In LSM, the specimen is illuminated by a thin sheet of light, perpendicular to the imaging
axis, defining the optical sectioning of this technique. In an ideal case, fluorescence signal should
be only generated at the plane of excitation at the specimen and the emission fluorescence can be
collected at either side of the light sheet. Imaging is performed using the conventional wide-field
microscopy system that determines its lateral resolution. As the thickness of the sheet is tailored
to be thin, it is possible to achieve excellent optical sectioning of the specimen and suppression
21
of out-of-focus light. Compared to confocal microscopy, LSM provides significantly less
photobleaching because the excitation occurs only at one plane, at a time, of the specimen
perpendicular to the imaging axis.
Figure 1.6 shows the experimental schematic of LSM. A laser beam is passed through a
cylindrical lens to generate the laser "sheet" at the focal plane of collection objective, which
penetrates into the sample. Thus, only the fluorophores in the vicinity of the focal plane of the
collection objective are excited. Fluorescence emission is passed through an emission filter for
fluorescence detection and is collected by the camera. LSM was demonstrated for various
biological applications such as optically sectioned imaging deep inside live embryos [9], 3D
visualization of neuronal network in whole mouse brain [10], single molecule imaging in cells
[11], deep and fast imaging living specimens using two-photon LSM [12].
Waterfilled
chamber_
i
Camera
Objective
Tube lens
b
h ght sheet
Detection
Glass plate
Filter
OtclfbrCollector
Cylindrical
lens
%ObJective
Sap-
From Laser array
illumination-
Fig. 1.6 Optical system arrangement of LSM. A cylindrical lens used to create a sheet of light that
penetrates the sample. Inset shows the close look of the illumination and selection mechanism in LSM [9]
1.5 Organization of the thesis
In this thesis, we have developed high throughput optical imaging instruments based on
the widefield depth resolved fluorescence imaging techniques such as the structured light
illumination microscopy and the temporally focused widefield two photon microscopy.
The thesis is organized as follows:
22
Chapter 2 describes the high 3D image cytometer based on widefield structured light microscopy
and high-speed remote depth scanning. Chapter 3 describes the depth resolved hyperspectral
imaging cytometer based on structured light illumination and Fourier transform interferometry.
Chapter 4 describes the two photon point scanning endomicroscopy for clinical diagnosis.
Chapter 5 describes the widefield two photon endomicroscope based on the temporally focused
two photon microscopy for higher throughput imaging. Chapter 6 describes a method for
improving axial resolution and contrast with structured light illumination in temporally focused
widefield two-photon microscopy. Chapter 7 describes the 3D-resolved fluorescence and
phosphorescence lifetime imaging using temporally focused wide-field two photon excitation.
Chapter 8 summarizes the thesis and describes the contributions of this thesis work.
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
M. Minsky, "Microscopy Apparatus," US Patent 3013467 (1961).
T. Wilson and C. J. R. Sheppard, Theory and PracticeofScanning Optical Microscopy
(Academic Press, New York, 1984).
J. B. Pawley, ed., Handbookof Confocal Microscopy (Plenum, New York, 1995).
B. R. Masters, Selectedpapers on confocal microscopy (SPIE, Bellingham, 1996).
P. Corcuff, C. Bertrand, and J. L. Leveque, "Morphometry of human epidermis in vivo by
real-time confocal microscopy," Arch. Dermatol. Res. 285, 475-481 (1993).
W. Denk, J. H. Strickler, and W. W. Webb, "Two-photon laser scanning fluorescence
microscopy," Science 248, 73-76 (1990).
W. R. Zipfel, R. M. Williams, and W. W. Webb, "Nonlinear magic: Multiphoton
microscopy in the biosciences," Nat. Biotechnol. 21, 1369-1377 (2003).
D. Kim, "Ultrafast optical pulse manipulation in three dimensional-resolved microscope
imaging and microfabrication," (Massachusetts Institute of Technology, Cambridge, MA,
2009).
J. Huisken, J. Swoger, F. Del Bene, J. Wittbrodt, and E. H. K. Stelzer, "Optical sectioning
deep inside live embryos by selective plane illumination microscopy," Science 305, 10071009 (2004).
H. U. Dodt, U. Leischner, A. Schierloh, N. Jahrling, C. P. Mauch, K. Deininger, J. M.
Deussing, M. Eder, W. Zieglgansberger, and K. Becker, "Ultramicroscopy: threedimensional visualization of neuronal networks in the whole mouse brain," Nat. Methods 4,
331-336 (2007).
M. Tokunaga, N. Imamoto, and K. Sakata-Sogawa, "Highly inclined thin illumination
enables clear single-molecule imaging in cells," Nat. Methods 5, 159-161 (2008).
T. V. Truong, W. Supatto, D. S. Koos, J. M. Choi, and S. E. Fraser, "Deep and fast live
imaging with two-photon scanned light-sheet microscopy," Nat. Methods 8, 757-760 (2011).
23
This page is intentionally left blank
24
Chapter 2
Three-dimensional image cytometer based on widefield
structured light microscopy and high-speed remote depth
scanning
2.1 Introduction
Flow cytometry is a powerful tool for gathering statistical data on large population of
cells to correlate multiple parameters such as size, internal complexity, phenotype within a cell
population [1]. In addition, flow cytometry can characterize the biochemical states of individual
cells, such as whether particular proteins are expressed, based on fluorometric assays. The key
strength of flow cytometry is speed that can routinely process 10,000 cell/sec; however,
processing speed up to 100,000 cell/sec has been demonstrated previously [2]. A drawback of
flow cytometry is that it provides little morphological information of these cells which is
important to complement biochemical information to fully determine the physiological condition
of cells. Therefore, there is a clear need to develop new cytometry methods that can obtain
cellular morphological information while maintaining comparable throughput as conventional
flow cytometry.
Image cytometry was developed to address this need by providing high content assays at
high throughput although the current throughput is not quite close to that of the flow cytometry.
Imaging provides morphological
context for understanding
physiological
change. As
demonstrated by Bakal et al. morphological analysis of cell shape allows association of protein
expression with mechanistic steps of cell migration [3]. In addition, Perlman et al have
demonstrated that morphological responses of cells allow the classification of drugs and to
identify mechanisms of new drugs [4].
A number of 2D image cytometer has been introduced with different instrument
complexity and information content. While they share common features such as automated cell
positioning and focusing to achieve high throughput imaging rate, these system are slightly
25
different in terms of target specimen preparation,
image acquisition method, and autofocus
mechanism [5]. Most of the commercially available system deals with adherent cell types that
are plated either on the slide glass or multi-well plate. On the contrary, imaging flow cytometry
captures two dimensional images of floating cells as they are transported through the fluidic
channel similar to the flow cytometry. The state of the art system can achieve 1000 cell/sec
imaging rate in bright field, dark field and fluorescence simultaneously by imaging multiple cells
with extended depth of field in widefield imaging mode [6]. Imaging flow cytometer has been
extended to in vivo detection of rare circulating cells [7].
However, 3D imaging capability is required for better quantify the morphological
features in 2D cell culture environment. For example, accurately quantifying the foci of H2AX
protein in nuclei is an important criteria for accessing the DNA double stand break that is related
to the initiation of the cancer [8]. These H2AX foci are distributed in 3D within nuclei and
sometimes hard to distinguish the overlapped foci with 2D widefield imaging methods. In
addition, optically sectioned 3D imaging is a more suitable imaging method for studying the
differentiation process induced by the interaction of the stem cells with the feeder cells in a coculture environment [9]. Other than those applications, there are numerous cases where 3D
imaging can provide more accurate measures on the morphological features such as cell size,
shape and volume for the physiological study of cells [10].
Not only in 2D cell culture environment, 3D resolved images can better represent cells in
3D tissue environment. Since cells in tissue interact with and are regulated by the extra cellular
matrix for their physiological and pathological behavior, cells cultured in 3D matrix can better
mimic the behavior of cells in tissue than cells cultured in two dimensional well plate or slide
glass. For example, cell-matrix interaction has proven to be an important factor for the outcome
of the nerve regeneration [11]. Therefore, image cytometer with 3D imaging capability can better
quantify cellular behavior in their natural condition. 3D imaging or in other words optically
sectioned imaging is typically achieved with the laser scanning microscopy such as confocal or
two photon microscopy. Either spinning disk or line scanning confocal microscopy have been
utilized to increase frame rate in most commercially available image based high throughput high
content imaging system [12]. Multiphoton microscopy is better suited for deep imaging in a
scattering opaque tissue specimen. High throughput multiphoton imaging has been achieved by
scanning multiple foci in parallel or by using high speed polygonal mirror scanner, which was
26
applied in the high speed 3D deep tissue cytometer [13]. Recent advances in optical clearing
technology have opened the door for studying a whole organ in single cellular level [14-16].
In this chapter, we introduce a new type of high throughput cell-based 3D cytometer
based on the optically sectioned widefield fluorescence imaging method. For a 80% confluent
cells, this system can image at about 800 cell/sec in 3D at submicron resolution, which
corresponds to about 20 min imaging time for 1 million cells. Throughput of the developed
system is orders of magnitude faster than the state of the art commercially available high
throughput image cytometer in terms of the resolvable pixel/sec. The large field of view, high
numerical aperture objective and the large pixilation, high frame rate sCMOS camera can capture
a large number of cells simultaneously in submicron resolution. Widefield optically sectioned
image is acquired by the structured light illumination. In addition, high speed depth scanning has
been made possible by the remote depth scanning. Motorized stage scan the sample
automatically during the image acquisition process, which enables a large area sample imaging.
This system could be useful for the 3D image based cell assay for the drug screening and the
image informatics study to infer the role of signaling pathway components of the cellular
system. .
2.2 Design of the overall system
Fig.2.1 shows the schematic layout of the depth-resolved image cytometer. Three diodepumped solid states lasers (CNI laser) with emission wavelength of 473nm, 561nm and 660nm
respectively were used as light sources that can excite broad range of fluorophores. The three
beams are combined into one collinear beam using the laser beam multiplexers (LMO1-503-25,
LM01-613-25, Semrock). Photobleaching effect is minimized by controlling passage of the
excitation light with the optical shutter (Ch-61, EOPC) which operates in synchrony with the
exposure status signal of the camera so that the sample is exposed to the excitation light only
when the camera is recording the emission light. The diffractive optical element (DS-033-Q-Y-A,
HOLO/OR) split the beam into +1 and -1 order and minimize the power loss by Oth order and
other higher diffractive order beam. Since the beams split by DOE reach the focal plane of the
objective via common path, they form a stable fringe pattern for SLI. The optical shutter (Ch-61,
EOPC) placed in one of the diffracted beam path blocks the beam for a uniform illumination (Ul)
27
and unblock the beam for a structured illumination (SI) which is formed by the interference of
the two beams. The optical shutter is triggered by the external TTL signal and enables high
speed switching between UI and SI. A quad-band dichroic mirror (FF410/504/582/669 -DiOl25x36, Semrock) was used to separate the excitation beam from the emitted fluorescence signal.
A quad-band emission filter (FFOl-440/521/607/700-25, Semrock) right in front of the detector
was used to block any stray excitation light reflected from the intermediate optical elements. The
z-scanning of the sample is performed remotely by first forming the perfect 3D image of the
sample in the focal space of the remote focusing objective and then by scanning the mirror in
axial direction with piezo actuator. The cytometer can be functional both as high throughput
imaging mode and hyperspectral imaging mode. For high throughput imaging mode, the
intermediate image formed at the focal plane of L8 is relayed to the imaging detector through the
two relay mirrors. Different color channel can be selected by a filter wheel with three band pass
filters (FFOl-520/35, FFOl-609/54, FFOl-692/40, Semrock). For hyperspectral imaging mode,
the Sagnac interferometer is utilized to record the interferogram which is subsequently processed
to build the spectral cube of every pixels in two dimensional field of view [17]. sCMOS camera
(PCO.EDGE 5.5, PCO) records a field of view with size of 420 pim x 350 p.im at maximum 100
frame per second at 2560 x 2160 pixelation. When the bit depth of a pixel is 16 bit, the total data
rate becomes approximately 1 GB/sec. Since the writing speed of the currently available hard
drive is limited to 400 MB/sec even in the case of the state of the art solid state drive, multiple
hard drives are configured as a RAID 0 to record the data in parallel which effectively increases
the writing speed and storage capacity. After z stack images for a field of view are acquired for
SI first and then for UI, the motorized stage (SCAN IM 120x100-2mm, Marzhauser ) translates
the sample to the next field of view and the same sequence of imaging is repeated. The data
acquisition procedures are fully automated and controlled by the custom-made control software
written in C# programming language.
28
PA
M
ROWJ
1 =00
=1000
EmF
F=
1660nmJ561nm J473nmJ
MD
OSl
M
DM
.
X
DOE
L=164.S
DM
LI= 75
L2=150
L3=75
L42=300
1"=150
DM
oBu
FFP
Fig. 2.1 Schematic diagram of the high throughput 3D imaging cytometry. The current diagram is
for the high throughput imaging mode and the filter wheel is the detection path is used for the
spectral imaging. When the blue box rotate 180 degree and the beam splitter is engaged it
function as a hyperspectral imaging mode. M: mirror, DM: dichroic mirror, DOE: diffractive
optical element, OS1: optical shutter for minimizing the photobleaching effect, OS 2 : optical
shutter for generating UI and SI, PBS: polarizing beam cube splitter, FW: filter wheel with three
emission filter, QWP: quarter-wave plate, FW: band pass filter wheel, EmF: emission filter, PA:
piezo actuator, FFP: front focal plane, OBJ: imaging objective lens (Zeiss 20x Water NAL.0),
ROBJ: remote focusing objective lens (Nikon 20x Air NA0.75)
2.3 Structured light illumination (SLI) for depth-resolved widefield imaging
A class of depth-resolved imaging techniques based on SLI have been proposed to select
a particular imaging plane and to reject out-of-focus background for standard wide-field singlephoton microscopy [18, 19]. Of these methods, one effective approach we adapted is termed
'HiLo microscopy' which generates an optically sectioned image by post-processing the
uniformly illuminated image (UI) and the structured light illuminated image (SI) [19]. This
algorithm is based on an assumption that 2D image can be divided into low frequency and high
frequency contents. Since the high frequency contents are in-focus by its nature, the goal of
using SLI is to encode the in-focus low frequency contents especially laterally zero frequency
component with high frequency SLI. More specifically, the in-focused high frequency contents
are extracted by high-pass filtering UI with a Gaussian shaped high-pass filter. The in-focus low
frequency contents are extracted by low-pass filtering the absolute of UI subtracted by SI with
the complementary low pass filter to the high pass filter. The cutoff frequency of the Gaussian
29
filter is determined by the sinusoidal spatial frequency of the structured illumination.
Subsequently, the optically sectioned image is obtained by combining the two with an adjusting
factor so that the transition from low to high frequencies occur smoothly [20]. HiLo microscopy
has been widely used in the context of the background rejection for light-sheet microscopy [20,
21], the temporally focused widefield two photon microscopy [22] and the depth-resolved
microrheology [23]. Compared to the phase shifted SLI which requires at least three images
taken at every n/3 phase of the fringe illumination that is used by some of the commercial high
content image cytometer [24], HiLo based SLI requires only two images for a depth resolved
image. This is a great advantage for high throughput imaging since the switching between SI and
UI can be performed in high speed by shuttering one of the two beams with an optical shutter. In
addition, HiLo method is insensitive to the motion artifact of a sample since the precise phase
control is not required. Fig.2.2 shows one example how the HiLo method can be used for
rejecting the background signal. The fringe pattern projected on the in-focused part of the sample
is clearly visible as shown in Fig.2.2(b) and the HiLo processing remove the common
background from out-of-focal regions and produce much clear image as shown in Fig.2.2(c).
(a)
(b)
(c)
Fig.2.2 Background rejection with HiLo algorithm (a) Uniformly illuminated image (b) Structurally
illuminated image (c) HiLo processed image
2.4 Axial resolution of the SLI measured with a thin Rhodamine solution.
Theoretical axial resolution of SLI can be estimated using the defocused 2D optical
transfer function (OTF) derived by Stokseth [25].
C(u,rn)= f(r)
{F
2J, (um [1 - m/2])]
r[-/](2.1)
um [I - m/2]
where f(m) = 1 - 0.69m + 0.076m2 + 0.043m 3 , m is the normalized fringe frequency and is
related to the real fringe period Tg via m = /(Tg NA), NA = n sin(a) and u is the normalized
30
defocus and is related to the actual defocus z via u = 4kznsin2 (a/2), k = 2n/). In incoherent
detection, OTF represents the contrast of the fringe detected at the image plane and is equivalent
to the signal generated when a thin sheet of fluorescence is scanned through the focus. Ideally,
the best optical sectioning is achieved when the normalized fringe frequency comes close to 1
[26]. However, as the fringe frequency is increased the modulation depth is reduced which in
turn reduces the signal to noise ratio.
Fig.2.3 shows the radially integrated fluorescence intensity at each depth in the vicinity
of the focal plane of the objective for a thin sheet of florescent solution for both widefield (WF)
and SLI imaging condition. In case of WF, the integrated fluorescence from each z plane is
constant, which implies that there is no depth discrimination for a laterally zero frequency
component. This effect is caused by a "missing cone" of OTF. Practically, this corresponds to the
uniform background comes from the defocused objects. In confocal microscopy, the pin hole
conjugated to the focal point filters out this background. In SLI, the SI acts as a virtual pinhole
and filter out computationally the low frequency out-of-focus background beyond the depth of
field of the objective. In case of fringe period of 1.3pm, the expected axial resolution in terms of
the intensity full width at half maximum (FWHM) is 1.28 pm and the experimentally measured
value is FWHM is 1.84 ± 0.05pm.
The thin florescent solution is prepared by first dissolving Rhodamine power
(Rhodamine 6G, Sigma-Aldrich) in alcohol and diluted to 300ptM concentration. Then, a drop of
Rhodamine solution is placed on a slide glass and covered with a cover slip and squeezed to
form a meniscus around the edge of the cover slip. The volume of a drop of Rhodamine solution
is determined by multiplying the area of the cover slip with the desired thickness that needs to be
less than the axial resolution.
Total fluorescence from z planes
1.0
+WF
+
HiLo
Theory
0.5+
Z
0.0
-4
-3
-2
-1
0
1
2
3
4
Distance from focal plane [gm]
Fig.2.3 Optical sectioning measured with thin Rhodamine solution. WF: Widefield, HiLo, Theory
31
2.5 High speed remote depth scanning
Typically, the microscope objective is scanned in depth direction with piezo-scanner to
acquire the volume image of 3D object. However, direct objective scanning is relatively slow for
the purpose of high throughput imaging because of the inertia of the objective. Although 10Hz
volume imaging rate has been demonstrated by resonantly driving the objective piezo scanner
[27], this can also potentially resonate the immersion medium and induce motion of sample.
One of the way to improve the scanning speed without sample disturbance is to form perfect 3D
image of object remotely and sequentially scanning the remotely formed 3D object in the axial
direction with a small light-weight mirror driven by a piezo actuator [28]. The perfect 3D
imaging can be achieved by designing the imaging system such that it satisfies both the sine and
the Herschel condition. As a consequence of these constraints, the lateral and the longitudinal
magnifications of the imaging system becomes equal to the ratio of the immersion media
refractive indices of object and image space [29]. As proposed by Botcherby et al
[28], the
perfect 3D imaging system can be constructed by ensuring that the intermediate optical elements
satisfy Eq.(2.2) which can be derived from the perfect imaging condition.
A2
-2
f,
n2MIF
MJ 2
nIM2F
(2.2)
wheref1 andf 2 are the focal length of the imaging and the remote focusing objective respectively.
ni and n2 are the refractive index of the immersion medium in object and image space
respectively. M and M2 are the magnification of the objectives. F1 and F 2 are the nominal focal
length of the objectives. Practically, it is important that the pupil plane of the imaging objective
(W Plan-Apochromat 20x 1.0 N.A. water, Zeiss ) is imaged onto the pupil plane of the remote
focusing objective (CFI Plan Apochromat Lambda 20x 0.75 N.A. air, Nikon) by the 4-f system
consisting of the Zeiss tube lens with the focal length of 164.5mm (58-452, Edmund Optics) and
the achromatic doublet with the focal length of 150mm (AC504-150-A, Thorlabs) so that the
phase distortion induced by the imaging objective is cancelled out by the equal amount of
negative phase distortion induced by the remote focusing objective and consequently the
distortion-free 3D image is formed with isotropic magnification in all direction [30]. The axial
scanning is performed by moving the mirror (NT68-321, Edmund Optics) attached to the piezo
actuator (P-830.10, Physik Instrumente). The travel range of the current piezo actuator is 15pm,
which corresponds to 22.5 pm scanning range in the object space by the combined effects of the
32
magnification from object to image space and the mirror scanning in the image space. Since an
air immersion objective (ROBJ, see Fig. 2.1) can be chosen for remote focusing, sample
disturbance does not occur even for high speed scanning. As ROBJ is designed for imaging
through a 170pm coverslip, a coverslip is placed in between ROBJ and the remote focusing
mirror to prevent additional spherical aberration. To improve the detection efficiency of the
unpolarized fluorescence emission light, a 14 plate (2-APW-L/4-018A, Altechna) and a
broadband polarizing cube beamsplitter (lOFCl6PB.3, Newport) are used as shown in Fig.2.1.
The point spread function (PSF) of the remote focusing is measured by imaging a 0.5 ptm
fluorescent bead immersed in 2% Agarose gel at axially different locations through the focal
region of the imaging objective (OBJ, see Fig. 2.1). After a bead is placed at the focal plane of
the objective, the position of the bead relative to the focal plane of the objective is varied by
moving the objective manually which is mounted on the micrometer driven translation stage
(SMlZ, Thorlabs). The positive direction means the objective is moved away from the bead and
the negative direction means the objective is displaced close to the bead. At each position of the
objective, the remote focusing mirror is scanned axially for a uniformly illuminated excitation
beam so that the full diffraction pattern of the single bead is recorded. Fig.2.4 shows the PSFs
from -100 pm to 100 ptm at 20 pm interval. Asymmetric pattern of PSF at the focal plane
represents some of the residual spherical aberration. It may partly originate from the index
mismatch of the immersion medium of the imaging objective and the bead immersed in the
agarose gel or it may partly originate that the phase distortion induced by the imaging objective
is not perfectly canceled out by the remote focusing objective. In addition, as the bead is located
further away from the focal plane the PSF is more strongly elongated which is a typical pattern
of the spherical aberration. Since the remote focusing theory is based on the linear approximation
that the point source is near the focus of the imaging objective, the measured PSF shows more
amount of spherical aberration as the bead is displaced away from the focal plane of imaging
objective [30].
z[pm]
-100
-80
-80
-40
-20
20
0
40
60
80
Fig.2.4. Remote focusing PSF at different focal positions
33
100
2.6 Application of the high throughput 3D image cytometer
HiLo based SLI can effectively reject the out-of-focus background and enable high speed
3D resolved imaging. Fig.2.5 shows the 3D image stacks of mouse kidney section (F-24630,
Molecular Probe) obtained with the uniform illumination ( Fig.2.5(a) ) and the same section after
the HiLo processing ( Fig.2.5(b) ). Elements of the glomeruli and convoluted tubules labeled
with Alexa Fluor( 488 wheat germ agglutinin is excited with the 473nm laser line and the
emission light is detected with 520nm/35nm band pass filter (FF01-520/35, Semrock). The
filamentous actin prevalent in glomeruli and the brush border labeled with red-fluorescent Alexa
Fluor@ 568 phalloidin is excited with the 561nm laser line and the emission light is detected
with the 609nm/54nm band pass filter (FF01-609/54, Semrock). The out-of-focus background
that blurs images is removed in the HiLo processed images and consequently the contrast is
improved.
(b)
(a)
Fig.2.5 3D image stack of mouse kidney obtained with (a) Uniform illumination (b) after HiLo processing.
Green color represents elements of the glomeruli and convoluted tubules labeled with Alexa Fluor( 488
wheat germ agglutinin. Red color represents the filamentous actin prevalent in glomeruli and the brush
border labeled with red-fluorescent Alexa Fluor® 568 phalloidin.
The developed system enables image informatics based on high content high throughput
imaging. Morphological features such as nuclear size, perimeter of the cytoplasm, 3D
distribution of the protein can be extracted and statistically processed to provide a morphological
context of the physiological changes. As shown in Fig.2.6(a), the 3D map of the mitochondria
can be measured to study the metabolic states of the cells. Another example where high
throughput 3D image cytometer could be applicable is the image informatics study to infer the
34
role of signaling pathway components of the cellular system. For example, Fig.2.6(b) shows the
fibroblast activated by TGFs which is important factor that determines the outcome of the tissue
regeneration. In short term, TGFP activation causes SMAD protein to enter the nucleus and in
longer term, SMAD protein exit the nucleus and it causes the morphological changes on the cells
such that cell shape is elongated and more actin fiber are generated.
(b)
(a)
Fig.2.6. (a) 3D image of the muntjac skin fibroblast cells (F36925, Invitrogen). The green color represents
filamentous actin labeled with green fluorescent Alexa Fluor 488 phalloidin. The red color represents
mitochondria labeled with an anti-OxPhos Complex V inhibitor protein mouse monoclonal antibody in
conjunction with orange fluorescent Alexa Fluor® 555 goat anti-mouse IgG (b) TGFP activated
fibroblast cells. Green color represents actin filament labeled with rhodamine phalloidin and blue color
represents SMAD protein labeled with Alexa Fluor 488
The developed 3D image cytometer can also acquire a large sample size at submicron
resolution where the size limit is only constrained by the scanning range of the motorized stage.
Fig.2.7 shows a section of the whole mouse kidney that was imaged with 473nm laser line and
520nm/35nm band pass filter. Total 16x16 FOVs are stitched together in the post processing.
This clearly demonstrates that the developed image cytometer can achieve a high resolution and
a large area sample imaging at the same time.
35
Fig.2.7 Multiscale view of mouse kidney. Ix image is stitched with 16x16 FOVs. Area marked with white
square is zoomed in successively.
2.7 Rare Cell Detection in 2D cell culture
The performance of the 3D cell cytometer was characterized by quantifying rare cell
subpopulations in 2D specimens in vitro [13]. The ability to detect a rare cell can be an important
criteria for the cancer diagnosis. For example, there are about 1-10 circulating tumor cell (CTC)
out of 1 million white blood cells [31]. As the cancer progresses, the number of CTC in the blood
stream increases. The ability to detect CTC in early stage of cancer requires a high throughput
system that can screen out CTC in a short time scale.
2.7.1 2D cell culture preparation
A549 human alveolar adenocarcinoma cells transfected with histone H2B-mCherry
cDNA for nuclear staining were maintained in two T75 flasks (Nunclon A Surface) in Dulbecco's
modified Eagle medium (Gibco/Invitrogen 12100) supplemented with 10% fetal bovine serum
(Gibco/Invitrogen) and 1%
penicillin-streptomycin. Puromycin 0.01% was added for the
selection of mCherry-expressing cells with the medium change every 36 to 48 h. To prepare rare
cells with cytoplasm staining prior to the imaging, one of the flasks of the grown cells were
36
labeled with 5-chloromethylfluorescein diacetate (CMFDA Cell Tracker
TM
Green, Invitrogen)
and incubated for 20 minutes, and then replaced with serum free culture medium for another 20
minutes. For passage, cells at 70-80% confluency were detached from both flasks by treating
with 0.05% trypsin-ethylenedinitriletetraacetic acid (Trypsin-EDTA) at 37 'C in an incubator
for 3 minutes. Medium was then added to inhibit the enzymatic reaction of Trypsin-EDTA. Subculture seeding density was kept at 2-3
x
104
cells/ml.
In one group of cells, nuclei were expressing with mCherry with emission peak at 560nm.
In the other group of cells, cytoplasm were labeled with CMFDA with emission peak at 520nm
in addition to the nuclei staining. The latter group of cells with both nuclei and cytoplasm
labeling played the role of the rare cells. The two group of cells, detached using the
aforementioned procedure for passage, were resuspended to two cell suspensions of 104 cells/pd.
Imaging samples were prepared using the two cell suspensions with a range of mixing ratios: 1:1,
1:10, 1:102, 1:103, 104 and 1:105 . The mixed populations of cells were plated in a 8 well chamber
and incubated for about 6 hours, with each well chamber contains about 5 x 104 cells. It should
be noted that at least 20 rare cells were used to generate the specified ratios to achieve reasonable
statistical accuracy. For example, in the sample of 1:105 ratio, the total number of cell was 2
millions and the number of rare cells was about 20. All cell samples were fixed using 4%
Paraformaldehyde (PFA) and maintained in PBS to be used for long-term imaging.
2.7.2 Data acquisition and analysis
Generally, the image acquisition procedure of the 3D cell cytometer is as follows. For a
single field of view with size of 420 pm x 350 pm, SI image stack is first acquired with the SLI
generated by the interference of the two beams with shutter (OS 2 ) open and then for the same
FOV the UI image stack is acquired with the uniform illumination by blocking one of the beam
with shutter (OS 2 ) closed. The motorized stage translate the sample to the next FOV and the
same sequence for acquiring UL and SI image stack is repeated until the whole sample size is
covered. For the multichannel imaging, above procedure is repeated for each detection color
selected by the band pass filter wheel. To maintain the immersion medium in place during the
stage scanning the water chamber is constructed around the imaging objective lens and filled up
with immersion medium. Representative 3D image of rare cell sample at 1:1 mixing ratio is
shown in Fig.2.8.
37
Fig.2.8 Representative 3D image of rare cell sample at 1:1 mixing ratio
For the rare cell detection experiment, the prepared samples of two group of cells mixed
at different ratios are imaged in 2D with uniform illumination. Before starting the image
acquisition, the tilting of the sample holder is adjusted to maintain the whole imaging area in
focus over the required scanning range of the motorized stage. Cytoplasm labeled with Cell
Tracker Green is excited with the 473nm laser line and the emission light is imaged with
520nm/35nm band pass filter (FFO1-520/35, Semrock). Nucleus labeled with m-Cherry is
excited with the 561nm laser line and the emission light is imaged with the 609nm/54nm band
pass filter (FF01-609/54, Semrock). Fig.2.9 shows the representative image of 1:104 sample
where the nucleus is labeled with m-Cherry (represented as blue color) and the cytoplasm
labeling with Cell Tracker Green (represented as green color ). The acquired image is processed
to count the number of cell in each population. The cell counting algorithm was implemented in
MATLAB (MathWorks, Natick, MA) and run by the GPU processor. The detailed image
processing algorithm is described in detail in the next section.
38
Fig.2.9 Representative image of 1: 104 ratio sample where the nucleus is labeled with m-Cherry
(represented as blue color) and the cytoplasm labeling with Cell Tracker Green (represented as green
4
color ). The size of single FOV is 420 gm x 350 pim and the image shown is part of the 1: 10 sample
where 20x15 FOVs are stitched together.
Fig.2.10 shows the rare cell counting result. The expected ratio and the corresponding
measured ratio is plotted in log scale in both x and y axis. The accurate correlation between the
expected ratio and the measured ratio can be confirmed from the slope (0.9299) and the
goodness-of-fit (R2 = 0.9964) of the linear regression line, which demonstrate the capability of
the developed system for detecting a rare cells down to 1:105 ratio.
105-
y =1.8656x 0 .92 99
104-
R -square :0.9964
U
103.
4)
102.
10110010.1.
10-1
-
100
101
102
103
I
I
104
106
Expected ratio
Fig.2. 10 Rare cell subpopulation counting result
39
2.7.3 Image processing procedure
Flatfield correction
The raw images acquired by the cytometer system are generally affected by the field inequality
of the illumination field of view (FOV) caused by the non-uniform intensity distribution of the
excitation laser beam profile. Therefore, B-spline based technique is employed to fit for the
excitation pattern and normalize the intensity distribution over the FOV before image mosaicing.
Since the high throughput imaging often involves sample size on the order of hundreds of
millimeters which is a 106 times of a FOV, a completely data driven approach is employed to
determine the background as well as signal pattern and used a combined additive and
multiplicative model to normalize the FOV. Firstly, the background is approximated by taking
the minimum projection greater than a predefined noise level of the detector over all the images
of the sample (IBG). Secondly, the illumination pattern was approximated by the mean projection
of the same image set after capping the pixels with photon count greater than a predefined
threshold in order to account for the significantly brighter regions (Is). Then the flat-field
correction was estimated for each image I (for each FOV) using Eq.(2.3).
I-
I-IBBG
(2.3)
IBS - IBBG
where the IBBG is the 2D B-spline fitted to IBG and IBs is the 2D B-spline fitted to Is.
Image mosaicing
The flat field corrected images were then stitched together using a maximum cross correlation
based algorithm [32] to generate an image mosaic for each sample. The images were first
arranged in a 2D matrix according to the scan pattern of the motorized stage and the neighboring
images were overlapped according to the extent of overlap introduced during image acquisition.
Due to the limited accuracy of the stage positioning there are slight changes in the overlap in the
different regions of the image mosaic. Hence for each boundary between two FOVs in the image
matrix the size of overlap was changed in a sequence of steps in both X and Y directions in a
predefined range and the cross correlation coefficient was calculated for each step. The XY
position, which results the maximum cross correlation coefficient, was chosen as the best overlap
for the particular boundary between the particular FOVs. This was repeated for all the
40
boundaries between all the images (FOVs) in the mosaic to generate the final stitched image of
the sample.
Segmentation and cell counting algorithm
The cell counting algorithm was designed according to the labeling of the culture. All the cells
contain the nucleus label and hence, it was used to detect and count the total number of cells in
the sample. The rare cells contain the cytoplasm label, which was imaged in a separate channel,
and it was used in conjunction with the nucleus to separate the rare cells from the normal ones.
Nuclei counting
The nuclei of the cells labeled using mCherry were exited using a 561nm laser line and were
imaged in one channel (nucleus channel). Firstly the stitched image of the nucleus channel was
applied with a maximum cutoff intensity, i.e. all the pixels with greater photon count than the
cutoff was assigned the cutoff photon count. Secondly the resulting image was spatially filtered
using a circular averaging filter to smooth the image. Then morphological opening [33] using a
circular structuring element was applied on the image. These steps preprocess the original image
to help identify local intensity peaks, which are used to count the number of nuclei in the sample.
Then the extended maxima transform, which is the recognition of local maxima on the h-maxima
transform [34] was performed to identify the intensity peaks, which represents the nuclei. Then
the number of connected components of the resulting binary image were counted to determine
the number of nuclei, and hence the number of cells, in the sample. The nuclei counting pipeline
is shown in Fig.2.1 1(a).
Rare cell counting
In order to count the number of cytoplasm labeled cells first the stitched image form the
cytoplasm channel was segmented using a predefined threshold and then morphological opening
was performed on the resulting binary image. This threshold was kept in a low enough value to
include all the labeled cells since the counting accuracy should be high enough to detect the rare
cell to normal cell ratio for all the samples with extreme mixing ratios such as 1:104 and 1: 10'.
However this means inclusion of the false positives and hence further refining is necessary to
separate true positives from false positives. Therefore we characterized the true positive, i.e.
41
cytoplasm and nucleus labeled cells using multiple image features of two channels. The image
features used were, cell area (region enclosing from the cell boundary and is a measurement of
cell volume), mean photon count of the cell area in cytoplasm channel, maximum photon count
of the cell area in cytoplasm channel, the degree of overlap of the corresponding nucleus (nuclei
for cell clusters), the area ratio of the corresponding nucleus and the cytoplasm. Measuring the
above features requires proper cytoplasm and nucleus segmentation and therefore we applied a
level-sets algorithm [35] to determine the correct segmentation boundary of the cytoplasm and
the nucleus. The binary images from the nuclei counting and the cytoplasm thresholding were
used as the starting points for the level-sets. For the clustered regions of the sample the above
step determines the cluster boundary.
In order to separate the cells in the cell clusters we applied watershed [36] on the distance
transform of a combined binary images resulting from the extended maxima transform and the
level-sets of the nucleus channel according to the following equations. (Fig.2. 1I(b))
DEX
=
distanceTransform(BWx)
DLS = distanceTransform(BWLS)
D-
DEX
(DEX + DLS)
BWN = BWLS & waterShead(D)
Here DEx and DLS are respectively distance transforms of resulting binary image, BWEX from
nuclei counting step and resulting binary image, BWLS from nucleus segmentation using level
sets. D is the combined distance image and BWN is the final nuclei segmentation after cluster
splitting using watershed on D. Finally the correctly segmented nucleus and cytoplasm channels
were used to measure the previously explained image features for each cytoplasm labeled
segment and this multi-dimensional feature vector was thresholded to filter out the false positives
and count the number of nuclei that includes label in the corresponding cytoplasm channel. The
entire rare cell counting pipeline is shown in Fig.2. 11(b).
42
(A) Nuclei counting
(B) Rare cell counting
Nucleus
Cytoplasm
channel
Level sets
channel
TFh
Intensity
cutoff
Levelsets
Average
Transform
d
13
Distance
filtering
Morphological
D
x/
Watershed
(Dx+DL)
Opening
A
Extended Maxima
Transform
J
I
D
Image Feature
Distance
Transform4
based filtering
DEX
Rare cell
count
Nuclei count
Fig.2.11 Cell counting pipeline (a) Nuclei counting (b) Rare cell counting
2.8 Conclusions
This paper presented a high throughput cell-based 3D cytometer based on the optically
sectioned SLI widefield fluorescence imaging and the high speed remote depth scanning with
fully automated sample scanning by a motorized stage. The developed system enables image
cytometry investigation to study cultured cell morphologies with statistical significance
comparable to the flow cytometer. For a highly confluent cells, this system can image at about
800 cell/sec in 3D at submicron resolution. Compared to the commercially available high
throughput high contents imaging systems which are either based on spinning disk or line
scanning confocal microscopy and the three phase shifted structured light illumination
microscopy, the developed system does not need any scanning and requires only two images of
UI and SI for optically sectioned image. This feature combined with an optimized system design
using high NA, large FOV objective and large pixilation, high frame rate sCMOS camera
enables an order of magnitude faster throughput in terms of the number of resolvable point per
second over the commercially available systems. Whereas the commercially available 3D high
contents imaging systems is mostly focused on the background rejected single layer imaging of
43
cells, the developed system can be applicable for a 3D imaging of cells in 2D cell culture
environment. The statistical accuracy of the high throughput imaging capability of this
instrument is verified by quantitatively measuring the rare cell populations from 1:10 down to
1:101.
Literatures
C. E. Pedreira, E. S. Costa, Q. Lecrevisse, J. J. M. van Dongen, A. Orfao, and E. Consortium,
"Overview of clinical flow cytometry data analysis: recent advances and future challenges,"
Trends Biotechnol. 31, 415-425 (2013).
2. J. F. Leary, D. F. Schmidt, J. G. Gram, S. R. Mclaughlin, C. D. Torre, and S. Burde, "HighSpeed Flow Cytometric Analysis and Sorting of Human Fetal Cells from Maternal Blood for
Molecular Characterization," Ann Ny Acad Sci 731, 138-141 (1994).
3. C. Bakal, J. Aach, G. Church, and N. Perrimon, "Quantitative morphological signatures
define local signaling networks regulating cell morphology," Science 316, 1753-1756 (2007).
4. Z. E. Perlman, M. D. Slack, Y. Feng, T. J. Mitchison, L. F. Wu, and S. J. Altschuler,
"Multidimensional drug profiling by automated microscopy," Science 306, 1194-1198 (2004).
5. D. L. Taylor, J. R. Haskins, and K. Giuliano, High Content Screening:A Powerful Approach
to Systems Cell Biology and DrugDiscovery, Methods in Molecular Biology (2007), Vol.
356.
6. D. A. Basiji, W. E. Ortyn, L. Liang, V. Venkatachalam, and P. Morrissey, "Cellular image
analysis and imaging by flow cytometry," 27, 653-669 (2007).
7. H. Lee, C. Alt, C. M. Pitsillides, M. Puoris'haag, and C. P. Lin, "In vivo imaging flow
cytometer," 14, 7789-7800 (2006).
8. W. M. Bonner, C. E. Redon, J. S. Dickey, A. J. Nakamura, 0. A. Sedelnikova, S. Solier, and
Y. Pommier, "OPINION gamma H2AX and cancer," Nat Rev Cancer 8, 957-967 (2008).
9. A. Soto-Gutierrez, N. Navarro-Alvarez, D. Zhao, J. D. Rivas-Carrillo, J. Lebkowski, N.
Tanaka, I. J. Fox, and N. Kobayashi, "Differentiation of mouse embryonic stem cells to
hepatocyte-like cells by co-culture with human liver nonparenchymal cell lines," Nat Protoc
2, 347-356 (2007).
10. T. R. Kiehl, D. Shen, S. F. Khattak, Z. J. Li, and S. T. Sharfstein, "Observations of Cell Size
Dynamics Under Osmotic Stress," Cytom Part A 79A, 560-569 (2011).
11. D. S. Tzeranis, "Imaging studies of peripheral nerve regeneration induced by porous collagen
biomaterials," Ph.D. (Massachusetts Institute of Technology, Cambridge, MA, 2013).
12. E. Mei, P. A. Fomitchov, R. Graves, and M. Campion, "A line scanning confocal fluorescent
microscope using a CMOS rolling shutter as an adjustable aperture," J Microsc-Oxford 247,
269-276 (2012).
13. K. H. Kim, T. Ragan, M. J. R. Previte, K. Bahlmann, B. A. Harley, D. A. Wiktor-Brown, M.
S. Stitt, C. A. Hendricks, K. H. Almeida, B. P. Engelward, and P. T. C. So, "Threedimensional tissue cytometer based on high-speed multiphoton Microscopy," Cytometry A
71A, 991-1002 (2007).
14. H. Hama, H. Kurokawa, H. Kawano, R. Ando, T. Shimogori, H. Noda, K. Fukami, A.
Sakaue-Sawano, and A. Miyawaki, "Scale: a chemical approach for fluorescence imaging
and reconstruction of transparent mouse brain," Nat. Neurosci. 14, 148 1-U 1166 (2011).
1.
44
15.K. Chung and K. Deisseroth, "CLARITY for mapping the nervous system," Nat. Methods 10,
508-513 (2013).
16. M. T. Ke, S. Fujimoto, and T. Imai, "SeeDB: a simple and morphology-preserving optical
clearing agent for neuronal circuit reconstruction," Nat. Neurosci. 16, 1154-U1246 (2013).
17. Z. Malik, D. Cabib, R. A. Buckwald, A. Talmi, Y. Garini, and S. G. Lipson, "Fourier
transform multipixel spectroscopy for quantitative cytology," J. Microsc. (Oxf.) 182, 133-140
(1996).
18. M. A. A. Neil, R. Juskaitis, and T. Wilson, "Method of obtaining optical sectioning by using
structured light in a conventional microscope," Opt. Lett. 22, 1905-1907 (1997).
19. D. Lim, K. K. Chu, and J. Mertz, "Wide-field fluorescence sectioning with hybrid speckle
and uniform-illumination microscopy," Opt. Lett. 33, 1819-1821 (2008).
20. J. Mertz and J. Kim, "Scanning light-sheet microscopy in the whole mouse brain with HiLo
background rejection," J. Biomed. Opt. 15, 016027 (2010).
21. D. Bhattacharya, V. R. Singh, C. Zhi, P. T. C. So, P. Matsudaira, and G. Barbastathis, "Three
dimensional HiLo-based structured illumination for a digital scanned laser sheet microscopy
(DSLM) in thick tissue imaging," Opt. Express 20, 27337-27347 (2012).
22. H. Choi, E. Y. S. Yew, B. Hallacoglu, S. Fantini, C. J. R. Sheppard, and P. T. C. So,
"Improvement of axial resolution and contrast in temporally focused widefield two-photon
microscopy with structured light illumination," Biomed. Opt. Express 4, 995-1005 (2013).
23. J. Michaelson, H. J. Choi, P. So, and H. D. Huang, "Depth-resolved cellular microrheology
using HiLo microscopy," Biomed. Opt. Express 3, 1241-1255 (2012).
24. A. Weigel, D. Schild, and A. Zeug, "Resolution in the ApoTome and the confocal laser
scanning microscope: comparison," J. Biomed. Opt. 14, 014022 (2009).
25. P. A. Stokseth, "Properties of a Defocused Optical System," J. Opt. Soc. Am. 59, 1314-1321
(1969).
26. D. Karadaglic and T. Wilson, "Image formation in structured illumination wide-field
fluorescence microscopy," Micron 39, 808-818 (2008).
27. W. Gobel, B. M. Kampa, and F. Helmchen, "Imaging cellular network dynamics in three
dimensions using fast 3D laser scanning," Nat Methods 4, 73-79 (2007).
28. E. J. Botcherby, R. Juskaitis, M. J. Booth, and T. Wilson, "Aberration-free optical refocusing
in high numerical aperture microscopy," Opt. Lett. 32, 2007-2009 (2007).
29. M. Born and E. Wolf, Principles of Optics, 7th ed. (Pergamon, 1999).
30. E. J. Botcherby, R. Juskaitis, M. J. Booth, and T. Wilson, "An optical technique for remote
focusing in microscopy," Opt. Commun. 281, 880-887 (2008).
31. M. C. Miller, G. V. Doyle, and L. W. Terstappen, "Significance of circulating tumor cells
detected by the CellSearch system in patients with metastatic breast colorectal and prostate
cancer," J. Oncol. 2010, 617421 (2010).
32. V. R. Singh, J. C. Rajapakse, H. Yu, and P. T. C. So, "Intensity normalization of two-photon
microscopy images for liver fibrosis analysis," Proc Spie 7903(2011).
33. R. C. Gonzalez, R. E. Woods, and S. L. Eddins, Digital image processingusing MA TLAB, 2
ed. (Pearson Prentice Hall, Knoxville, 2009).
34. P. Soille, MorphologicalImage Analysis: Principlesand Applications (Springer-Verlag
1999).
35. Y. Zhang, B. J. Matuszewski, L. K. Shark, and C. J. Moore, "Medical image segmentation
using new hybrid level-set method," 71-76 (2008).
36. F. Meyer, "Topographic Distance and Watershed Lines," Signal Process 38, 113-125 (1994).
45
This page is intentionally left blank
46
Chapter 3
Depth resolved hyperspectral imaging cytometer based on
structured light illumination and Fourier transform
interferometry
3.1 Introduction
Spectral resolved imaging is a very powerful method to quantify cellular biochemical
environment identifying chemical species and quantifying their concentration [1]. By labeling
cellular or tissue constituents with different fluorescent labels, multicolor imaging allow more
precise determination of specimen morphology and composition; an example is the karyotyping
of human chromosomes [2]. The availability of many fluorescence probes with emission spectra
sensitive to their chemical environment further allows intracellular mapping of biochemical
states including ion concentration, pH, and voltage. Spectral resolved imaging was also shown to
be an effective tool for studying protein-protein interactions based on resonance energy transfer
measurement [3]. Spectral resolved imaging is also valuable for quantitatively identifying
spectral signatures of endogenous biochemical species and to measure their concentrations in
vivo. In clinics, spectroscopic measurement of cellular biochemistry can often provide
information complementary to morphological analysis affording more accurate diagnosis, more
precise demarcation of surgical margin, and more reliable evaluation of treatment outcome [4-6].
Spectral resolved imaging of course does not have to be based on fluorescent contrast, many
non-fluorescent analytes can be quantified based on their vibronic spectra using Raman imaging
[7].
While bandpass filter wheel based spectral imaging is useful in applications where a few
wavelength bands are sufficient for analysis, more sophisticated applications requires higher
spectral resolution and fast temporal resolution. The goal of the spectrally resolved image is to
record a 3D spectral cube which is composed of the pixels in 2D spatial dimension with its full
spectral signature in spectral dimension.
Currently available hyperspectral imaging methods can be classified as scanning and
non-scanning type. The scanning type methods can be further classified into three categories of
47
spatial, spectral and optical path difference (OPD) scanning depending on the domains in which
the scanning is performed. In the first case, the full spectrum is recorded for a point or a line with
spectrally dispersive elements such as prism or grating and with 2D detector array. 4D data cube
is sequentially recorded by scanning in 3D spatial locations [1]. In the second case, the full
spatial 2D field of view is imaged with 2D detector simultaneously and a single wavelength 2D
image is sequentially recorded by scanning the wavelength with the tunable bandpass filters such
as the acousto-optical tunable filter (AOTF), the liquid crystal tunable filter (LCTF), Fabry-Perot
tunable filter and the angle tuned thin film bandpass filter [8]. In the third case, spectral data is
inferred after taking a sequence of 2D image data by scanning OPD between the two emission
beam paths in the interferometer. This category is termed as Fourier transform spectroscopy
(FTS). There are different types of FTS depending on how OPD is scanned in the interferometer.
OPD scanning can be implemented in various ways but the most commonly adapted mechanisms
perform OPD scanning by either translating the mirror in one of the beam path in Michelson
interferometer [7, 9] or rotating the whole Sagnac interferometer including the beam splitter and
the two reflective mirrors with respect to the incoming beam [10] or translating the Wollaston
prism in the birefringent FTS [11]. Among these, Sagnac and birefringent type FTS have the
advantages in stability because of its common path geometry.
The non-scanning type FTS acquires 3D (x, y, A) data cube in a single exposure without
scanning at all. This types of spectrometer have the advantage in terms of temporal resolution but
typically there is a tradeoff between spectral and spatial resolution and often they require
extensive computation in post-processing. For example, the image mapping spectrometer (IMS)
records 3D (x, y, A) data cube in single shot by sharing the CCD detector for recording both
spatial and spectral data simultaneously. However, the spatial resolution (or the field of view ) is
reduced as the spectral resolution increases [12].
For the depth resolved spectral imaging, the 3D imaging techniques such as the point
scanning confocal microscopy or the two photon microscopy can be combined with spectrometer
based on dispersive elements [1]. Alternatively, the depth resolved widefield technique such as
the temporally focused widefield two photon microscopy [13, 14] or the light sheet fluorescence
microscopy [15] can be combined with the wavelength scanning spectrometer. Another approach
for the depth-resolved widefield imaging especially for single photon fluorescence imaging is to
use a structured light illumination (SLI). SLI approach has been demonstrated in the Sagnac
interferometer based imaging FTS where the excitation light passing through FTS forms a SLI
48
which encodes the depths information and is used to reject the out-of-focus photons from the
emission light which pass through the same FTS [16]. Also, SLI images based on phase-shifted
sinusoidal illumination has been adapted to reject out-of-focus light in IMS [17].
In this chapter, we describe a new implementation of depth-resolved hyperspectral
imaging cytometer based on SLI and FTS. Hyperspectral data cube is acquired with the Sagnac
interferometer type FTS and the signals from out-of-focal planes are rejected by SLI based on
HiLo microscopy. The current configuration has the advantage of achieving both high spatial and
spectral resolution images. The overall system configuration is described in Section 3.2. The
general working principle of the FTS based on Sagnac interferometer is described in Section 3.3.
The proposed method of depth resolved FTS is mathematically derived in the Section 3.4. In
section 3.5, the spectral background rejection with SLI is experimentally demonstrated with the
fluorescent bead immersed in the uniform background. The depth-resolved hyperspectral 3D
imaging is demonstrated with a kidney tissue section and densely aggregated cells labeled with
fluorophores with two different colors.
3.2 Overall instrument design of the imaging Fourier transform spectrometer
Fig.3.1 shows the schematic layout of the depth-resolved hyperspectral image cytometer.
Three diode-pumped solid states lasers (CNI laser) with emission wavelength of 473nm, 561nm
and 660nm respectively were used as light sources that can excite broad range of fluorophores.
The three beams are combined into one collinear beam using the laser beam multiplexers
(LMO1-503-25, LM01-613-25, Semrock). Photobleaching effect is minimized by controlling
passage of the excitation light with the optical shutter (Ch-61, EOPC) which operates in
synchrony with the exposure status signal of the camera so that the sample is exposed to the
excitation light only when the camera is recording the emission light. The diffractive optical
element (DS-033-Q-Y-A, HOLO/OR) split the beam into +1 and -1 order and minimize the
power loss by 0th order and other higher diffractive order beam. Since the beams spitted by DOE
reaches to the focal plane of the objective via common path, they form a stable fringe pattern for
SLI. The optical shutter (Ch-61, EOPC) placed in one of the diffracted beam path blocks the
beam for a uniform illumination (UT) and unblock the beam for a structured illumination (SI)
which is formed by the interference of the two beams. The optical shutter is triggered by the
external TTL signal and enables high speed switching between UI and SI. The dichroic mirror
was used to separate the excitation beam from the emitted fluorescence signal and the emission
49
filter right in front of the detector was used to block any stray excitation light reflected from the
intermediate optical elements. The z-scanning of the sample is performed remotely by first
forming the perfect 3D image of the sample in the focal space of the remote focusing objective
and then by scanning the mirror in axial direction with piezo actuator [18]. The hyperspectral
image is acquired by recording the interferogram formed by the Sagnac interferometer which is
subsequently processed to build the spectral cube of every pixels in two dimensional field of
view.
PA
M
-------------------ROBJ
L,=100
Lg=200
QWP
M
/PB
L7=150
L10=150
EmF
660nm I561nm 473nm
O
M
DM
L6=164.5
L= 6 .
DOE
DM
L,=75
L,=150
L3= 75
01S2
0M
L4
= 300
L5 = 150
OBJ
FFP
Fig.3.1. Schematic diagram of the high throughput depth-resolved imaging Fourier transform
spectrometer. Enclosed by dotted line is the Sagnac interferometer. M: mirror, DM: dichroic mirror,
DOE: diffractive optical element, OS: optical shutter, PBS: polarizing beam cube splitter, BS: 50R150T
plate beam splitter, QWP: quarter-wave plate, EmF: emission filter, PA: piezo actuator, FFP: front focal
plane, OBJ: objective lens (Zeiss 20x Water NAL.0), ROBJ: remote focusing objective lens (Nikon 20x
Air NAO.75)
3.3 Principle of imaging Fourier transform spectrometer based on Sagnac
interferometer
In imaging Fourier transform spectrometry, each pixel of the 2D detector records
simultaneously the spectrum of the source at the spatial location that is the conjugate point of the
pixel. The emission light from the object first forms an intermediate image which is relayed to
the 2D detector by the telescope. The interferometer placed in between the telescope split the
emission light into two paths and induce phase delay (OPD) in one of the beam path. Then, the
50
two beams that are phase delayed recombined at the 2D detector which records the time average
intensity of the two electric fields that are phase delayed by the interferometer. Mathematically,
these processes can be described as follows. The incident electric field upon the interferometer
can be expressed as
E(x, y, v) = A(x, y, v) exp [i (2;fvt + #(x, y, v))]
(3.1)
where v = c/l, c is the speed of light, A is the wavelength of the emission light, x and y are the
spatial location in the focal plane, A is the amplitude of the electric field. When OPD of the two
beam paths in the interferometer differs by 6, the intensity pattern recorded at the detector is the
time average of the sum of the electric fields from path 1 and 2.
E +E2 =A(x,y,v)exp[i(2nvt+#(x,y,v))](1+exp[i(27rv/c)6])
(3.2)
I(x, y, 3) = 2 A(x, y,v)1 2 (I + cos((21rv/c)S))
(3.3)
For a broad band source, it is the sum of all individual frequency components.
I(x, y,S)
=
2f A(x, y,v)12 (I + cos((27rv/c)6))dv
(3.4)
The detected intensity can be measured as a function of OPD between the two beam paths, which
is called the interferogram. In Eq.(3.4), the first term is the dc term and the second term contains
the spectral information. The intensity of source spectra, IA(x,y,v)1 2, can be readily recovered by
cosine transform (the real part of the Fourier transform) of the interferogram. Mathematically,
the interferogram is the autocorrelation function of the electric field. The inverse Fourier
transform of the autocorrelation function of the electric field amplitude as a function of position
is the spectrum, i.e., flux density as a function of wave number. This is known as the WienerKhinchine theorem [19].
Fig.3.2
shows
the imaging Fourier transform
spectrometer
based on
Sagnac
interferometer. The basic configuration of the interferometer design is based on the commercial
device (SpectraCube300, Applied Spectral Imaging). As shown in Fig.3.2(a), the emission light
from the intermediate image plane is collimated by the first lens and split into two paths which
corresponds to the transmitted beam and the reflected beam at the beam splitter respectively.
Then, the two beams that are phased delayed are focused to the sCMOS camera (PCO.EDGE 5.5,
PCO) by the second lens. OPD originates from the fact that the two beam paths in the Sagnac
interferometer are not parallel within the beam splitter and have different optical path length for
a nonzero rotation angle of the Sagnac interferometer with respect to the position where the
normal to the beam splitter in the Sagnac interferometer makes a 450 with respect to the optical
51
axis [10]. Optical path difference between the two beam paths can be express in Eq. (3.5) which
is a function of the thickness (t) and the refractive index (n) of the beam splitter, the angle of the
optical axis of the telescope on the beam splitter (8) and the rotation angle of the Sagnac
interferometer (6) with respect to the central position as shown in Fig. 3.2(a) [20].
OPD(p,0,t, n) = t [(n2
sin2(8 +))"'
(3.5)
+ 2sinfpsin ]
-(n2 -sin2(6 -))"'
In case of small 6 near the 6=0, the OPD can be approximated as a linear function of 6 as
expressed in Eq. (3.6).
OPD(p,0,t,n)= 21 sin p [1-(2 - sin 2
-0. cos
(3.6)
#
Fig.3.2(c) shows the OPD scanning range for the spatial point on the optical axis. Since the
spatial points off the optical axis have a non-zero input angle with respect to the optical axis after
the first lens, the origin of the OPD scanning range shifts in proportion to the distance of the
spatial point from the optical axis. Thus, each pixel along the horizontal direction experiences a
OPD span that are shifted in proportional to the distance off the optical axis for the same
rotational angle of the Sagnac interferometer.
(a)
(b)
(c)
60 - - - - - - - - - - - - - 20........................
......
...........
0 ........................
........ ...
-- -- - - --40 --20 ............. ............ .......... . ...
...... ...........
0. ..........
0 04............
I
.
.............
-60 ...
~
U
\
-2'\
.1
0
1
Sagnac rotation angle (8) [degree]
2
Fig. 3.2 (a) Ray tracing from the intermediate image plane to the detector at central position of the Sagnac
interferometer. Color represents the spatial field points. (b) Photograph of the setup installed in the
emission beam path of the image cytometer. (c) OPD as a function of the rotational angle of the Sagnac
interferometer for the spatial field point on the optical axis (the rays represented in green color in (a) with
the following parameters : = 45*, t = 2.93mm, n = 1.517 (Refractive index of N-BK7)
The sampling process is performed in two steps. First, the 2D detector array capture the
interferogram across the field of view at a given angle of 6. As shown in Fig.3.3, OPD has a
linear variation along the x direction due to the non-zero input angle of the ray from the off-axis
spatial points while the OPD along the y direction is identical. Second, the zero OPD line is
scanned across the field of view along x direction by rotating the Sagnac interferometer so that
each pixel observes at least the single-sided interferogram. The double sided interferogram can
52
provide a higher signal to noise ratio and induce minimal phase distortion compared to single
sided interferogram at the expense of increased data acquisition time. For every pixel to observe
the double sided interferogram the zero OPD line has to be scanned more than half the width of
the double sided interferogram from the right hand edge of the field of view and finish more than
half the width of the double sided interferogram from the left hand edge of the field of view. The
sampling periods in space and time are determined by the pixel size of the 2D detector array and
the stepping size of the OPD shift respectively [10]. To satisfy the Nyquist sampling theorem, a
period of the fringe has to be sampled at least by two pixels of the 2D detector and the OPD
stepping size has to be smaller than the half the wavelength. Therefore, the minimum detectable
wavelength can be expressed as Eq. (3.7).
Amin( or 1/ma
(3.7)
)= 2A OPD
where the sampling frequency, a =1/ AOPD. The spectral resolution is determined by the total
OPD span due to the Fourier transforming relation between the interferogram and the spectrum
and can be expressed in Eq. (3.8).
X
(a)
(3.8)
1
A=
OPD.
(b)
1000
(c)
1
0.8
500
0.6
0
0
..
~0.4
0
0.2.
-500
-100
-..
0
Optical Path Difference[um]
10
00
500
600
700
wavelength [nm]
800
Fig.3.3. Sampling of the interferogram (a) Fringe pattern of a white light LED at 0 = 0* sampled by the
sCMOS camera. The dotted yellow vertical line represents a zero OPD which shifts to the left as the
interferometer rotates. Pixels in each column see the same OPD and each column see a linearly varying
OPD (b) Sampled interferogram observed by the yellow pixel at AO = 0.005* (2 pixel or 170nm in OPD
space) (c) Recovered spectrum of LED after quad band emission filter with minimum detectable
wavelength of 340nm.
The raw interferogram is first band pass filtered by digital FIR filter with pass band
between 400nm and 700nm. In case of biological sample, the photo bleaching is observed during
the long range of OPD scanning and corrected by modeling the decaying dc values of
53
interferogram with exponentially decaying function and subsequently dividing the raw
interferogram with the exponentially decaying function. Digital bandpass filtering also has
similar effect of correcting the photon bleaching since the low frequency of exponential decay is
filtered out by the band pass filter. Afterward, the interferogram is phase corrected [21-23],
apodized with hanning filter, zero padded and then Fourier transformed by fast Fourier transform
algorithm and then taken the absolute value.
The actual OPD generated by the interferometer deviates from the theoretical estimation
given by Eq. (3.5) in a number of reasons. First of all, since the OPD is a function of the
refractive index of the beam splitter, each wavelength see a different range of OPD depending on
the dispersion property of the beam splitter material. In addition, OPD is sensitive to the beam
splitter angle and slight misalignment generate a huge error in the OPD estimation. To
compensate these errors, the spectrometer is first calibrated with three color lasers at 473nm,
561nm, 660nm wavelength. The interferogram first recorded with each color laser as a light
source and then the wavelength is obtained using the OPD calculated using Eq.(3.5). Then the
calibration factor is calculated by dividing the expected wavelength by the measured wavelength
for each lasers. Then, the OPD is rescaled by multiplying the calculated OPD with the average of
the three calibration factors. This procedures can be summarized as follows.
CF=-
473
A.
,CF=
561
A.m
2m
,
CF3 =
660
CF =(C, +CF2 +CF)/3
OPDeled = CF -OPD
After the initial calibration, the spectrum of the known emission wavelength fluorophores
are measured and compared with their reference values. Fig. 3.4 shows the raw interferogram
and the corresponding spectrums of a 6pm Yellow-Green fluorescent bead (F-8859, Molecular
Probe), Rhodamine 6G solution (Sigma-Aldrich), a 4ptm Red fluorescent bead (F-8858,
Molecular Probe). The samples are excited with the 473nm laser and the emission light is
detected with the long pass dichroic mirror (FF495-DiO2-25x36) and the long pass emission
filter (BLPOI-488R-25) for measuring the full emission spectrum.
54
(a)
4000
(b)
3000
2000
2000
2000
4
10000
-200-2000
-4000
-30
-20
-10
0
000
-2000
-3000
0
10
Optical Path Dlfferencelumi]
20
30
-30
-20
-10
10
0
Optical Path Difference[um]
20
30
-30
1
0
10
20
30
(f)
0.8
0.8-
--
0.6
0.6
0.4
OA
0.4
0.2
0.2
0.2
500
-10
Optical Path Difference[uml)
11
-
-
so0
-20
(e)
(d)
0.8
(c)
4000
550
600
wavelength (nm]
650
700
450
-
500
550
0.6
-
600
wavelength mm]
650
700
~
50
500
550
wavelength
600
650
700
[nm7
Fig. 3.4. Interferogram of 6pm yellow green fluorescent bead (a), Rhodamine 6G solution (b), 4gm red
fluorescent bead (c) and its the processed spectrum are (d), (e) and (f). Green curve in spectral graph is
the reference value from http://www.lifetechnologies.com/us/en/home/life-science/cell-analysis/labelingchemistry/fluorescence-spectraviewer.html
3.4 Depth-resolved imaging Fourier transform spectrometry
Since the fringe formed by the interferometer is only determined by the light distribution
in the input aperture of the interferometer, they form interferogram with the same modulation
depth whether the light is from in- or out-of-focus plane. Therefore, the recorded interferogram
contains both in-focus and the out-of-focus spectra information and the post-processed spectral
cube from these raw interferogram would not provide depth-resolved spectral information. One
of the methods for rejecting the spectral background signal is to use a structured light
illumination (SLI) to encode the depth information in the (x, y, X) spectral cube. The degree of
modulation depth in the X-plane contains the depth information.
So far, a class of depth-resolved imaging techniques based on SLI have been proposed to
select a particular imaging plane and to reject out-of-focus background for standard wide-field
single-photon microscopy [24, 25]. Of these methods, one effective approach we adapted is
termed 'HiLo microscopy' which generates an optically sectioned image by post-processing the
uniformly illuminated image (UI) and the structured light illuminated image (SI) [25]. This
algorithm is based on an assumption that 2D image can be divided into low frequency and high
55
frequency contents. Since the high frequency contents are in-focus by its nature, the goal of
using SLI is to encode the in-focus low frequency contents especially laterally zero frequency
component with high frequency SLI. More specifically, the in-focused high frequency contents
are extracted by high-pass filtering UL with a Gaussian shaped high-pass filter. The in-focus low
frequency contents are extracted by low-pass filtering the absolute of UI subtracted by SI with
the complementary low pass filter to the high pass filter. The cutoff frequency of the Gaussian
filter is determined by the sinusoidal spatial frequency of the structured illumination.
Subsequently, the optically sectioned image is obtained by combining the two with an adjusting
factor so that the transition from low to high frequencies occur smoothly [26]. HiLo microscopy
has been widely used in the context of the background rejection for light-sheet microscopy [26,
27], the temporally focused widefield two photon microscopy [28] and the depth-resolved
microrheology [29].
HiLo method can be applied to the spectral background rejection by first constructing the
(x, y, X) spectral cubes for both UI and SI and then post-processing each k-planes from UI and SI
with HiLo algorithm. Following the similar approach by Heintzmann et al.[16], we derived the
mathematical formulation behind the spectral background rejection using SLI. The fringe pattern
is imposed along the x direction by the interference of the two plane wave, which can be
expressed as :
I,,(k,,, x) = 2(1 + cos(kex))Io(k)
(3.9)
where x is the spatial dimension, Io is the intensity of one of the beam. The emission spectrum of
a fluorophore at x can be expressed as
(3.10)
I,.(k, k,., x) = msLIIex(kx, x)Sm (kn x)p(x)Sx (kx,x)
where p(x) is the local fluorophore distribution, Sexem (kexem, x) is the excitation and emission
spectrum of the fluorophore, msLI is the degree of modulation of the emission light excited by the
structured illumination which decreases as the object is defocused. According to Eq.(3.4), the
interferogram observed by a pixel in the detector as the OPD is scanned can be expressed as
I,,,(9,k.,x) =f,(1 +m cos(kmJf))I.,7,,,k.,,,x)k,,
=[ msI,(k, x)p(x)S,(k.,x)]0 (1+m cos(k,,6)) S,,(k,
3.1
x)d&,,,
where 8 is the optical path difference, mb, is the degree of modulation of the two interfering
emission light, which can vary depending on the beam splitting ratio. From the Fourier transform
of Eq. (3.11), the spectral cube of SI can be obtained as
56
Ss,(keM,x) =
", Id/ (,ke,x)e-j'k-d9
'5x..2
(3.12)
= [ ms1 I,(k., X)] [mbsp(Sx (k, x)] Sm (kr, x)
First term of Eq.(3.12) implies that each X plane of the spectral cube preserves the depth
information encoded by the structured light illumination. Now the situation is similar to the
intensity background rejection by SLI.
3.5 Experimental validation of spectral background rejection
3.5.1 Hyperspectral imaging
The hyperspectral imaging is demonstrated with the prepared slide containing the
muntjac skin fibroblast cells (F36925, Invitrogen) stained with three different color fluorescent
dyes. The filamentous actin is labeled with green fluorescent Alexa Fluor 488 phalloidin with
emission peak at 519nm. Mitochondria are labeled with an anti-OxPhos Complex V inhibitor
protein mouse monoclonal antibody in conjunction with orange fluorescent Alexa Fluor@ 555
goat anti-mouse IgG with emission peak at 571nm. Nuclei are labeled with the red fluorescent
TO-PRO-3 nucleic acid stain with emission peak at 661nm. The spectral images is first acquired
with the band pass filter as shown in Fig. 3.5(a) and then the hyperspectral image is taken for the
area marked in red box in Fig. 3.5(a) as shown in Fig. 3.5(b).
57
(a)
(b)
509nm
521nm
534nm
547nm
561nm
576nm
592nm
608nm
626nm
Fig. 3.5 (a) Two color image of the muntjac skin fibroblast cells (F36925, Invitrogen). The green color
represents filamentous actin imaged with the 525nm/35nm bandpass filter (FFOl-520/35, Semrock) and
the orange color represents mitochondria imaged with 609nm/54nm band pass filter (FF01-609/54,
Semrock). (b) X plane images of the region marked in red box in (a) at selected wavelengths.
3.5.2 Spectral background rejection by SLI
We first demonstrate the spectral background rejection in a situation where a 6 pm
yellow-green fluorescent bead is placed in the focal plane and a uniform Rhodamine solution is
placed in the out-of-focal plane. The sample is excited with both 473nm and 561nm laser to
equalize the emission light from both colored samples. The emission light is detected with the
quad-band dichroic mirror (FF410/504/582/669 -DiOl-25x36, Semrock) and the quad-band
emission filter (FFOl-440/521/607/700-25, Semrock). Fig. 3.6(a) shows the spectral images for
UI, SI and after the HiLo processing. In UI, the spectra from both the in-focused bead and the
out-of-focus Rhodamine solution are detected, which demonstrates that there is no depth
58
discrimination. In SI, the fringe pattern is visible only on the in-focused bead. After the HiLo
processing the spectrum from the out-of-focus Rhodamine is rejected. Fig. 3.6(b) shows the
spectra of a point in the center of the bead. After HiLo processing, the out-of-focus spectrum is
rejected.
(a)
UI
SI
(b)
HiLo
120
"
100
A=521 nm
HiLo
80
60
CD
k=608nn
40
20
50
500
550
600
650
700
Wavelength [nm]
Fig. 3.6 (a) Spectral images from UI, SI and after HiLo processing at X = 521nm and X = 608nm (b)
Spectra for UI and after HiLo processing. The spectrum from the out-of-focus Rhodamine solution is
rejected after the HiLo processing.
Next, we obtained a depth resolved hyperspectral images with the same prepared cell
sample used for Fig. 3.5 and the unifrom Rhodamine solution. As in the previous case, the cells
are in-focus and the Rhodamine solution is out-of-focus. The samples are excited with the
473nm laser and the emission light is detected with the long pass dichroic mirror (FF495-DiO225x36) and the long pass emission filter (BLP01-488R-25) for measuring the full emission
spectrum. As shown in Fig. 3.7, the background signal from the Rhodamine is present over the
broad emission wavelength range from 525nm to over 600nm with strongest contribution around
550nm. The HiLo processing removes this background and improves the contrast. The
mitochondria which has similar emission wavelength with the Rhodamine is more effectively
represented by rejecting the background noise.
59
UI
528nm
547nm
HiLo
576nm
528nm
547nm
576nm
Fig. 3.7 X plane images of the muntjac skin fibroblast cells (F36925, Invitrogen)
background for UI and after HiLo processing.
with Rhodamine
We further demonstrate the depth-resolved hyperspectral imaging with a prepared slide
of sectioned mouse kidney (F24630, Invitrogen). The sample is imaged with the same imaging
condition for Fig. 3.6. Each spectral channel of the hyperspectral images are combined to
produce a true color image as shown in Fig. 3.8. The thickness of the sample is 15 m and the
very top surface was put in focus during data acquisition. The contrast of the HiLo processed
images improves over the UI and the weak fluorescent features are more clearly visible.
60
UI
HiLo
Fig. 3.8 True color images of mouse kidney (F24630, Invitrogen) for UI and after HiLo processing. Green
colored feature represents the elements of the glomeruli and convoluted tubules labeled with Alexa Fluor
488. Orange colored feature represents the filamentous actin and the brush border labeled with Alexa
fluor 568.
3.6. Conclusion
This paper presented a depth resolved widefield hyperspectral imaging cytometer based
on the imaging Fourier transform spectrometer and the structured light illumination. While the
Sagnac interferometer in the emission beam path does not have a depth resolving power, the
structured light illumination generated in the excitation beam path has been demonstrated to
encode depth information in the recorded interferogram and the spectral background has been
rejected with HiLo algorithm.
Since the OPD has a linear distribution across the field of view at any given instance in
the Sagnac type imaging FTS, the zero OPD line has to be scanned across the entire field of view
for each pixel at least to observe the single-sided interferogram. Currently, sampling
interferogram is performed according to the Nyquist sampling criteria. Therefore, the total data
acquisition time increases in proportion to the width of the field of view and can induce a
significant photobleaching in case of biological sample. A recent development in the signal
processing such as fast orthogonal
search algorithm or compressive sensing technique may
61
allow optical spectrum to be inferred with significantly fewer sampling of the interferogram
while keeping the same spectral resolution [30, 31].
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
C. Buehler, K. H. Kim, U. Greuter, N. Schlumpf, and P. T. C. So, "Single-photon counting
multicolor multiphoton fluorescence microscope," J. Fluoresc. 15, 41-51 (2005).
E. Schrock, S. duManoir, T. Veldman, B. Schoell, J. Wienberg, M. A. FergusonSmith, Y.
Ning, D. H. Ledbetter, I. BarAm, D. Soenksen, Y. Garini, and T. Ried, "Multicolor spectral
karyotyping of human chromosomes," Science 273, 494-497 (1996).
C. Thaler, S. V. Koushik, P. S. Blank, and S. S. Vogel, "Quantitative multiphoton spectral
imaging and its use for measuring resonance energy transfer," Biophys. J. 89, 2736-2749
(2005).
Z. Malik, M. Dishi, and Y. Garini, "Fourier transform multipixel spectroscopy and spectral
imaging of protoporphyrin in single melanoma cells," Photochem. Photobiol. 63, 608-614
(1996).
T. D. Wang, G. Triadafilopoulos, J. M. Crawford, L. R. Dixon, T. Bhandari, P. Sahbaie, S.
Friediand, R. Soetikno, and C. H. Contag, "Detection of endogenous biomolecules in
Barrett's esophagus by Fourier transform infrared spectroscopy," P. Natl. Acad. Sci. U.S.A.
104, 15864-15869 (2007).
H. Akbari, L. V. Halig, D. M. Schuster, A. Osunkoya, V. Master, P. T. Nieh, G. Z. Chen, and
B. W. Fei, "Hyperspectral imaging and quantitative analysis for prostate cancer detection," J
Biomed Opt 17(2012).
C. J. H. Brenan and I. W. Hunter, "Design and characterization of a visible-light Fouriertransform raman spectrometer," Appl. Spectrosc. 49, 1086-1093 (1995).
P. F. Favreau, T. C. Rich, P. Prabhat, and S. J. Leavesley, "Tunable thin-film optical filters
for hyperspectral microscopy," Proc. Spie 8589, 85890R (2013).
M. R. Carter, C. L. Bennett, D. J. Fields, and F. D. Lee, "Livermore imaging Fourier
transform infrared spectrometer (LIFTIRS)," Proc. Spie 2480, 380-386 (1995).
Z. Malik, D. Cabib, R. A. Buckwald, A. Talmi, Y. Garini, and S. G. Lipson, "Fourier
transform multipixel spectroscopy for quantitative cytology," J. Microsc. (Oxf.) 182, 133-140
(1996).
A. R. Harvey and D. W. Fletcher-Holmes, "Birefringent Fourier-transform imaging
spectrometer," Opt. Express 12, 5368-5374 (2004).
L. Gao, R. T. Kester, and T. S. Tkaczyk, "Compact Image Slicing Spectrometer (ISS) for
hyperspectral fluorescence microscopy," Opt. Express 17, 12293-12308 (2009).
D. Oron, E. Tal, and Y. Silberberg, "Scanningless depth-resolved microscopy," Opt. Express
13, 1468-1476 (2005).
G. H. Zhu, J. van Howe, M. Durst, W. Zipfel, and C. Xu, "Simultaneous spatial and temporal
focusing of femtosecond pulses," Opt. Express 13, 2153-2159 (2005).
J. Huisken, J. Swoger, F. Del Bene, J. Wittbrodt, and E. H. K. Stelzer, "Optical sectioning
deep inside live embryos by selective plane illumination microscopy," Science 305, 10071009 (2004).
R. Heintzmann, K. A. Lidke, and T. M. Jovin, "Double-pass Fourier transform imaging
spectroscopy," Opt. Express 12, 753-763 (2004).
L. Gao, N. Bedard, N. Hagen, R. T. Kester, and T. S. Tkaczyk, "Depth-resolved image
mapping spectrometer (IMS) with structured illumination," Opt. Express 19, 17439-17452
(2011).
62
18. E. J. Botcherby, R. Juskaitis, M. J. Booth, and T. Wilson, "Aberration-free optical refocusing
in high numerical aperture microscopy," Opt. Lett. 32, 2007-2009 (2007).
19. R. J. Bell, Introductory Fourier Transform Spectroscopy (Academic press, New York and
London, 1972).
20. D. Cabib, Z. Friedman, S. Lipson, and R. Buckwalf, "Method for simultaneously measuring
the spectral intensity as a function of wavelength of all the pixels of a two dimensional
scene," US Patent 5539517 (1996).
21. M. L. Forman, W. H. Steel, and G. A. Vanasse, "Correction of Asymmetric Interferograms
Obtained in Fourier Spectroscopy," J Opt Soc Am 56, 59-& (1966).
22. L. Mertz, "Auxiliary Computation for Fourier Spectrometry," Infrared Phys 7, 17-& (1967).
23. H. Sakai, G. A. Vanasse, and M. L. Forman, "Spectral Recovery in Fourier Spectroscopy," J
Opt Soc Am 58, 84-& (1968).
24. M. A. A. Neil, R. Juskaitis, and T. Wilson, "Method of obtaining optical sectioning by using
structured light in a conventional microscope," Opt. Lett. 22, 1905-1907 (1997).
25. D. Lim, K. K. Chu, and J. Mertz, "Wide-field fluorescence sectioning with hybrid speckle
and uniform-illumination microscopy," Opt. Lett. 33, 1819-1821 (2008).
26. J. Mertz and J. Kim, "Scanning light-sheet microscopy in the whole mouse brain with HiLo
background rejection," J. Biomed. Opt. 15, 016027 (2010).
27. D. Bhattacharya, V. R. Singh, C. Zhi, P. T. C. So, P. Matsudaira, and G. Barbastathis, "Three
dimensional HiLo-based structured illumination for a digital scanned laser sheet microscopy
(DSLM) in thick tissue imaging," Opt. Express 20, 27337-27347 (2012).
28. H. Choi, E. Y. S. Yew, B. Hallacoglu, S. Fantini, C. J. R. Sheppard, and P. T. C. So,
"Improvement of axial resolution and contrast in temporally focused widefield two-photon
microscopy with structured light illumination," Biomed. Opt. Express 4, 995-1005 (2013).
29. J. Michaelson, H. J. Choi, P. So, and H. D. Huang, "Depth-resolved cellular microrheology
using HiLo microscopy," Biomed. Opt. Express 3, 1241-1255 (2012).
30. M. J. Korenberg, C. J. H. Brenan, and I. W. Hunter, "Raman spectral estimation via fast
orthogonal search," Analyst 122, 879-882 (1997).
31. V. Studer, J. Bobin, M. Chahid, H. S. Mousavi, E. Candes, and M. Dahan, "Compressive
fluorescence microscopy for biological and hyperspectral imaging," P. Natl. Acad. Sci.
U.S.A. 109, E1679-E1687 (2012).
63
This page is intentionallyleft blank
64
Chapter 4
Two photon point scanning endomicroscopy for clinical
diagnosis
Endoscopy operates on the same resolution and depth penetration regime as a typical
laboratory microscope but its miniaturization enables clinical applications. Endoscopy allows
noninvasive examination of the gastrointestinal (GI) track and the epithelial surface of other
internal organs of a patient. Since many cancer types occur on the epithelial surface of interior
body cavities, endoscopes, and endomicroscopes (a term that we will use interchangeably), are
routinely used in clinical diagnosis and treatment today.
The importance of developing more effective endoscopes can be seen from the incidence
rate of some of these diseases. For example, the incidences of colorectal, esophageal, and
cervical cancers are over 12,000, 150,000, and 15,000 new cases per year in USA, respectively.
To further illustrate the use of this new technology for cancer detection, we will examine the
situation with cervical cancer in greater depth [1-6]. The onset of cervical cancers has been
associated with a sexually transmitted human papillomavirus (HPV) infection; an infection
which affects over 40 million people. Approximately one million new cases of HPV are detected
each year. Some forms of HPV have been identified as the primary cause of cervical
intraepithelial neoplasia. HPV is often detected because of either the observation of genital warts
or from an abnormal Pap smear. The common Pap smear cannot localize the lesions. Patients
with an abnormal Pap smear are often referred to colposcopy, a conventional microscopic
imaging method of the cervix. Colposcopy has had false positive rates as high as 40%-50%.
Furthermore, high-grade disease may be misclassified as low grade. Punch biopsy, another
common technique, is subject to false negative results through incomplete sampling. This is an
area where endomicroscopy may be applied to identify the most critical biopsy sites and provide
more informative diagnosis, since endomicroscopy can provide subcellular level images similar
to histopathology.
In addition to the initial diagnosis, endomicroscopy may also provide a benefit of longterm, periodic monitoring of premalignant conditions. For example, Barrett's esophagus is very
65
common and is the result of acid reflux. Barrett's esophagus has a high chance of developing
into cancer and must be carefully monitored clinically [7, 8]. Endomicroscopy may permit more
accurate noninvasive monitoring of the potential progress of the disease at an organ site where
frequent and high-coverage excisional biopsy is highly undesirable. In the case of cervical
cancer, genital warts often involve surgical intervention, with all the associated risks thereof.
Low-grade lesions may be topically treated with trichloroacetic acid or 20% podophyllin
solution. More extensive cases are treated with cryosurgery, laser vaporization, or loop
electrosurgical excision. Since early stage genital warts often regress and excision procedures
may compromise patient fertility, a conservative management scheme has often been suggested.
The success of conservative management schemes depends on the existence of reliable clinical
markers for cancer progression, and imaging techniques to monitor them. Endomicroscopy may
allow a more accurate, noninvasive determination of the propensity for Barrett's esophagus and
cervical lesions to progress into a more invasive form.
4.1. Review of the endoscopic microscopy
4.1.1. Non-depth resolved Endomicroscopes
Conventional endoscopes operate in a wide-field imaging mode and most often at low
resolution. Endoscopes come in different formats but typically consist of a light source for
illumination, a lens for image formation, a fiber bundle for relaying the image to the image
sensor in the proximal end and an instrumentation channel for other treatments such as excisional
biopsy. Currently, endoscopy is routinely used in medical procedures for the diagnosis and
treatment of some diseases. For example, the bronchoscope, the colonoscope and the
hysteroscope are used for the inspection of the trachea and bronchi, the colon and large intestine
and the inside of the uterus, respectively [9]. Recently, the capsule type endoscope has been
introduced that takes pictures as it travels through the GI tract by the bowel movement, and
sends the images wirelessly to the receiver [10]. Patients swallow the pill type capsule and it can
acquire images in either from a head-on view (PillCam, Given Imaging [11]) or from a side view
(Sayaka, RF [12] ). The advantage of this type of endoscope is that it can examine the whole GI
tract, which is especially useful for imaging the small intestine where upper endoscopy and
colonoscopy cannot easily reach. The capsule type endoscope is also compatible with
examination on an outpatient basis, providing patient convenience and minimizing medical cost.
66
However, capsule type devices typically do not provide as high quality images as conventional
endoscopy.
In addition to generating image contrast based on scattered light, other image contrast
mechanisms such as tissue auto-fluorescence, spectroscopy, Raman scattering and fluorescence
lifetime can increase the endoscopic diagnostic accuracy [13]. Richards-Kortum and coworkers
have demonstrated that a high-resolution microendoscopy of intact oral mucosa has similar
diagnostic accuracy to the histopathological diagnosis result [14]. However, these techniques are
still confined to the surface imaging of the tissues and subject to limited utility for other types of
diseases, such as cancer occurring in the epithelial layer of tissue.
4.1.2. Depth resolved endomicroscopes
Since wide-field endomicroscope imaging can only provide tissue surface information, it
cannot replace traditional histopathological diagnosis where microscopic examination of
histological tissue sections provides morphological information on individual cells and their
organization within tissues. Therefore, wide-field microendoscopy is typically used to identify
suspicious tissue sites where an excisional biopsy will be taken so that a definitive clinical
diagnosis based on histopathological analysis can be performed. However, these procedures are
clearly invasive.
More importantly, excisional biopsy is time consuming, involving tissue
excision, fixation and subsequent examination by trained pathologists; the long cycle time of this
process often extends the time required to keep a patient in the operating room when excision
biopsy is being used for surgical margin determination. With the advent of high resolution 3D
endoscopic imaging technologies with the potential for providing images comparable with
excisional histopathology, there is a clear opportunity today to significantly extend the 3D, depth
resolved imaging capability of endoscopes and laparoscopes in clinical diagnosis. The three
classes of high-resolution endomicroscopes that have been developed today are based on optical
coherence tomography (OCT), confocal microscopy, and nonlinear optical (NLO) microscopy.
4.1.2.1 OCT endoscope
Optical coherence tomography is based on the optical ranging of reflected light pulses
from surfaces based on precise timing of femtosecond pulses interferometrically. Among the
three 3D endomicroscope approaches, endoscopes based on OCT are the most advanced because
67
the miniaturization of this technology is easier for two important reasons. Firstly, OCT systems
inherently obtain depth information based on the timing of the low-coherence optical pulses such
that no mechanical scanning in the depth direction is required. 2D sagittal or coronal sections of
the tissue can be generated by only mechanically scanning the light ray along a single direction
either radially or axially. Since scanning along only one direction is required, this greatly
simplifies micro-scanner design. Secondly, OCT automatically provides depth-resolved
information when the imaging optics have a large depth of field, i.e., a relatively low numerical
aperture. This longer depth of field, and hence longer working distance, significantly reduces the
complexity involved in assembling the micro-optical components in the distal end of the
endoscope at the expense of lower resolution. OCT endoscopes have entered into a number of
clinical trials including the identification of Barrett's esophagus [8, 15-23] and colonic polyps
[18, 23, 24]. OCT endoscope development is advanced enough that devices with an outer
diameter less than 2 mm have been fabricated and applied to the evaluation of intracoronary
stenting and the study of atherosclerotic plaques in patients [25-30]. While OCT endoscopes are
finding important clinical applications, the combination of using low NA optics, the relatively
narrow bandwidth of light sources, and the inherently low contrast of the OCT signal in tissues
limits the lateral and axial resolutions of these systems to about 5-7prm [31]. Therefore, OCT
endoscopes often do not provide images with a resolution comparable to standard
histopathology.
4.1.2.2 Confocal endoscope
Today, endoscopes with subcelluar resolution are based on confocal detection or nonlinear optical (NLO) excitation principles. Unlike in an OCT system, confocal and NLO
endomicroscopes obtain information from a single 3D-resolved location in the tissues. Scanning
along two directions must be performed to generate a 2D image. The need to incorporate
scanning along at least two directions presents a challenge in endoscope design. Nevertheless,
significant progress has been made in confocal based systems. Today, some confocal
endomicroscopes are becoming commercially available and preliminary clinical tests using these
devices have started [32]. Three classes of confocal endomicroscopes have been developed. The
most popular approach for confocal endomicroscopes utilizes high density, flexible fiber optical
bundles and performs scanning at the proximal end of the device [33-36]. The advantage of this
68
class of endomicroscope is its simplicity. Since scanning is performed at the proximal end,
outside the patient, the scanning mechanism does not require miniaturization, but the resolution
of these systems are limited by the fiber bundle pitch size and the static pattern noise caused by
imperfections fiber bundle fabrication. The second class of device incorporates micro-scanning
devices that scan micro-optic components, such as the tip of a fiber optic, which are integrated in
the distal end of the endomicroscope [37]. This class of device is very promising as high
resolution images have been demonstrated. However, the state-of-the art device is still fairly
large with over 6 mm outer diameter and relatively modest resolution (0.7 mm lateral and 7 mm
axial). Significant progress was also made in a third class of devices that is based on MEMS
mirrors or a MEMS mirror array integrated at the distal end of an endomicroscope [38-40].
While MEMS mirror based confocal endoscopes have shown potential for clinical imaging at an
early date and with good promise, there are few significant clinical instrument breakthroughs
along this direction recently due to the difficulty and cost of MEMS mirror fabrication.
4.1.2.3 Nonlinear optical (NLO) endoscope
Endomicroscopy based on NLO excitation provides complementary information to
confocal systems. NLO endomicroscopes
can better utilize molecular level contrast.
Fluorescence and second harmonic generation are efficient contrast mechanisms allowing
molecular imaging and metabolic imaging, although preliminary work is underway using gold
nanoparticles as contrast agent in reflected light confocal endomicroscopes [41]. NLO
microendoscopy
has
comparable
excitation
penetration
as
reflected
light
confocal
endomicroscopes when operated in the infrared wavelengths. For fluorescence imaging, a
confocal endomicroscope has shallower penetration depth due to the use of shorter excitation
wavelengths. Furthermore, the use of a detection aperture in confocal microendoscopy results in
significant rejection of scattered photons that can be retained in nonlinear optical
endomicroscopes allowing deeper imaging with higher signal to noise ratio in principle. Finally,
NLO endomicroscopy produces less tissue photodamage as compared to a fluorescence confocal
endomicroscope, due to the inherent localization of the excitation volume. Given these potential
advantages, a number of NLO endomicroscopes have been developed despite the greater
technical challenges in constructing these systems.
69
Endoscope imaging started with preliminary work on human subjects using laboratory
format microscopes [42]. These instruments were ultimately unsuitable for clinical work. The
most significant progress in clinical applications using laboratory-scale NLO microscopes is the
work of Konig, Kaatz, and coworkers on dermal lesion diagnosis [43]. This group of
investigators obtained, for the first time, regulatory approval to evaluate the clinical utility of
NLO microscopy in the diagnosis of skin disorders with a substantial population of patients.
The extension of these systems for clinical examination of internal organs faces three
major technical challenges. Firstly, the efficiency of second order non-linear optical excitation is
a linear function of laser pulse width. Due to non-linear optical effects, such as self-phase
modulation, that occur during transmission of high intensity light through a typical silica core
optical fiber, femtosecond pulses can be significantly broadened in a fiber endomicroscope,
which greatly decreases the excitation efficiency of these systems. Secondly, due to pulse
dispersion considerations, the successful confocal endomicroscope designs that utilize distal
scanning through a fiber optics bundle are not feasible for NLO excitation. The results from
initial experiments are not optimal [44]. Therefore, micro-scanning mechanisms must be
packaged in the proximal end of the device while the instrument outer diameter must be kept
within a few millimeters. Thirdly, fluorescence and second harmonic signals are weak.
Typically, each pixel of a video rate image has a maximum photon count between hundreds and
thousands of photons when the sample is labeled with a good fluorophore such as Rhodamine.
However, the photon count can be as few as tens to hundreds when imaging is based on tissue
endogenous fluorophores, such as NAD(P)H. Therefore, unlike a reflected light confocal system
where a strong optical signal can be obtained, a NLO endomicroscope must be designed to
maximize light collection efficiency.
Significant progress has been made in overcoming the first challenge as photonic
bandgap crystal technology is coming of age, allowing the creation of ultra-broadband,
omnireflective mirrors and waveguides [45, 46]. Photonic bandgap crystal technology is poised
to produce novel optical switches and multiplexers [47-49]. Most importantly for the future of
endomicroscopy, photonic bandgap crystal technology allows the creation of novel waveguides
for femtosecond pulses with controllable dispersion [50, 51].
Initial design of the nonlinear endomicroscope used the conventional bench top
multiphoton microscope and scanning mechanism with a long gradient index lens (GRIN) for
70
relaying the excitation light. Practically, the depth of the NLO microscopy is ultimately limited
by the scattering of the excitation light by tissues. For example, the typical penetration depth is
about 500gm in brain tissue which corresponds to about 2 to 3 scattering mean free path lengths
[52]. Rigid type endomicroscopes have been used to image deep in the tissues such as green
fluorescent protein (GFP) expressing neurons in the hippocampus of the mice brain [53, 54] and
to image the long term tumor progression in the mice colon [55] that are not easily accessible
with the conventional technique. Initial clinical studies have been conducted by Konig and his
coworkers for the deep tissue skin imaging with second harmonic generation (SHG) and
autofluorescence as contrast mechanism [56]. While these devices allow cutting-edge study of
neurobiology and tumor biology in small animals, they are not feasible for clinical human use
other than the skin due to their length limitation and their rigidity.
Significant progress has been made in overcoming the need for multiple axis scanning
mechanisms. Fiber optic based nonlinear endomicroscopes started with miniature, handheld twophoton systems using Lissajous resonant scanning of a short length single mode fiber first
developed by Helmchen and coworkers [57]. This system was used to study the vasculature
structure and dendritic calcium dynamics of a freely moving rat. Schnitzer and coworkers
developed a similar system but with much lighter weight that can be directly mountable on the
mice brain. The unique feature of this device is that the miniaturized objective lens at the tip of a
long relay lens was inserted into the mice brain and could image hippocampal vessels as well as
vessels near the neocortical surface [58, 59]. These kinds of head mountable multi-photon
endomicroscope can provide a useful tool for the neuroscientist to study the intact living
mammalian brain without constraining the motion of the mice.
More
clinically applicable
endomicroscope
started
with a number of parallel
developments, including further miniaturization of the Helmchen design by resonantly scanning
a dual core photonic crystal fiber instead of single mode fiber for more compact design of the
imaging probe by the groups of Gu and Li [60-62]. The major advance here involves the use of a
dual core fiber with single mode, low dispersion transmission of excitation light and multimode,
higher efficiency collection of the emitted signal. The addition of an axial translation capability
with shape memory alloy actuator in a recent design enabled 3D scanning NLO endomicroscope
[63]. As opposed to Li's design performs spiral scanning using tube type piezo actuator, the
recent development by Xu and coworker performs raster scanning which generates more uniform
71
illumination pattern [64]. A parallel development by Chen and coworkers utilizing 2D-scanning
MEMS mirrors has resulted in another class of NLO endomicroscopes [65, 66]. In addition to
actuation of the scanning algorithm, distal scanning endoscopes are limited by a number of other
optical constraints, such as the NA of miniaturized lenses. Recent advances in coupling spherical
lens to the tip of GRIN lens has first resulted in NLO endomicroscopes with objective numerical
apertures approaching 0.8 [67].
For compactness and simplicity, most NLO endomicroscopes uses the same optical fiber
for excitation and detection. Since the excitation and emission wavelengths in nonlinear
microscope imaging are widely separated, the endomicroscope optics optimized for excitation is
in general very suboptimal for emission detection due to chromatic aberration. In case of using
the single fiber for the excitation beam delivery and the emission beam collection, it results in
significant loss in collection efficiency. Li group have partially solved this problem by splicing
the tip of the dual core fiber with multimode fiber which maintains the single mode excitation
beam delivery and effectively increases the signal collection area. By further introducing an
effective achromatic triplet as the objective, the chromatic misalignment problem is now largely
overcome [68].
The main contrast mechanisms implemented in most NLO endomicroscopes are based on
two-photon fluorescence and second harmonic generation. A system has been recently designed
based on third harmonic generation and has been applied to study the dermal fibrosis and
hyperkeratosis [69]. In addition, an exciting new system has been developed by Ben-Yakar and
coworkers that combines imaging and microsurgery capability in the same compact system [70].
While normal NLO endomicroscopes produce a 3D image stack by scanning the focal spot in
3D, the nonlinear excitation based on the principle of spatio-temporal focusing pioneered by
Silberberg and Xu groups greatly simplifies instrumentation design by producing a depth
resolved 2D section without using any scanning mechanisms [71, 72]. Depth scanning can be
accomplished by tuning the group velocity dispersion at the proximal end as proposed and
demonstrated by Durst at el [73].
4.2 Side-view 3D point scanning two photon endomicroscope
The currently developed point scanning two photon endomicroscope is either head-on
view which cannot be directly applied to examine the side wall of the GI track or limited only for
72
(Prism
2D scanning without depth scanning capability. In this section, we introduce a novel 3D two
photon endomicroscope that can better examine the side wall of the GI track with all 3D
scanning mechanism integrated in the distal end.
4.2.1 Design of the two photon endomicroscope
(a)
Power
Control Unit
DM
DCPCF
Ti-Sa
EF
Prechirping
Unit
rmLens
PMT
(b)
Cross section of DCPCF
(c)
Fiber resonator ( X )
Prism (Y), GRIN lens (Z) scanner
(e)
Fig. 4.1 (a) Optical layout of the two photon excitation endomicroscope (b) Cross-section of the DCPCF
for the excitation and emission beam delivery. (c) Fiber resonator. The tip of DCPCF is resonated by the
fiber resonator for the beam scanning in fast (X) axis. (d) MEMS actuator oscillates the prism for slow
axis scanning (Y) and translate the GRIN lens for the depth scanning (Z) (e) Endomicroscope assembled
in the housing. DM : dichroic mirror, EF : emission filter, PMT : photomultiplier tube, DCPCF : double
clad photonic crystal fiber
Fig. 4.1(a) shows the optical design of the TPE endomicroscope system. A ultrafast
optical pulses from a Ti:Sappire laser with lOOfs pulse width, 80MHz repetition rate at the center
wavelength of 780nm (MaiTai, Spectra-Physics) was used as the excitation light source.
73
Excitation light is delivered to the distal end through the double clad photonic crystal fiber
(DCPCF) (DC- 165-16-Passive, Crystal Fibre A/S). A DCPCF has a dual core structure where the
excitation light is delivered through the single mode core and the emission light delivered back
through the multimode inner clad and the core to the photomultiplier tube (PMT). A pre-chirping
unit was used to pre-compensate the linear pulse dispersion induced by DCPCF and maximize
the two-photon excitation efficiency, which is inversely proportional to the pulse width of the
excitation beam. After propagating through the DCPCF, the excitation beam was guided to the
MEMS actuator, going through a prism and lastly a gradient index (GRIN) lens as the objective
(GT-LFRL-100-017-0.5NA, GRINTECH ). The emission from the focal spot in the sample then
traveled back through the optical train, and was guided to a photomultiplier tube (PMT) detector
via a dichroic mirror (DM).
3D volume imaging is obtained by scanning the focal point of GRIN objective in three
axes by using the fiber resonator and MEMS actuator which are all assembled in the endoscope
housing as shown in Fig. 4.1(e). The fiber resonator resonates the tip of the DCPCF at its first
resonance frequency to scan in the fast axis (X) and the prism assembled to the MEMS actuator
oscillate along the X axis guided by the rotary flexural bearing to scan in the slow axis (Y). The
combination of the fiber and prism motion generates a raster-scanning pattern. GRIN lens
assembled to the shuttle of MEMS actuator moves linearly guided by the translational flexural
bearing to scan in the depth direction (Z). The principle of MEMS actuator is based on the
thermomechanical actuation where the current flow induces the Joule heating and thus
subsequently induced the elongation of the beam. This small motion is amplified and guided by
the flexure mechanism [74]. The scanning speed can be variable depending on the length of the
fiber tip. For example, when the length of the fiber tip is 19.6mm, its first resonance frequency
and the frame rate become 700Hz and 3.5Hz, respectively. The total scanning volume of the
current design is 1 00um x 1 00um x 1 00um. The endoscope housing is designed to hold the fiber
resonator and the MEMS actuator and to allow fine adjustment for the alignment between optical
components.
74
4.2.2. Characterization of the DCPCF for the two photon excitation
4.2.2.1. Single mode excitation beam delivery through the inner core of the DCPCF
DCPCF has a dual core structure. For the multi-photon endoscope application, it has
unique feature of delivering excitation beam through its inner core and emission beam through
its core and inner clad. Thus, it can significantly simplify the design of the endoscope
structure[75]. We choose the double clad photonic crystal fiber(DCPCF) (DC-165-16-Passive,
Crystal Fibre A/S) for this application. The use of a single fiber simplifies the endomicroscope
design and makes the device more compact.
This fiber delivers single mode illumination beam
in the near infrared (NIR) spectrum and collects the visible emission light as a multimode beam.
Fig. 4.2(a) shows the far field image of the inner clad of the DCPCF and the single mode beam
from its core. The triangular shaped bright beam is the single mode beam from the core and the
honey comb structure surrounding the core is the inner clad.
As pointed out by Helmchen et al. it's important to maintain single mode beam for the
efficient two photon excitation [76]. To confirm the single mode property of the beam from the
core we measured the intensity distribution and wavefront of the beam from the core. As can be
seen in Fig. 4.2(b) the intensity profile of the beam from the core has a Gaussian distribution. In
addition, the wavefront measured with the Shack-Hartman wavefront sensor (HASO, Imagine
Optic) shows that the wavefront aberration of the beam from the core has a root mean square
(RMS) value of 0.01X as shown in Fig. 4.2(c). These measurements confirm that the beam
delivered from the core is truly a single mode beam.
(a)
(b)
(c)
froM*
from
14
he
coe,
=
RS
75
00
VI 14M00
e
-300*
r,
=
.
'MM
1
4.2.2.2. Effect of the pulse dispersion during the beam delivery through DCPCF
Typically, the pulse width of the input pulse is broadened during the propagation through
the solid core region of the fiber which is caused by the group velocity dispersion (GVD). The
common solution for this problem is to prechirp the laser beam either with grating or prism pair.
However, these methods can only compensate the linear pulse broadening due to GVD and
cannot compensate the nonlinear pulse broadening effect caused by self-phase modulation
(SPM). To characterize how the grating pair prechirping unit can compensate the linear pulse
broadening, we measured the pulse width of the single mode beam from the core with an
autocorrelator by varying the distance between the grating pair as shown in Fig. 4.4. The length
of the DCPCF is 172cm. Original pulse width of the air delivered Ti-Sapphire pulsed beam is
110 fsec. Without prechirping the pulse width is 2027fsec and we could achieve the 205fsec
when the distance between the grating pair is 48cm. From this result, the GVD parameter of
DCPCF of solid core is 234 fsec 2 /cm.
Spectrum Analyzer
Optical Autocorrelator
Pulse Stetcher or Compressor
eflectio
-
Can
Grating Pair
Pmhi
Focuser*"
*'ointor
PuReflecaaor
GraigPr
-100ft/-10nm
TEOT
phto
irscp
Purecene
F
Fiteicroe
Flip
F ie
Lrpins
R
corrorto
specdman
Multiphoton Excitation Microscope
Fig. 4.3 Experimental setup for characterizing the dispersion of DCPCF. The diagram is adapted from
Kim et al [77].
76
-
--
S1200
1000
-
-
~800------c 6000
0
20
10
30
00
40
Grating distance [cm]
Fig. 4.4 Temporal pulse width measurement by varying the distance between the grating pair
The prechirped single mode beam after the fiber is delivered to the normal two photon
microscope with 40x oil immersion objective using Fluorescein solution as a sample. Fig. 4.5
shows the TPE measured with different amount of GVD induced by the grating pair pre-chirping
unit. The slope of each curve is close to 2 demonstrating that the signal is generated by the two
photon excitation process. The intercept of the linearly interpolated curve with the y axis is
related to the TPE, which is inversely proportional to the pulse width of the excitation beam.
Two Photon Efficiency of 232uM Fluorescein
*Air
)g
_No
o)
'
___
prechirping
_A8cm
C1 7cm
0 26cm
0
---
_)0
_0
M38cm
_+50cm
N
E
10
27
2.7
Normalized Power [A.U.]
Fig. 4.5 Two photon efficiency measured with Fluorescein. The slope of each curve is close to 2, which
demonstrates that the signal is generated by the two photon excitation process. The intercept of the
linearly interpolated curve with the y axis is related to the two photon excitation efficiency, which is
inversely proportional to the pulse width of the excitation beam.
77
As shown in the Fig. 4.6, TPE is getting close to the air delivered TPE as the grating
distance increases. Due to the 3 rd order pulse dispersion effect, the prechirping unit cannot
restore the temporal pulse width to the air delivered value. At the optimum prechirping distance
of 48cm we could measure a factor of 2 lower TPE compared with that of the air delivery.
30
25- -----
25
WE
------
- 20---
-
15 10-
*-e-
------------ i---- -+----
L -+ ----
- --
--
-
Pulse Width
TP E
-
0
0
10
20
30
40
50
Grating distance [cm]
Fig. 4.6 Ratio plot of the TPE and the pulse width by varying the distance between the grating pair. 0cm
grating distance means no prechirping.TPE ratio = TPEtjr/ Photon Count DCPCF , Pulse Width ratio =
FWHMDCPCF/ FWHMAr
4.2.3 Imaging experiments and preliminary results
Custom-developed data acquisition software and control electronics were constructed to
synchronize the different scanning axes with the photon counting circuitry monitoring the optical
signal from the photomultiplier tube. To demonstrate 3D optically sectioned imaging capability
of the TPE endomicroscope system a stack of 15 micron fluorescent beads was used for
preliminary imaging experiments. Fig. 4.7 shows the bead cross-sectional image after correcting
the artifact induced by the resonance scanning scheme [78]. The input current is the half the
value for the full motion range and the measured field of view is 50umx6Oum.
Fig. 4.7 A stack of fluorescent beads images with a field of view of 50 x 60 microns. Each bead is of 15
micron in diameter and each frame is 3 micron apart in Z-axis. The images is acquired at 3.5 frame/sec
78
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
N. J. Agnantis, A. Sotiriadis, and E. Paraskevaidis, "The current status of HPV DNA
testing," Eur J Gynaecol Oncol 24, 351-356 (2003).
B. S. Apgar and G. Brotzman, "Management of cervical cytologic abnormalities," Am Fam
Physician 70, 1905-1916 (2004).
G. De Palo, "Cervical precancer and cancer, past, present and future," Eur. J. Gynaecol.
Oncol. 25, 269-278 (2004).
R. Dunleavey, "Incidence, pathophysiology and treatment of cervical cancer," Nurs. Times
100, 38-41 (2004).
S. Gupta and P. Sodhani, "Why is high grade squamous intraepithelial neoplasia underdiagnosed on cytology in a quarter of cases? Analysis of smear characteristics in discrepant
cases," Indian J. Cancer 41, 104-108 (2004).
N. C. Lambrou and L. B. Twiggs, "High-grade squamous intraepithelial lesions," Cancer J 9,
382-389 (2003).
G. W. Falk, "Barrett's esophagus," Gastroenterology 122, 1569-1591 (2002).
S. A. Faruqi, V. Arantes, and M. S. Bhutani, "Barrett's esophagus: current and future role of
endosonography and optical coherence tomography," Dis Esophagus 17, 118-123 (2004).
M. I. Golding, D. B. Doman, and H. J. Goldberg, "Take your Pill(Cam): it might save your
life," Gastrointest Endosc 62, 196-198 (2005).
W. A. Qureshi, "Current and future applications of the capsule camera," Nat. Rev. Drug
Discov. 3, 447-450 (2004).
A. R. Eliakim, "Video capsule endoscopy of the small bowel (PillCam SB)," Curr Opin
Gastroen 22, 124-127 (2006).
Sayaka, "RF Co., Ltd", retrieved http://www.rfsystemlab.com/en/sayaka/.
R. S. Dacosta, B. C. Wilson, and N. E. Marcon, "New optical technologies for earlier
endoscopic diagnosis of premalignant gastrointestinal lesions," J Gastroen Hepatol 17, S85S104 (2002).
T. J. Muldoon, D. Roblyer, M. D. Williams, V. M. T. Stepanek, R. Richards-Kortum, and A.
M. Gillenwater, "Noninvasive imaging of oral neoplasia with a high-resolution fiber-optic
microendoscope," Head Neck-J Sci Spec 34, 305-312 (2012).
P. Sharma, "Review article: emerging techniques for screening and surveillance in Barrett's
oesophagus," Aliment. Pharmacol. Ther. 20 Suppl 5, 63-70; discussion 95-66 (2004).
J. M. Poneros, "Diagnosis of Barrett's esophagus using optical coherence tomography,"
Gastrointest. Endosc. Clin. N. Am. 14, 573-588, x (2004).
J. M. Poneros and N. S. Nishioka, "Diagnosis of Barrett's esophagus using optical coherence
tomography," Gastrointest. Endosc. Clin. N. Am. 13, 309-323 (2003).
N. S. Nishioka, "Optical biopsy using tissue spectroscopy and optical coherence
tomography," Can J Gastroenterol 17, 376-380 (2003).
B. C. Jacobson and J. Van Dam, "Enhanced endoscopy in inflammatory bowel disease,"
Gastrointest Endosc Clin N Am 12, 573-587 (2002).
G. Zuccaro, N. Gladkova, J. Vargo, F. Feldchtein, E. Zagaynova, D. Conwell, G. Falk, J.
Goldblum, J. Dumot, J. Ponsky, G. Gelikonov, B. Davros, E. Donchenko, and J. Richter,
"Optical coherence tomography of the esophagus and proximal stomach in health and
disease," Am J Gastroenterol 96, 2633-2639 (2001).
79
21. C. Pitris, C. Jesser, S. A. Boppart, D. Stamper, M. E. Brezinski, and J. G. Fujimoto,
"Feasibility of optical coherence tomography for high-resolution imaging of human
gastrointestinal tract malignancies," J Gastroenterol 35, 87-92 (2000).
22. X. D. Li, S. A. Boppart, J. Van Dam, H. Mashimo, M. Mutinga, W. Drexler, M. Klein, C.
Pitris, M. L. Krinsky, M. E. Brezinski, and J. G. Fujimoto, "Optical coherence tomography:
advanced technology for the endoscopic imaging of Barrett's esophagus," Endoscopy 32,
921-930 (2000).
23. S. Brand, J. M. Poneros, B. E. Bouma, G. J. Tearney, C. C. Compton, and N. S. Nishioka,
"Optical coherence tomography in the gastrointestinal tract," Endoscopy 32, 796-803 (2000).
24. T. Asano and R. S. McLeod, "Dietary fibre for the prevention of colorectal adenomas and
carcinomas," Cochrane Database Syst Rev, CD003430 (2002).
25. A. H. Chau, R. C. Chan, M. Shishkov, B. MacNeill, N. Iftimia, G. J. Teamey, R. D. Kamm,
B. E. Bouma, and M. R. Kaazempur-Mofrad, "Mechanical analysis of atherosclerotic
plaques based on optical coherence tomography," Ann Biomed Eng 32, 1494-1503 (2004).
26. G. J. Tearney, H. Yabushita, S. L. Houser, H. T. Aretz, I. K. Jang, K. H. Schlendorf, C. R.
Kauffman, M. Shishkov, E. F. Halpern, and B. E. Bouma, "Quantification of macrophage
content in atherosclerotic plaques by optical coherence tomography," Circulation 107, 113119 (2003).
27. J. L. Bouma, L. R. Aronson, D. G. Keith, and H. M. Saunders, "Use of computed
tomography renal angiography for screening feline renal transplant donors," Vet Radiol
Ultrasound 44, 636-641 (2003).
28. B. E. Bouma, G. J. Tearney, H. Yabushita, M. Shishkov, C. R. Kauffman, D. DeJoseph
Gauthier, B. D. MacNeill, S. L. Houser, H. T. Aretz, E. F. Halpern, and I. K. Jang,
"Evaluation of intracoronary stenting by intravascular optical coherence tomography," 89,
317-320 (2003).
29. H. Yabushita, B. E. Bouma, S. L. Houser, H. T. Aretz, I. K. Jang, K. H. Schlendorf, C. R.
Kauffman, M. Shishkov, D. H. Kang, E. F. Halpern, and G. J. Tearney, "Characterization of
human atherosclerosis by optical coherence tomography," Circulation 106, 1640-1645
(2002).
30. J. G. Fujimoto, S. A. Boppart, G. J. Tearney, B. E. Bouma, C. Pitris, and M. E. Brezinski,
"High resolution in vivo intra-arterial imaging with optical coherence tomography," Heart
82, 128-133 (1999).
31. D. C. Adler, Y. Chen, R. Huber, J. Schmitt, J. Connolly, and J. G. Fujimoto, "Threedimensional endomicroscopy using optical coherence tomography," Nat Photon 1, 709-716
(2007).
32. N. Q. Nguyen and R. W. L. Leong, "Current application of confocal endomicroscopy in
gastrointestinal disorders," J Gastroen Hepatol 23, 1483-1491 (2008).
33. C. Liang, K. B. Sung, R. R. Richards-Kortum, and M. R. Descour, "Design of a highnumerical-aperture miniature microscope objective for an endoscopic fiber confocal
reflectance microscope," Appl Optics 41, 4603-4610 (2002).
34. A. R. Rouse, A. Kano, J. A. Udovich, S. M. Kroto, and A. F. Gmitro, "Design and
demonstration of a miniature catheter for a confocal microendoscope," Appl Optics 43,
5763-5771 (2004).
35. M. Sakashita, H. Inoue, H. Kashida, J. Tanaka, J. Y. Cho, H. Satodate, E. Hidaka, T.
Yoshida, N. Fukami, Y. Tamegai, A. Shiokawa, and S. Kudo, "Virtual histology of
80
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
colorectal lesions using laser-scanning confocal microscopy," Endoscopy 35, 1033-1038
(2003).
T. F. Watson, M. A. A. Neil, R. Juskaitis, R. J. Cook, and T. Wilson, "Video-rate confocal
endoscopy," J Microsc-Oxford 207, 3 7-42 (2002).
R. Kiesslich, J. Burg, M. Vieth, J. Gnaendiger, M. Enders, P. Delaney, A. Polglase, W.
McLaren, D. Janell, S. Thomas, B. Nafe, P. R. Galle, and M. F. Neurath, "Confocal laser
endoscopy for diagnosing intraepithelial neoplasias and colorectal cancer in vivo,"
Gastroenterology 127, 706-713 (2004).
D. L. Dickensheets and G. S. Kino, "Micromachined scanning confocal optical microscope,"
Optics Letters 21, 764-766 (1996).
P. A. Himmer, D. L. Dickensheets, and R. A. Friholm, "Micromachined silicon nitride
deformable mirrors for focus control," Optics Letters 28, 1280-1282 (2001).
D. M. Rector, D. M. Ranken, and J. S. George, "High-performance confocal system for
microscopic or endoscopic applications," Methods 30, 16-27 (2003).
K. Sokolov, M. Follen, J. Aaron, I. Pavlova, A. Malpica, R. Lotan, and R. Richards-Kortum,
"Real-time vital optical imaging of precancer using anti-epidermal growth factor receptor
antibodies conjugated to gold nanoparticles," Cancer Res. 63, 1999-2004 (2003).
B. R. Masters, P. T. C. So, and E. Gratton, "Multiphoton excitation fluorescence microscopy
and spectroscopy of in vivo human skin," Biophys J 72, 2405-2412 (1997).
K. Konig, M. Speicher, R. Buckle, J. Reckfort, G. McKenzie, J. Welzel, M. J. Koehler, P.
Elsner, and M. Kaatz, "Clinical optical coherence tomography combined with multiphoton
tomography of patients with skin diseases," J Biophotonics 2, 389-397 (2009).
W. Gobel, J. N. Kerr, A. Nimmerjahn, and F. Helmchen, "Miniaturized two-photon
microscope based on a flexible coherent fiber bundle and a gradient-index lens objective,"
Opt Lett 29, 2521-2523 (2004).
S. D. Hart, G. R. Maskaly, B. Temelkuran, P. H. Prideaux, J. D. Joannopoulos, and Y. Fink,
"External reflection from omnidirectional dielectric mirror fibers," Science 296, 510-513
(2002).
B. Temelkuran, S. D. Hart, G. Benoit, J. D. Joannopoulos, and Y. Fink, "Wavelengthscalable hollow optical fibres with large photonic bandgaps for C02 laser transmission,"
Nature 420, 650-653 (2002).
A. E. Attard, "Optical bandwidth in coupling: the multicore photonic switch," Appl Opt 42,
2665-2673 (2003).
T. A. Ibrahim, K. Amarnath, L. C. Kuo, R. Grover, V. Van, and P. T. Ho, "Photonic logic
NOR gate based on two symmetric microring resonators," 29, 2779-2781 (2004).
M. Soljacic, M. Ibanescu, S. G. Johnson, Y. Fink, and J. D. Joannopoulos, "Optimal bistable
switching in nonlinear photonic crystals," Phys Rev E Stat Nonlin Soft Matter Phys 66,
055601 (2002).
G. McConnell and E. Riis, "Two-photon laser scanning fluorescence microscopy using
photonic crystal fiber," J Biomed Opt 9, 922-927 (2004).
M. T. Myaing, J. Y. Ye, T. B. Norris, T. Thomas, J. R. Baker, Jr., W. J. Wadsworth, G.
Bouwmans, J. C. Knight, and P. S. Russell, "Enhanced two-photon biosensing with doubleclad photonic crystal fibers," Opt Lett 28, 1224-1226 (2003).
P. Theer and W. Denk, "On the fundamental imaging-depth limit in two-photon
microscopy," J Opt Soc Am A 23, 3139-3149 (2006).
J. C. Jung and M. J. Schnitzer, "Multiphoton endoscopy," Opt Lett 28, 902-904 (2003).
81
54. M. J. Levene, D. A. Dombeck, K. A. Kasischke, R. P. Molloy, and W. W. Webb, "In vivo
multiphoton microscopy of deep brain tissue," Journal of Neurophysiology 91, 1908-1912
(2004).
55. P. Kim, E. Chung, H. Yamashita, K. E. Hung, A. Mizoguchi, R. Kucherlapati, D. Fukumura,
R. K. Jain, and S. H. Yun, "In vivo wide-area cellular imaging by side-view
endomicroscopy," Nat Methods 7, 303-305 (2010).
56. K. Konig, A. Ehlers, I. Riemann, S. Schenkl, R. Buckle, and M. Kaatz, "Clinical two-photon
microendoscopy," Microsc Res Techniq 70, 398-402 (2007).
57. F. Helmchen, M. S. Fee, D. W. Tank, and W. Denk, "A miniature head-mounted two-photon
microscope: High-resolution brain imaging in freely moving animals," Neuron 31, 903-912
(2001).
58. B. A. Flusberg, J. C. Lung, E. D. Cocker, E. P. Anderson, and M. J. Schnitzer, "In vivo brain
imaging using a portable 3.9 gram two-photon fluorescence microendoscope," Optics Letters
30, 2272-2274 (2005).
59. G. F. Karl Deisseroth, Ania K. Majewska, Gero Miesenbock, Alice Ting, Mark J. Schnitzer,
"Next-Generation Optical Technologies for Illuminating Genetically Targeted Brain
Circuits," The Journal of Neuroscience 26, 10380-10386 (2006).
60. H. C. Bao, J. Allen, R. Pattie, R. Vance, and M. Gu, "Fast handheld two-photon fluorescence
microendoscope with a 475 mu m X 475 mu m field of view for in vivo imaging," Opt. Lett.
33, 1333-1335 (2008).
61. M. T. Myaing, D. J. MacDonald, and X. D. Li, "Fiber-optic scanning two-photon
fluorescence endoscope," Opt Lett 31, 1076-1078 (2006).
62. Y. C. Wu, Y. X. Leng, J. F. Xi, and X. D. Li, "Scanning all-fiber-optic endomicroscopy
system for 3D nonlinear optical imaging of biological tissues," Opt Express 17, 7907-7915
(2009).
63. Y. C. Wu, Y. Y. Zhang, J. F. Xi, M. J. Li, and X. D. Li, "Fiber-optic nonlinear
endomicroscopy with focus scanning by using shape memory alloy actuation," J Biomed
Opt 15(2010).
64. D. R. Rivera, C. M. Brown, D. G. Ouzounov, I. Pavlova, D. Kobat, W. W. Webb, and C.
Xu, "Compact and flexible raster scanning multiphoton endoscope capable of imaging
unstained tissue," P. Natl. Acad. Sci. U.S.A. 108, 17598-17603 (2011).
65. W. Y. Jung, S. Tang, D. T. McCormic, T. Q. Xie, Y. C. Ahn, J. P. Su, I. V. Tomov, T. B.
Krasieva, B. J. Tromberg, and Z. P. Chen, "Miniaturized probe based on a
microelectromechanical system mirror for multiphoton microscopy," Opt Lett 33, 13241326 (2008).
66. S. Tang, W. G. Jung, D. McCormick, T. Q. Xie, J. P. Su, Y. C. Ahn, B. J. Tromberg, and Z.
P. Chen, "Design and implementation of fiber-based multiphoton endoscopy with
microelectromechanical systems scanning," J Biomed Opt 14(2009).
67. R. P. J. Barretto, B. Messerschmidt, and M. J. Schnitzer, "In vivo fluorescence imaging with
high-resolution microlenses," Nat Methods 6, 51 1-U561 (2009).
68. Y. C. Wu, J. F. Xi, M. J. Cobb, and X. D. Li, "Scanning fiber-optic nonlinear
endomicroscopy with miniature aspherical compound lens and multimode fiber collector,"
Opt. Lett. 34, 953-955 (2009).
69. J. H. Lee, S. Y. Chen, C. H. Yu, S. W. Chu, L. F. Wang, C. K. Sun, and B. L. Chiang,
"Noninvasive in vitro and in vivo assessment of epidermal hyperkeratosis and dermal
fibrosis in atopic dermatitis," J Biomed Opt 14(2009).
82
70. C. L. Hoy, N. J. Durr, P. Y. Chen, W. Piyawattanametha, H. Ra, 0. Solgaard, and A. BenYakar, "Miniaturized probe for femtosecond laser microsurgery and two-photon imaging,"
Opt Express 16, 9996-10005 (2008).
71. D. Oron, E. Tal, and Y. Silberberg, "Scanningless depth-resolved microscopy," Opt. Express
13, 1468-1476 (2005).
72. G. H. Zhu, J. van Howe, M. Durst, W. Zipfel, and C. Xu, "Simultaneous spatial and
temporal focusing of femtosecond pulses," Opt. Express 13, 2153-2159 (2005).
73. M. E. Durst, G. H. Zhu, and C. Xu, "Simultaneous spatial and temporal focusing for axial
scanning," Opt. Express 14, 12243-12254 (2006).
74. S. C. Chen and M. L. Culpepper, "Design of contoured microscale thermomechanical
actuators," Journal of Microelectromechanical Systems 15, 1226-1234 (2006).
75. L. Fu, X. S. Gan, and M. Gu, "Nonlinear optical microscopy based on double-clad photonic
crystal fibers," Optics Express 13, 5528-5534 (2005).
76. F. Helmchen, D. W. Tank, and W. Denk, "Enhanced Two-Photon Excitation Through
Optical Fiber by Single-Mode Propagation in a Large Core," Applied Optics 41, 2930-2934
(2002).
77. D. Kim, H. Choi, S. Yazdanfar, and P. T. C. So, "Ultrafast Optical Pulse Delivery With
Fibers for Nonlinear Microscopy," Microsc. Res. Techniq. 71, 887-896 (2008).
78. M. J. Sanderson, "Acquisition of Multiple Real-Time Images for Laser Scanning
Microscopy," Microscopy and Analysis 18, 17-23 (2004).
83
This page is intentionally left blank
84
Chapter 5
Improving femtosecond laser pulse delivery through a
hollow core photonic crystal fiber for temporally focused
two-photon endomicroscopy
5.1 Introduction
In the last two decades, a number of new biomedical imaging technologies have emerged
that fundamentally changed the procedures of disease diagnosis and expanded the frontier of the
biological research that were not accessible previously. For example, two photon microscopy has
now become an indispensible tool for the study of the brain function and disease based on the
morphological map of the neuronal network [1]. Optical coherence tomography is routinely used
in the diagnosis of retinal diseases such as glaucoma and age related macular degeneration [2].
More recently, photoacoustic tomography has shown its great potential in in-vivo monitoring of
the tumor angiogenesis [3].
Among the other imaging modalities, multiphoton microscopy can provide both
morphological and spectroscopic information on the state of the disease from the endogenous
fluorophores in tissue. For example, cancer cells show the distinctive morphological changes in
their carcinogenic progression such as the increase of nuclear to cytoplasmic ratio. In addition,
the decrease in the redox ratio of the metabolic coenzyme FAD and NADH has been observed [4,
5].
Multiphoton microscopy can also utilize second harmonic generation (SHG) to monitor
collagen state in the extracellular matrix. Recent study has shown that SHG can be useful
diagnostic marker for the muscular dystrophy and ovarian cancer [6]. In addition, we have also
recently shown that SHG from abnormally increased collagen fiber content is correlated with
liver fibrosis progression [7]. For these reasons, multiphoton microscopy in endoscopic format
has great potential as an optical biopsy tool for diagnosing diseases state such as colorectal,
esophageal, and cervical cancer which occurs on epithelial surface of interior body cavities that
cannot be reached without miniaturizing the physical size of the device.
85
Although multiphoton endomicroscopy is a promising technology, it is technically
challenging for miniaturizing the device in that it requires the 3D beam scanning mechanism for
the volume imaging at the distal end of the optical fiber that are used to deliver the excitation
beam to the remote site [8-12]. Also, a point scanning makes the image acquisition process
inherently slow. These problems can be circumvented by utilizing the temporally focused
widefield two photon microscopy (TFM) that can greatly reduce instrument complexity and offer
the potential for higher speed imaging based on parallelization by producing a depth resolved
images without the need for scanning mechanisms at the distal end [13, 14]. Axial scanning can
also be performed remotely by chirping excitation pulses in the proximal end [15]. However, it
has been demonstrated that high peak power pulses from the regenerative amplifier is required to
take full advantage of this widefield imaging method [16]. The use of high energy pulses poses a
great challenge in the delivery of the excitation beam since the fiber can be easily damaged by
the extremely high intensity focused beam at the inlet of the fiber.
In this paper, we propose a strategy, based on manipulating laser pulse energy and
repetition rate with a compact pulse splitter and laser group velocity dispersion with a pulse
compressor, for delivering femtosecond, high energy pulses from a regeneratively amplified Tisapphire laser through a hollow core photonic crystal fiber (HCPCF). We demonstrate that the
excitation efficiency of a temporally focused widefield two photon endomicroscope (TFEM) can
be substantially improved.
5.2 Improving the two photon excited fluorescence signal using pulse splitter
Light guidance by the standard step index fiber is based on the total internal reflection
with core having higher refractive index than the surrounding cladding. However, this solid core
fiber is not optimum for delivering ultrafast high peak power pulses because of the distortion of
the temporal shape of the input pulses induced by the intense nonlinear interaction of the core
material with the high peak power pulses such as self-phase modulation and group velocity
dispersion (GVD). On the contrary, light guidance of a HCPCF is based on the photonic band
gap effect where the light of the certain wavelength band is trapped inside the hollow fiber core
surrounded by the periodic wavelength scale lattice of air holes in the cladding [17]. Since the
light propagates through the air, the temporal distortion of the input pulse is minimized [18].
86
Because of this unique properties, HCPCF have been utilized for delivering high peak power
pulses for various applications including two photon fluorescence microscopy [19], laser
micromachining [20] and laser surgery [21].
While the two photon laser scanning microscopy (TPLSM) requires 0.01-0.1nJ pulse
energy from a Ti-sapphire laser for the optimum excitation efficiency, TFM requires 104 - 105
times higher pulse energy to optimally excite the whole field of view. Considering the power loss
during beam delivery through the optical fiber and the intermediate optical elements, even the
full power from Ti-sapphire laser cannot provide sufficient peak power for the optimum two
photon excitation at the sample. Although it has been demonstrated that the high peak power
from the regeneratively amplified Ti-sapphire laser could be used for recording even the
Brownian motion of the fluorescent bead in a TFM, it cannot be directly applied in the TFEM
since such a high peak power pulse could damage the fiber either by the thermal or the ionization
effect.
Fig. 5.1 shows a flat cleaved surface of HCPCF (HC-800-01, NKT photonics) and the
damaged surface by the high peak power pulses from the regeneratively amplified Ti-sapphire
laser at 800nm wavelength with 130 fsec pulse width and 10 kHz repetition rate (Legend Elite,
Coherent). The damage of HCPCF mostly occurs at the inlet of the fiber when the focused beam
spot size and shape does not match with the mode field diameter of HCPCF. The maximum
transmittable power is closely related to the coupling efficiency. If the fiber coupling is not
performed at its optimum condition, the fiber is damaged before it reaches the absolute
maximum value set by the damage threshold.
(a)
Mb
(M
Fig. 5.1 Far field image of the end surface of HC-800-01 fiber (NKT photonics) (a) clean surface (b)
damaged surface. These images are acquired by illuminating the proximal end of the fiber with a white
is
light LED and imaging the distal end with a 40x air objective and a CCD camera. The bright dot shown
total
via
structures
these
the photonic lattice structure where the light from LED propagate through
internal reflection (c) intensity pattern of the output beam from the hollow core fiber.
87
There are several factors that determines the coupling efficiency such as the flatness of
the cleaved surface, the input beam diameter, the numerical aperture (NA) of the coupling lens,
NA of the fiber and the size of the mode diameter of the fiber. The rule of thumb of maximizing
the coupling efficiency is that the NA of the focused beam should be less than the NA of the
fiber. At the same time, the focused beam spot size should be less than the size of the core
diameter. The HCPCF (HC-800-01, NKT photonics) used for delivering the pulses from the
regenerative amplifier has NA of 0.2, a mode field diameter (1/e
2
width) of 8.8pm and the zero
dispersion wavelength near 805nm.
The damage threshold of the fiber is measured by increasing the input pulse energy
to
the fiber at every 20 nJ until the damage occurs in the fiber. The pulse energy from the
regenerative amplifier is controlled by using a half wave plate (1ORP02-800, Newport) and a
polarizer (GL15-B, Thorlabs). First, the output beam diameter from the regenerative amplifier is
reduced to the aperture size of the coupling lens using the telescope and then it is coupled to the
fiber using the aspheric lens with NA of 0.16 (C260TME-B, Thorlabs). The optimum coupling
lens has been selected after trying several coupling lens with NA varying from 0.15 to 0.25. For
0.16 NA coupling lens, 90% coupling efficiency have been achieved. The damage in the fiber
occurs sequentially. First, the constant coupling efficiency is maintained up to 360 nJ input pulse
energy and then starts to drop slowly as cracks are started to appear in the core. Subsequently,
the power drops suddenly with the total destruction of the core as shown in Fig. 5.2. The fiber
damage by the ionization effect can be avoided by coupling in and out of the fiber within the
vacuum chamber. In this condition, high energy femtosecond pulses of up to 1.8 pJ was
transmitted through HCPCF without fiber damage [22]. However, this strategy is difficult for
endoscopic application especially at the distal end.
88
HC800-01 coupling without splitter
5
-00- Measured
400
-
90% coupling
C 300
2000.
1000
0.I
0
200
100
500
400
300
Input Pulse Energy [nJ]
Fig. 5.2 Coupling efficiency plot of HC800-01 fiber. The length of fiber is 80cm.
For a given FOV, there is a required pulse energy to optimize excitation efficiency.
However, this pulse energy often cannot be used due to the lower damage threshold of the fiber.
To better utilize the remaining pulse energy available from the regenerative amplifier, it is better
to split each regenerative amplifier pulse into multiple pulses instead of just attenuating the laser.
This approach increasees the number of available pulses for two photon excitation with pulse
energy less than the damage threshold of the fiber. Theoretically, two photon absorption (TPA)
per fluorophore per second can be expressed as Eq. (5.1).
2
TPA = 3()2
1
2
"hc' FOV 2Pe
krf=S()
1
hc FOV2
edsef
,
(5.1)
where 6 is TPA cross-section, A is the wavelength, c is the speed of light, h is the Plank constant,
Ppeak
is the peak power of pulse, ep,,s, is the pulse energy,
f
is the pulse repetition rate, T is the
pulse width and FOV is the illumination field of view [1]. Eq.(5.1) implies that the signal in
TFEM can be maximized by setting the pulse energy (peak power) below the fiber damage
threshold and by increasing the number of pulses as many as possible. In a sense, this is similar
to an optimization problem where the objective function is TPA and the optimization parameters
are pulse energy and repetition rate with the fiber damage threshold as a constraint. Our solution
is to implement a pulse splitter that multiplies the incoming pulse to many identical pulses while
keeping the pulse energy below the damage threshold.
89
Fig. 5.3 shows the pulse splitter that multiplies an incoming pulse to 64 identical pulses
and the timing diagram showing the pulses at the intermediate steps. The detail design is adapted
from the work by Ji et al [23]. To summarize the sequence of pulse multiplication steps, the
pulse is split into two by non-polarizing beam splitter (BS 1 ). One of them goes to PS1 directly
and the other is delayed by delay line (11) and goes to the other input port of PS 1 . Each input
pulse to PS1 is multiplied to 4 pulses by the 4x pulse splitter where the two pulses from each
output ports are time delayed by 74 ps. Pulses from one of the output ports of PSI is delayed by 12
and recombined with pulses from the other output ports of PSI at BS 2 . BS 2 copies these pulses to
two sets of same pulses. One group of pulses goes directly to PS 2 and the other group of pulses
go to DL which add about 2.5ns time delay before entering the other input port of PS 2 . Each
input pulse to PS 2 is multiplied to 4 pulses by the 4x pulse splitter where the two pulses from
each output ports are time delayed by 37 ps. Pulses from one of the output ports of PS 2 is delayed
by 13 and recombined with pulses from the other output ports of PS 2 using half-wave plate (HWP)
and polarizing beam cube splitter (BS 3 ). The time delayed pulses in PS 1 and PS 2 are produced by
splitting the input pulses at the interface of the two different refractive index materials so that
each pulse travels different optical path length and recombined at the beam splitting interface in
such a way that the recombined pulses from the output ports of pulse splitter propagate
collinearly.
After pulse splitting, the pulses are prechirped by the single prism type pulse
compressor to compensate for the group velocity dispersion that may be induced by the
intermediate optics and the fiber. Since the separation between the pulses are much smaller than
the lifetime of the fluorophores which is typically in nanoseconds range, the benefit of increasing
the repetition rate can only be meaningful when two photon excitation efficiency is low enough
so that the ground state depletion does not occur as in many cases during temporally focused
wide field two photon imaging. In addition, increasing the repetition rate while decreasing the
pulse intensity quadratically can sometimes reduce photodamage and photobleaching while
maintaining the same signal level [23].
90
PS
HWP
852
PS
PS 2
BS'
BI
10kHz 00nm
Regenerative
Amplifier
DL
S
FCL
BE
/
r lviV
M
ccM<
HCPCF
Probe
(
p
Endoscope
FM
Input pulse
I
At PS
PSIOuti
PSI Out 2
AtBSA2
PS2Outi 111
Jill
PS20Out 2
LL
AtB13
11
JilA
1111
11
WI
IA II
1111
1A
l
11AA
IAI
1111
IAI
1111
At PS2
IAI
1AAA
hll 1111 Jil
h Il
l
illAi11 Jil AAA
111A ilAI
JiII
II
III
Dlayby 12
Delay by 11
I A
Jll
74 ps 37 ps
Delay by I
eiay uy UL
Fig. 5.3 Schematic diagram of the 64x pulse splitter and the timing diagram of the pulse multiplication at
each step. After the pulse splitter, the beam goes through the single prism pulse compressor to
compensate the group velocity dispersion induced by HCPCF. The 4x pulse splitter diagram in the box is
adapted from [23], BE : beam expander, FM: flip mirror, BS, 2 : non-polarizing beam cube splitter, BS 3 :
polarizing beam cube splitter , PSI: pulse splitter with 74 ps pulse separation, PS 2 : pulse splitter with 37
ps pulse separation, DL : delay line, HWP: half wave plate, P: prism, CCM: corner cube mirror, FCL:
fiber coupling lens, HCPCF: hollow core photonic crystal fiber
5.3 Increase the damage threshold by stretching the pulse
The damage of HCPCF typically occurs at the inlet of the fiber where the lattice structure
of the air hole and silica surrounding the core is fractured by the strong reaction of the ionized air
molecules with the silica in the cladding generated by the extremely high peak power focused
beam. Thus, reducing peak power by stretching the pulse would allow more power to be
transmitted before damaging the fiber. Although this method has not been tried for femtosecond
pulses, it has already been applied in delivering the high energy nanosecond pulse [24].
91
Pulse width can be controlled by adding negative group delay dispersion (GDD) using a
pulse compressor. As shown in Fig. 5.4, the amount of GDD induced by the pulse compressor is
linearly proportional to the distance between the prism and the corner cube mirror (CCM). The
pulse width is measured with the single shot autocorrelator (Delta single shot autocorrelator,
Minioptic Technology). From the measured pulse width, GDD can be calculated assuming that
the shape of the input pulse is in a transform limited Gaussian form using Eq.(5.2) [25]. Input
pulse width was 130 fsec2 .
GDD
- "__
2
41n2
-2
(a)
[fsec 2]
(5.2)
(b)
+ Pulse Width [fsec]
+GDD (fsec21
1000-
Goo-
-- 000
30
20010
15
20 2
30 35 40
CCM position [cm]
45
50
Fig. 5.4 (a) Schematic drawing of single prism pulse compressor adapted from [26]. Large tuning range of
GDD is achieved by using a prism made of highly dispersive glass, SF66. GDD is tuned by changing the
distance between the prism and the corner cube mirror. (b) Measured pulse width as a function of the
corner cube mirror position. (blue curve). GDD is calculated using Eq.(5.2) from the measured pulse
width. Input pulse width was 130 fsec2 .
Damage threshold of HCPCF (HC-800-02, NKT photonics) is measured with the
configuration shown in Fig. 5.3. HC-800-02 has a NA of 0.2, a mode field diameter (1/e
2
width)
of 5.5 ptm and the zero dispersion wavelength near 778nm. Damage threshold is measured first
without pulse stretching and then at four different CCM positions each at 20 cm, 30 cm, 40 cm,
50 cm. The pulse width at these conditions correspond to 150 fsec, 187 fsec, 359 fsec, 632 fsec,
922 fsec respectively. At each measurement conditions, the input power is increased at 5mW
increment until the coupling efficiency starts to degrade from the initial value more than 10%.
The damage threshold is defined as the power level at which the coupling efficiency drops 10%
from that of the non-damaged fiber. At each conditions, the measurements were repeated five
92
times. Typical coupling efficiency was 80% which is lower than that without the pulse
compressor because the beam shape is slightly elongated horizontally while the beam propagate
through the pulse compressor. According to Fig. 5.5, the damage threshold increased about
factor of two while the input pulse width is increased or the peak power is decreased about a
factor of six. The damage threshold does not linearly increase with an increase in pulse width or
a decrease in peak power. This may be caused by the thermal stress built up by the pulses with
picosecond time separation whereas the time constant for the thermal dissipation of HCPCF is
nanosecond range. Currently, the input pulses from the 10 kHz regenerative amplifier are
separated by 100 msec time delay and this single input pulse is copied to 64 pulses with 37 psec
time delay. This thermal stress can be alleviated by separating the pulses more uniformly in
between the pulses of the 10 kHz pulses. However, this would increase the overall footprint of
the pulse splitter.
,
100-
68
0
.
.. .. .... .....
.
.........
...... ....
02;0
40
6;0
8;0
100
Input Pulse Width [fsec]
Fig. 5.5 Damage threshold of HC-800-02 (NKT photonics) measured without pulse stretching (150 fsec)
and CCM positions at 20 cm (187 fsec ), 30 cm (359 fsec), 40 cm (632 fsec), 50 cm (922 fsec).
Although stretching pulse may allow more power to be delivered through the fiber, it
decreases the two photon absorption as can be predicted by Eq.(5.1). Ideally, the pulse is first
stretched with the negative dispersion provided by the pulse compressor to reduce the ionization
effect and then the pulse width is recovered to the transform-limited pulse width with the
positive dispersion provided by HCPCF. For 800nm working wavelength, HC-800-01 has a
positive GDD and HC-800-02 has a negative GDD. Since the prism type pulse compressor can
add a large negative GDD, HC-800-01 model is better suited for the currently proposed pulse
delivery scheme. Once the beam is coupled to HCPCF, over 95% of the optical power is
93
confined in the air core. Therefore, the material dispersion is minimal and the waveguide
dispersion is the dominant factor for the group velocity dispersion induced in the HCPCF [27].
GDD of HCPCF (HC-800-01) is calculated from the pulse widths measured before and after the
fiber. The input pulse width to the fiber was 130fsec and the pulse width after propagating 80cm
long fiber was 200fsec. From this, GDD can be calculated to be 7000 fsec2 . The negative GDD
of the fiber can be cancelled out by the same amount of positive GDD of the pulse compressor
which can be obtained when CCM of pulse compressor is positioned at 20cm as shown in Fig.
5.6.
8000
6000
4.
S400-
~20O-00
0
OS, .0
00
,
0
5
,
,
,
,
,
,
I
,
10 15 20 25 30 35 40 4s 50
CCM position [cm]
Fig. 5.6 Pulse width measured after the fiber (HC-800-01) for various amount of negative GDD from the
pulse compressor. CCM at 0cm corresponds to the pulse width measured without the pulse compressor.
The minimal pulse width after the fiber is achieved when CCM is at 20cm position.
We combine both ideas of pulse splitting and pulse stretching to improve the high peak
power pulse delivery through the fiber. After 64x pulse multiplication, the damage threshold
increased to 60mW. After the pulse stretching it further increased to 75mW. The reason the
damage threshold does not increase 64 times more compared to that of without pulse
multiplication (4mW) could be the thermal effect as explained in Fig. 5.5. Since the pulse width
increases to 187fsec at 20cm CCM position, it increases the damage threshold and allows more
power to be transmitted through the fiber as shown in Fig. 5.7. As shown in Fig. 5.5, a factor 2
more power can be delivered through the fiber while recovering the original input pulse width by
using long enough length of the fiber so that its positive GDD is matched with the negative GDD
from the pulse compressor.
94
80-- 187 fsec
E 60- 60- -e-130 fsec
200
0
20
40
60
80
100
120
Input Power [mW]
Fig. 5.7 Damage threshold of the fiber after 64x pulse multiplication without pulse stretching (red) and
with pulse stretching of CCM at 20cm position (blue).
As shown in Fig. 5.2, the fiber damage occurs at 360nJ pulse energy. We expected that
the 64x pulse splitting would enable to deliver 64x pulses with 360nJ before damaging fiber.
However, it turns out that the fiber is damaged at much lower pulse energy of about 94nJ
(average power of 60mW at 640 kHz) after 64x pulse splitting. This implies that increasing
repetition rate using the current scheme may not guarantee to improve TPA since the linear
increase of TPA by the increased repetition rate would be caught up by the quadratic decrease of
TPA by decreased pulse energy. Ultimately, it is desirable to have a variable repetition rate
regenerative amplifier that can provide more flexibility in tuning the pulse energy keeping it
below fiber damage threshold and tuning the repetition rate to generate as many pulses as
possible within the total power limit of the light source or until the occurrence of ground
depletion state of the fluorophore.
5.4 Design of a temporally focused widefield two photon endomicroscope (TFEM)
Endoscopic implementation of the TFM requires a design strategy for implementing the
size of the probe small enough to be applicable for the GI track imaging while achieving a
reasonable field of view and resolution at the same time. This section describes the design
principles for a TFEM based on the previous literatures for optimizing the performance of a
TFM [28, 29].
95
The form factor of TPE require the use of micro-lenses as objectives. Currently, the NA
of the available micro-lenses is relatively small compared to normal objectives. For example, the
highest NA of commercially available microlenses is 0.8 that have been achieved by the
compound lens made of a GRIN lens and a plano-convex lens [30]. This limits the achievable
depth resolution of a TFEM. To utilize the full NA of the objective the focal length of the tube
lens has to be such that the full spectrum of the excitation beam just fills the back aperture of the
objective. At the same time, the excitation field of view is demagnified by the focal length ratio
of the tube lens and the objective. Therefore, the focal length of the focusing lens has to be
selected such that both the depth resolution and the field of view are satisfied at the same time.
The angular spread AP of a spectrum with spectral width Ak of first order diffraction can be
obtained by differentiating the grating equation, assuming the incidence angle is constant [31].
A8 = G secJAL
(5.3)
where f is the first order diffraction angle, G is the groove frequency. Assuming the paraxial
approximation, the width of the focused spot size at the back aperture of the objective is f AP,
wheref is the focal length of the tube lens. Therefore, to utilize the full NA of the objective, each
design parameters have to be selected to satisfy the following relation
D = fG secpAA
(5.4)
where D is the diameter of the back aperture of the objective.
Fig. 5.8 shows the ray tracing of both excitation and emission beam path of TFE.
Excitation beam is delivered through HCPCF (HC-800-01, NKT photonics) and collimated by
plano-convex lens (NT65-308, Edmund Optics, f =2.54mm, 02mm) and reflected by the prism
(NT45-524, Edmund Optics ) with the custom gold mirror coating (Evaporated Coatings) to a
reflective grating (1200 line/mm, 2 mm x 2mm, LightSmyth). The grating is custom-made so
that the first order diffraction is maximized for the excitation wavelength at 800nm. The input
pulses are spectrally dispersed by the grating and focused by a plan-convex lens (Edmund Optics,
NT45-964, f =12mm, 03mm ) at the back aperture of custom-made GRIN objective (NA 0.8,
GRINTECH) that collimates each spectral component and recombines them at the focal plane of
the objective to restore original input pulse width. The image from the focal plane is magnified
and relayed through custom-made GRIN lens and beam cube splitter block (GRINTECH) and
detected by a 1/10" size CCD camera at the distal end directly or can be relayed to the high
sensitivity detectors such as EMCCD camera through the imaging fiber bundle located at the
96
proximal end. Depth scanning can be accomplished by tuning the group velocity dispersion at
the proximal end as proposed and demonstrated by Durst at el[15]. In this way, all the scanning
actuators at the distal end can be removed, which makes the endomicroscope structure more
compact and robust.
I
T b esG
Mirror
objective
Grating--
CCD or
imaging
fiber bundle
Excitation
FOV
Fig. 5.8 (top) Ray trace of the excitation beam path. Color represents wavelength. (bottom) Ray trace of
the emission beam path . Color represents spatial field. Excitation field of view (FOV) : 30 x 60 um,
Working distance of objective: 180um, NA of objective: 0.8, All ray tracing is performed by commercial
software ( Zemax, Radiant Zemax)
The optical components are assembled in the custom-made endoscope housing as shown in Fig.
5.9.
(a)
(b)
Fig. 5.9 (a) 3D modeling of the assembled TFEM. (b) Photo showing the assembled TPE. Size
8mm( diameter), 35mm (length)
5.5 Experimental validation of the pulse delivery methods
The proposed pulse delivery method for TFEM imaging is validated experimentally by
comparing the two photon excited fluorescence signal of the fluorescent beads in three different
97
cases. In the first case, pulses from 10kHz regenerative amplifier is delivered directly without
pulse multiplication and dispersion compensation. In the second case, a pulse is multiplied 64
times without dispersion compensation. In the third case, a pulse is multiplied 64 times with
dispersion compensation. The excitation beam for these conditions are delivered through the
fiber (HC-800-01) to the temporally focused widefield two photon endoscope probe. The power
levels after the fiber delivery are 2.5mW, 50mW, 50mW respectively. These values are obtained
when the input power to the fiber are set to about 70% of the damage threshold to prevent the
fiber damage during long-term imaging process. The emission signal is detected in transmission
mode with 40x air objective and the EMCCD camera.
(b)
(a)
(c)
(d)
120
o6
10-
S4
1
2
3
Fig. 5.10 (a) Normalized two photon signal for three cases. 1 .No pulse multiplication without dispersion
compensation 2.With 64x pulse multiplication and without dispersion compensation 3.With 64x pulse
multiplication and dispersion compensation (b), (c), (d) are 2jim bead images for case 1, 2 and 3
respectively.
The two photon signal is compared with the densely packed 1p~m fluorescent beads (F13081, Molecular Probes) sandwiched by the two cover slips. The image is acquired at 100 msec
exposure time and repeated 10 times for each condition. Fig. 5.10(a) shows the normalized total
fluorescence signal summed for the whole field of view. Compared to case 1, the signal level
increases a factor of 7.5 in case 2 and a factor of 11.5 in case 3. In case 2, while the repetition
rate is increased to 64 times, the peak power of the pulse is reduced to 1/3 compared to case 1.
Since the two photon fluorescence signal per second has a quadratic dependence on the peak
power and a linear dependence on the repetition rate, about 7 fold increase can be expected. Also,
98
GDD of the fiber stretches the pulse width from 130fsec to 187fsec and the original pulse width
was recovered as shown in Fig. 5.6. Since the two photon absorption (TPA) is inversely
proportional to the pulse width, about 1.4 fold increase in two photon signal can be expected.
Therefore, by using the pulse splitter and dispersion compensation (case 3) 11.5 fold increase in
two photon signal have been achieved. This effect can also be verified in Fig.10(b)-(d) where
2pm fluorescent bead (F-8827, Molecular Probes) is imaged at each conditions. The field view is
quite elongated vertically because the input angle of the excitation beam to the grating surface is
73 degree. The FOV can be made square by expanding the beam horizontally with two
cylindrical lens pair before impinging on the grating.
5.6 Conclusion
This paper presented an improved
delivery of high peak power pulses from the
regenerative amplifier through the hollow core photonic crystal fiber for maximizing the two
photon excited fluorescence signal in the temporally focused wide-field two photon
endomicroscopy.
Signal improvement has been demonstrated by using the pulse splitter that multiplies a
pulse into 64 identical pulses with peak power below the damage threshold. Effective increase in
damage threshold of the fiber was achieve by stretching the pulses with negative GDD by pulse
compressor. At the same time, the negative GDD compensates the positive GDD by the fiber and
recover the original pulse width after the fiber. By employing the pulse multiplication and the
dispersion compensation in TFEM, 1 Ifold signal increase have been demonstrated.
If our assumption that thermal stress plays a role, variable
repetition rate regenerative
amplifiers would provide more flexibility in finding the optimum pulse energy and the repetition
rate that can maximize the signal without damaging the fiber. The damage of the fiber occurs in
most cases at the inlet of the fiber where the delicate lattice structure around the air core is
fractured by the high intensity focused beam. Recent development in fiber end treatment
technology has a great potential that can increase the damage threshold by collapsing the end
face and hermetically sealing the fiber [32]. In this way, the lattice structured does not in contact
with air directly and reduce the possibility of the damage by the ionization effect. Also, since the
input beam is guided to the core by refraction when passing through collapsed area, the beam
99
size at the end face can be made larger such that the damage threshold increases due to the
reduced intensity of the beam at the end face.
There are also many rooms for improvement in the endoscope design. The currently available
high NA GRIN lens is not well corrected for the chromatic aberration, which causes each
spectral component to be focused at different position in axial direction and broadens the pulse
width of the recombined beam at the focal plane and could lower the efficiency of the two
photon excitation. The power loss within the endoscope is mostly occurs at the grating. To
reduce the length of the endoscope size, a high grove density grating with 1200 line/mm was
used in the current design. The power loss is significant and only 15% of input power is
diffracted to the first order beam. The current design of the endoscope is 8mm in diameter. A
more compact design can be achieved by using a GRISM such that the excitation beam path can
be made straight in-line without the need for beam steering mirror [33, 34]. Finally, the GDD is
tuned by manually translating the corner cube mirror in the pulse compressor. This could be
made in high speed by tuning the GDD using a piezo bimorph mirror for both dispersion
compensation and for the remote z scanning [35].
References
1.
2.
3.
4.
5.
6.
W. Denk, J. H. Strickler, and W. W. Webb, "Two-photon laser scanning fluorescence
microscopy," Science 248, 73-76 (1990).
D. Huang, E. A. Swanson, C. P. Lin, J. S. Schuman, W. G. Stinson, W. Chang, M. R. Hee,
T. Flotte, K. Gregory, C. A. Puliafito, and J. G. Fujimoto, "Optical Coherence Tomography,"
Science 254, 1178-1181 (1991).
M. H. Xu and L. H. V. Wang, "Photoacoustic imaging in biomedicine," Rev. Sci. Instrum.
77, 041101 (2006).
M. C. Skala, K. M. Riching, A. Gendron-Fitzpatrick, J. Eickhoff, K. W. Eliceiri, J. G. White,
and N. Ramanujam, "In vivo multiphoton microscopy of NADH and FAD redox states,
fluorescence lifetimes, and cellular morphology in precancerous epithelia," P. Natl. Acad.
Sci. U.S.A. 104, 19494-19499 (2007).
M. C. Skala, J. M. Squirrell, K. M. Vrotsos, V. C. Eickhoff, A. Gendron-Fitzpatrick, K. W.
Eliceiri, and N. Ramanujam, "Multiphoton microscopy of endogenous fluorescence
differentiates normal, precancerous, and cancerous squamous epithelial tissues," Cancer Res.
65, 1180-1186 (2005).
0. Nadiamykh, R. B. LaComb, M. A. Brewer, and P. J. Campagnola, "Alterations of the
extracellular matrix in ovarian cancer studied by Second Harmonic Generation imaging
microscopy," BMC Cancer 10, 14 (2010).
100
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
D. C. S. Tai, N. Tan, S. Xu, C. H. Kang, S. M. Chia, C. L. Cheng, A. Wee, C. L. Wei, A. M.
Raja, G. F. Xiao, S. Chang, and J. C. Rajapakse, "Fibro-C-Index: comprehensive,
morphology-based quantification of liver fibrosis using second harmonic generation and
two-photon microscopy," J. Biomed. Opt. 14, 10 (2009).
H. C. Bao, J. Allen, R. Pattie, R. Vance, and M. Gu, "Fast handheld two-photon fluorescence
microendoscope with a 475 mu m X 475 mu m field of view for in vivo imaging," Opt. Lett.
33, 1333-1335 (2008).
G. J. Liu, T. Q. Xie, I. V. Tomov, J. P. Su, L. F. Yu, J. Zhang, B. J. Tromberg, and Z. P.
Chen, "Rotational multiphoton endoscopy with a 1 mu m fiber laser system," Opt. Lett. 34,
2249-2251 (2009).
R. Le Harzic, M. Weinigel, I. Riemann, K. Konig, and B. Messerschmidt, "Nonlinear optical
endoscope based on a compact two axes piezo scanner and a miniature objective lens," Opt.
Express 16, 20588-20596 (2008).
D. R. Rivera, C. M. Brown, D. G. Ouzounov, I. Pavlova, D. Kobat, W. W. Webb, and C.
Xu, "Compact and flexible raster scanning multiphoton endoscope capable of imaging
unstained tissue," P. Natl. Acad. Sci. U.S.A. 108, 17598-17603 (2011).
Y. C. Wu, J. F. Xi, M. J. Cobb, and X. D. Li, "Scanning fiber-optic nonlinear
endomicroscopy with miniature aspherical compound lens and multimode fiber collector,"
Opt. Lett. 34, 953-955 (2009).
D. Oron, E. Tal, and Y. Silberberg, "Scanningless depth-resolved microscopy," Opt. Express
13, 1468-1476 (2005).
G. H. Zhu, J. van Howe, M. Durst, W. Zipfel, and C. Xu, "Simultaneous spatial and
temporal focusing of femtosecond pulses," Opt. Express 13, 2153-2159 (2005).
M. E. Durst, G. H. Zhu, and C. Xu, "Simultaneous spatial and temporal focusing for axial
scanning," Opt. Express 14, 12243-12254 (2006).
L. C. Cheng, C. Y. Chang, C. Y. Lin, K. C. Cho, W. C. Yen, N. S. Chang, C. Xu, C. Y.
Dong, and S. J. Chen, "Spatiotemporal focusing-based widefield multiphoton microscopy
for fast optical sectioning," Opt. Express 20, 8939-8948 (2012).
P. Russell, "Photonic crystal fibers," Science 299, 358-362 (2003).
D. Kim, H. Choi, S. Yazdanfar, and P. T. C. So, "Ultrafast Optical Pulse Delivery With
Fibers for Nonlinear Microscopy," Microsc. Res. Techniq. 71, 887-896 (2008).
S. P. Tai, M. C. Chan, T. H. Tsai, S. H. Guol, L. J. Chen, and C. K. Sun, "Two-photon
fluorescence microscope with a hollow-core photonic crystal fiber," Opt. Express 12, 61226128 (2004).
J. D. Shephard, F. Couny, P. S. Russell, J. D. C. Jones, J. C. Knight, and D. P. Hand,
"Improved hollow-core photonic crystal fiber design for delivery of nanosecond pulses in
laser micromachining applications," Appl. Optics 44, 4582-4588 (2005).
S. 0. Konorov, V. P. Mitrokhin, A. B. Fedotov, D. A. Sidorov-Biryukov, V. I. Beloglazov,
N. B. Skibina, E. Wintner, M. Scalora, and A. M. Zheltikov, "Hollow-core photonic-crystal
fibres for laser dentistry," Phys. Med. Biol. 49, 1359-1368 (2004).
A. A. Ishaaya, C. J. Hensley, B. Shim, S. Schrauth, K. W. Koch, and A. L. Gaeta, "Highlyefficient coupling of linearly- and radially-polarized femtosecond pulses in hollow-core
photonic band-gap fibers," Opt. Express 17, 18630-18637 (2009).
N. Ji, J. C. Magee, and E. Betzig, "High-speed, low-photodamage nonlinear imaging using
passive pulse splitters," Nat. Methods 5, 197-202 (2008).
101
24. T. H. Wang, P. D. Kumavor, and Q. Zhu, "Application of laser pulse stretching scheme for
efficiently delivering laser energy in photoacoustic imaging," J. Biomed. Opt. 17, 061218
(2012).
25. D. L. Wokosin, "Pulse duration spectra and measurements for laser scanning microscope
systems," Proc. Spie 5700, 1-13 (2005).
26. S. Akturk, X. Gu, M. Kimmel, and R. Trebino, "Extremely simple single-prism ultrashortpulse compressor," Opt. Express 14, 10101-10108 (2006).
27. B. E. A. Saleh and M. C. Teich, Fundamentalsofphotonics, 2nd ed. (John Wiley & Sons,
2007).
28. D. Kim and P. T. C. So, "Three-Dimensional (3D) High-Speed Imaging and Fabrication
System Based on Ultrafast Optical Pulse Manipulation," Proc. Spie 7183, 71831 B (2009).
29. H. Dana and S. Shoham, "Numerical evaluation of temporal focusing characteristics in
transparent and scattering media," Opt. Express 19, 4937-4948 (2011).
30. B. Messerschmidt, A. Kraeplin, S. Schenkl, I. Riemann, M. Stark, A. Ehlers, A. Tchernook,
R. Le Harzic, and K. Koenig, "Novel concept of GRIN optical systems for high resolution
microendoscopy. Part 1: Physical aspects - art. no. 643202," Proc. Spie 6432, 43202-43202
(2007).
31. C. Palmer and E. Loewen, Difftaction gratinghandbook, 6th ed. (Newport, 2005).
32. "Photonic crystal fiber end-sealing," (NKT Photonics).
33. C. Pitris, B. E. Bouma, M. Shiskov, and G. J. Tearney, "A GRISM-based probe for
spectrally encoded confocal microscopy," Opt. Express 11, 120-124 (2003).
34. H. Dana and S. Shoham, "Remotely scanned multiphoton temporal focusing by axial grism
scanning," Opt. Lett. 37, 2913-2915 (2012).
35. A. Straub, M. E. Durst, and C. Xu, "High speed multiphoton axial scanning through an
optical fiber in a remotely scanned temporal focusing setup," Biomed. Opt. Express 2, 80-88
(2011).
102
Chapter 6
Improvement of axial resolution and contrast in
temporally focused widefield two-photon microscopy
with structured light illumination
6.1 Introduction
Temporally focused wide-field two-photon microscopy (TFM) of ultrafast optical pulses
is an imaging technique by which depth-resolved wide-field two-photon images can be acquired
without the need for raster scanning of the focal spot [1, 2]. Because of its simplicity and highspeed image acquisition capability, TFM has been adapted to various biological imaging
applications such as 3D super-resolution imaging [3], cellular dynamics imaging [4] and the
depth-resolved fluorescence and phosphorescence lifetime imaging [5]. In addition, TFM has
been applied to single cell precision optogenetic control of neuronal activity [6, 7] and high
throughput microfabrication [8, 9]. Recently, it has been also shown that TFM can maintain a
uniform illumination field of view even in a highly scattering medium [10].
However, TFM suffers from a broader axial resolution when compared to the standard
two-photon laser point scanning microscopy (TPLSM), which is caused by the under-utilization
of the numerical aperture (NA) of the objective [11]. The FWHM axial resolution achieved to
date is 1.6tm through optimizing the spectral width of the ultrafast pulse and the parameters of
the intermediate optical elements which is still worse than TPLSM by a factor of two [3, 8].
Further improvement in axial resolution down to 0.85ptm FWHM was achieved by filling the
back aperture of the objective with the spectrum of an ultrafast pulse that is dispersed in two
dimensions at the expense of increasing complexity of the system [11]. More significant problem
is that the axial extent of the excitation volume is progressively broadened in a turbid medium as
the imaging depth increases, which further increases the background noise [12]. Recently, it has
been shown that this effect can be partly avoided by using the line focused TFM which maintains
axial resolution even at 2.5 scattering mean free path (MFP) [13].
103
Apart from a reduced axial resolution TFM uses imaging detectors such as a CCD or a
CMOS. The resolution is based on the telecentric mapping of emission photons from the
specimen plane to the image plane. For deep imaging, the scattering of emission photons
degrades the image contrast obscuring the fine features of the specimen. In contrast, the scattered
emission photons of TPLSM are integrated with a large area detector and the spatial structure
and resolution of image depends only on the temporal sequence of the scanning process and the
excitation point-spread function [14].
In this paper, we demonstrate that the use of structured light illumination (SLI) in TFM
can effectively reject background scattered emission photons and thereby improve image
contrast when imaging in a turbid medium. Equally importantly is that the axial resolution of this
system is dictated by the spatial frequency of the structured light thereby allowing an axial
resolution that is significantly better than TFM even in the absence of scattering. The core idea
of this approach is that SLI acts as a virtual pinhole and the low frequency out-of-focus scattered
photons beyond the depth of field of the objective can be removable computationally.
6.2 Methods of generating structured light illumination in TFM
A class of depth-resolved imaging techniques based on SLI have been proposed to select
a particular imaging plane and to reject out-of-focus background for standard wide-field singlephoton microscopy [15, 16] and it has been shown that the axial resolution of these techniques is
comparable to that of the confocal microscope [17, 18]. Of these methods, one effective
approach we adapted is termed 'HiLo microscopy' which generates an optically sectioned image
by post-processing the uniformly illuminated image (UI) and the structured light illuminated
image (SI) [16]. More specifically, the in-focused high frequency contents are extracted by highpass filtering UI with a Gaussian shaped high-pass filter. The in-focus low frequency contents
are extracted by low-pass filtering the absolute of UI subtracted by SI with the complementary
low pass filter to the high pass filter. The cutoff frequency of the Gaussian filter is determined by
the sinusoidal spatial frequency of the structured illumination. Subsequently, the optically
sectioned image is obtained by combining the two with an adjusting factor so that the transition
from low to high frequencies occur smoothly [19]. HiLo microscopy has been widely used in the
104
context of background rejection for light-sheet microscopy [19, 20], TFM [21, 22] and depthresolved microrheology [23].
Fig. 6.1 shows the schematic diagram of an SLI TFM based on a Michelson
interferometer. Ultrafast optical pulses from Ti:Sappire laser with 100fs pulse width, 80MHz
repetition rate at the center wavelength of 780nm (MaiTai, Spectra-Physics, Mountain View,
CA) are diffracted off the reflective diffraction grating with the groove frequency of 600
grooves/mm (53004BK02-351R, Richardson Grating Lab, Rochester, NY) and each spectral
component of the beam is focused at the back focal plane of the objective (Zeiss C-Apochromat
40x, NA 1.2 water immersion, Zeiss MicroImaging, Thornwood, NY) that collimates each
spectral components and recombine them at the focal plane of the objective to restore the
original input pulse width.
BE
46RDG
Ultrafast Optical Pulse
from Ti:Sapphire Laser
Y4
f 200
EmTL
Ob
f =250
Grid
- -f=100
FFP
N
f =100
ExTL
BP,
X
f =100
GP
FP
Fig. 6.1. Generating SLI is possible through either an interferometer or a grid. The interferometric setup
shown here is much simplified for grid projection where the components in the dotted box are bypassed
and the light goes directly from RDG through the f=200mm and f=100mm lenses before passing through
the grid and onwards. BE: beam expander, RDG: reflective diffraction grating, NPB: non-polarizing beam
splitter, M: mirror, ExTL: excitation tube lens, EmTL: emission tube lens, DM: dichroic mirror, Obj:
objective, BFP: back focal plane, FFP: front focal plane, GP: grid projection, FP: fringe projection
There are two possible methods of generating SLI. First, it can be generated by projecting
a fringe (Fringe Projection (FP)) where the incoming beam diffracted off a grating is split by a
beam splitter and travels down two separate arms. The angle of mirrors 1 and 2 are adjusted so
that two parallel strips of pulse spectrum are focused at the back focal plane of the objective and
they interfere at the front focal plane of the objective resulting in fringe pattern. The optical path
105
length of the two arms is matched to maximize the contrast of SLI. The other method is
projecting a grid (Grid Projection (GP)) where the grid is placed at a plane conjugate to both the
grating and the front focal plane of the objective. The entire interferometric arm may be
bypassed so that the incoming beam off the grating passes directly through the f = 200mm lens
onto the f = 100mm lens before passing through the grid and the rest of the optical setup. In GP,
there are three strips of focused spectrum at the back focal plane each corresponding to 0th and
±1st order diffracted beam from the grid. The interference of these three orders generates the
fringe pattern at the focal plane of the objective. SI and UI are obtained sequentially by manually
blocking and unblocking one of the beam path in the Michelson interferometer for the case of FP.
However, one could use the optical chopper to automate this process. For the case of GP, we
manually move the grid in and out of the optical path at the position indicated in Fig. 6.1.
Another way is to use two different beam paths of which one includes a grid and the other
without a grid. SLI is accomplished by switching between the two beam paths with a
galvanometer mirror as demonstrated by Ford et al. [24]. Finally, one may combine TFM with a
spatial light modulator and generates an arbitrary excitation pattern for various applications
which need patterned two photon excitation [4, 7, 25].
The intensity at the front focal plane of the objective for FP can be expressed as
IFringe
=
1+
cos(2k sin Of y)
(6.1)
where k is the wave number of the center wavelength and Of is the half angle between the two
beams after the objective. Here, we ignore the complexity of the pulsed beam and take only the
center wavelength and assume equal intensity for the two interfering beams. Likewise, the
intensity for GP can be expressed as
IGnd
=1 +
4a cos(k sin Ogy) cos (k (Cos Og
-
1) z) + 2a2 (i + cos(2k sin Ogy))
(6.2)
where a is the relative strength of the first order beam compared to zero order beam and 0 g is the
angle between the zero order beam and the first order beam after the objective. In case of
rectangular grid, a
=
2/7. The 3D plots of both Eq. (6.1) and Eq. (6.2) are shown in Fig. 6.2(a)
and 6.2(b) respectively. In case of GP the fringe pattern is formed not only laterally (along y-axis)
but also axially (along z-axis). This property was utilized to increase the resolution both laterally
and axially in a conventional wide-field microscope [26].
The two methods of SLI generation is compared in terms of the fringe contrast and
density. The contrast of the sinusoidal fringe pattern is defined as
106
Contrast =
(6.3)
"""
I
+1
Instead of using Eq. (6.3), we calculate the contrast of either case in the frequency domain from
the recorded fringe patterns. This method is superior to the direct calculation of the contrast
using Eq. (6.3) in that it is less affected by the shape of the illumination pattern and the local
intensity variation due to the noise. For example, for the Gaussian illumination shape the
sinusoidal fringe pattern is expressed as
(6.4)
U
f(Y) = I(I+Mcos(ky))-exp2
K2a 2
where M and k are the contrast and the spatial frequency of the sinusoidal fringe pattern
respectively. The Fourier transform of Eq. (6.4) is
F(m) = [S(m)+I M(m- k)+
M,(m+k)
exp -
M2
(6.5)
where o represents convolution, 6(m) is delta function. Therefore, M can be calculated as
follows.
M
=2
F(m = ±k)
F(m = 0)
(6.6)
The coherent illumination fringe pattern is recorded by reflecting SLI from a mirror
placed at the focal plane of the objective. The emission filter in the detection path is removed to
allow detection in the excitation wavelength. The incoherent detection fringe pattern is recorded
by illuminating SLI on thin layer of Rhodamine solution. The thin Rhodamine solution is
prepared by first dissolving Rhodamine power in alcohol and diluted to 300pM concentration.
Then, a drop of Rhodamine solution is placed on the cover slip and covered with the same size
cover slip and squeezed to form a meniscus around the edge of the cover slip. The volume of a
drop of Rhodamine solution is determined by multiplying the area of the cover slip with the
desired thickness that needs to be less than the axial resolution of TFM. The recorded image is a
fluorescence image and the emission filter is placed in the detection path.
In principle, the contrast of the illumination fringe pattern for GP and FP should be close
to 1 as long as the pulse spectrum is within the pass band of the coherent optical transfer function
(OTF) of the excitation path as can be predicted by Eq. (6.1) and Eq. (6.2) and their plot in Fig.
6.2(a) and 6.2(b). The contrasts of the detected fringe pattern of thin Rhodamine solution are
attenuated by the incoherent OTF of the detection path. The average and standard deviation of 10
107
independently measured contrasts for the structured illumination period of Tg = 1.71 pLm are
summarized in Fig. 6.2(c), which shows 0.785
0.002 and 0.778
0.014 for GP and FP
respectively in case of illumination and 0.236
0.002 and 0.225
0.001 for GP and FP
respectively in case of detection. There are quite differences between the estimated contrast
values shown in Fig. 6.3(a) and the measured contrast values. This could be attributable to the
broad-band excitation and emission wavelengths. The estimated contrast of fringe pattern is
based on the peak wavelength of both excitation and emission band. In case of illumination, the
bandwidth of pulsed excitation laser has about 10nm FWHM (20nm full width). The sum of
interference pattern by each individual wavelength components has the effect of adding a DC
value and this reduces the contrast. In case of detection, this effect is much stronger since the
emission band of Rhodamine has about 50nm FWHM (100nm full width). The other reason
could be the out of focus background. The estimated thickness of the thin Rhodamine solution is
2um, which is thicker than the SLI resolution and the signal from out-of-focus background could
reduce the contrast.
In terms of the fringe pattern stability from the external disturbances such as mechanical
vibration or air flow, GP is more stable than those obtained through FP since GP is essentially a
common path geometry whereas FP is formed by the two different beam paths in Michelson
interferometer. In instances when signal level is low and longer exposures are required, the
contrast of FP is slightly worse than GP since the fringe pattern jitters due and the image of the
fringe is smeared and therefore averaged during the exposure time. Practically, the power loss
due to reflection from the grid surface is significant and this lowers the excitation efficiency.
The fringe density of the illumination pattern determines the signal to noise and the depth
resolution of HiLo processed image. FP can generate higher density pattern than GP for
objectives with limited aperture size such as GRIN lens. For example, in case of the fringe period
of 1.71tm, 6 g is about two times larger than Of, which means the distance between +1 and -1
order of the pulse spectrum of GP has to be two times larger than the distance between the two
pulse spectrum of FP.
108
(C)
(b)
(a)
(a)(b()
0.5U
0.-
0-0
20
Detection
0 Illumination
y [pm]
-2
[pm] -2
of the
Fig. 6.2. (a) The profile of the illumination fringe pattern of FP predicted by Eq. (1). (b) The profile
Z [pim] -2
-2
y [m]
Z
illumination fringe pattern of GP predicted by Eq. (2). ) = 780nm, Of =13.2 degree, Og = 27.2 degree,
the
a=2/a (c) The experimentally measured contrast of the coherent illumination fringe pattern and
to
compared
small
are
deviations
Standard
GP.
incoherent detection fringe pattern of both FP and
averages and not well visible in the plot.
6.3. Depth resolution improvement using SLI in TFM
6.3.1 Theoretical estimation
It has been shown that the integrated two photon excitation (TPE) signal of TFM at an
2
axial plane at position z decays proportional to z-1 [27] whereas that of the TPLSM has z
root of
dependence. Also, the shape of this integrated TPE curve of TFM is similar to the square
Lorentzian function [3], which has a long tail on both sides of the peak of the curve. Therefore,
tails
the axial confinement of the excitation volume of TFM is worse than TPLSM and the long
the
of the excitation volume generate out-of-focus background. Thus, the use of SLI in TFM has
potential of removing such background signal and improving the axial resolution of TFM.
Theoretical axial resolution of SLI can be estimated using the defocused 2D OTF derived
by Stokseth [28].
C(u,m)= f(M)
um[I-m2
(6.7)
3
2
and is
where f(m) = 1 - 0.69m + 0.076m + 0.043m , m is the normalized fringe frequency
related to the real fringe period Tg via m = V/(Tg NA), NA = n sin(a) and u is the normalized
2
In incoherent
defocus and is related to the actual defocus z via u = 4kznsin (a/2), k = 2n/.
is equivalent
detection, OTF represents the contrast of the fringe detected at the image plane and
to the signal generated when a thin sheet of fluorescence is scanned through the focus [29].
Fig. 6.3(a) shows the plot of Eq. (6.7) at the fringe periods of Tg = 3.42pim, 1.71p m,
1.04,
0.85 ptm, 0.43 pm which correspond to the normalized fringe frequencies of 0.13, 0.26, 0.52,
109
respectively. For the fringe period of 1.71 pm and higher, FWHM of the axial resolution is
expected to be better than that of TFM reported in the literatures [3, 8]. Furthermore,
theoretically, SLI has the potential of attaining depth resolution better than TPLSM when the
normalized fringe frequency is higher than 0.3 as shown in Fig. 6.3(b). Ideally, the best optical
sectioning is achieved when the normalized fringe frequency comes close to 1 but with the
tradeoff of the reduced signal to noise ratio [29]. For this comparison, the intensity of SLI is
obtained by normalizing the defocused 2D OTF and the total fluorescence intensity of TPLSM
generated at a given z-plane for a uniform specimen is calculated by integrating the two photon
intensity in each z section [30].
(a)
_____(b)
-Tg
1-
T.
Tg = 0.85
Tg=1.71
-Tg = 3.42
0.8
O
Z0%-Tg
0.8
0.6
0.6-
0.4!
0.4
=0.43
= 0.85
-Tg =1.71
-Tg = 3.42
TPLSM
E
0 V.2
0.2
0.5
1
Defocus [ptml
25
20
15
0
o0 5
Nomalized Defocus, u
1.5
Fig. 6.3. (a) Contrast decay of fringe pattern of the spatial period of Tg = 0.43, 0.85, 1.71, 3.42 pm as a
function of the defocus (distance from focal plane). (b) Normalized intensity of SLI and TPLSM as a
function of the normalized defocus unit
6.3.2 Experimental verification
A thin sheet of Rhodamine 6G (Sigma-Aldrich, St. Louis, MO) solution is used to
measure how much SLI improves the axial resolution in different scattering conditions. A
scattering medium is simulated by using, Lipofundin-20 (B. Braun, Germany), as the immersion
medium of the objective. This approach allows us to precisely vary the scattering coefficient of
the specimen by controlling the concentration of Lipofundin ranging from 2% to 5% v/v. The
reduced scattering coefficients (p's) of the Lipofundin solutions were measured with a
commercial near-infrared spectrometer (OxiplexTS, ISS Inc, Champaign, IL) at 690nm, 830nm
and the g's at 780nm is estimated by the linear interpolation, which is subsequently used to
calculate the scattering MFP (Is) and anisotropy coefficient, g was estimated to be 0.65 [31]. z
110
represents the scattering length and z/ ls is calculated assuming z is equal to the working distance
of the objective which is 280pm. The result of these measurements is summarized in Table 6.1.
The 2% scattering property is similar to the human dermis [32]
and has been used for
characterizing point spread function of TPLSM in turbid medium [33] and well characterized by
previous studies [31, 34].
Table 6.1. Reduced scattering coefficients
2%
3%
4%
p's (cm') at 690nm
14.6± 0.8
20.1 ±0.2
27.2 0.5
21.3 0.4
15.6 0.2
11.3 ±0.6
P's (cm') at 830nm
23.4
17.2
12.5
pU's (cm~') at 780nm
150.6
204.8
282.4
ls (im ) at 780nm
1.9
1.4
1.0
z /ls at 780nm
5%
32.6 0.6
25.6 0.4
28.1
125.4
2.2
We performed the measurement under two scattering conditions. First, the experiments
are performed without any scattering medium (0% Lipofundin) and then undertaken with 2%
Lipofundin as an immersion medium. The measurements were repeated 10 times independently
at each scattering condition. The FWHM value is estimated by fitting the raw data with a
Gaussian curve. From Fig. 6.4, it is evident that SLI improves the axial resolution of TFM
significantly. At 0% Lipofundin, TFM's FWHM is 3.92
+
0.05ptm and TFM HiLo's FWHM is
2.19 : 0.06[tm. At 2% Lipofundin, they are 5.21 ± 0.18 im and 2.37
+
0.05pm, respectively.
The axial resolution of TFM measured with 0.1ptm fluorescent polystyrene beads (F-8803,
Invitrogen, Carlsbad, CA) is 2.01 ± 0.10pm. We may deconvolve the measured depth profile of
the thin Rhodamine solution using the point spread function measured with 0.1 ptm bead and
estimate that the thickness of the Rhodamine solution to be about 2 pm. As a comparison the
theoretical TPLSM response to the thin Rhodamine solution is also shown in Fig. 6.4(a) with
FWHM of 2.08 ptm. Considering the thickness of the Rhodamine solution is about 2 pm, the
axial resolution is broadened about 1.67 times at 1 scattering MFP condition. This broadening of
the axial resolution has already been reported in the literature [12]. Although the direct
comparison with the previous results might not be possible since the important parameters
affecting the axial resolution including the grating groove frequency, the pulse width and the
objective (NA, magnification) are not the same as the setup used for this paper, we can make
comparison of general trend of broadening effect. According to the Fig. 3 in [12], at 200 pm
111
scattering depth which corresponds to about 1 scattering MFP, the FWHM of TFM is broadened
about 1.6 times compared to Opm scattering depth for both 40x, NA=0.8 and 60x, NA=1 imaging
conditions. This result agrees well with our result shown in Fig. 6.4. It is interesting to note that
HiLo processed data is not much affected by the scattering medium, which implies that HiLo
based SLI works robustly even in highly scattering medium.
(b)
(a)
1
.Ir-TFMHiLo'O%
6
0-TFM2%
0.8
Z-
TFM
MHiLo TFM
-- TFM HiLo 2%
5
4
0.2
-7-6-54-3-2-1
1234567
0
0%
2%
Axical Position [aim]
Fig. 6.4. (a) Axial resolution measured with the thin Rhodamine solution. Fringe period of SLI is 1.71 im
(red) TFM HiLo at 0% intralipid (blue) TFM HiLo at 2% intralipid (green) TFM at 0% intralipid (black)
TFM at 2% intralipid (orange) TPLSM in Fig. 6.3(b) convoluted with 2 pim thick Rhodamine solution. (b)
Averages and standard deviations of FWHM of 10 measurements for 0% and 2% scattering condition of
both TFM and HiLo TFM
The axial resolution improvement is further demonstrated with a prepared slide of
sectioned mouse kidney (F24630, Invitrogen, Carlsbad, CA). At 0% Lipofundin condition, the
sample is imaged at 0.5pjm step size for a total of 14im scanning range axially. Fig. 6.5 shows
the xz section view of TFM image without SLI and HiLo processed TFM images with fringe
period of 3.42 im, 1.71 pm, 0.85 pim respectively. As the fringe frequency increases the sectioning
capability of SLI increases and finer features of the specimen becomes visible more clearly. The
intensity profiles along the axial direction at the point indicated by the yellow arrows are also
shown on the right side of Fig. 6.5. The weak intensity object in the middle of the sample
indicated by green arrows, which is hidden by the two high intensity objects above and below,
becomes discernible with SLI at Tg = 0.85pm. This effect is better visualized in the normalized
intensity plot where the intensity is normalized with the peak intensity value. Clearly, the SLI
approach improves TFM axial resolution.
112
150 -No SLI
--Tg = 3.42 pm
No SLI
1 0
z
-Tg=1,1 pm
- Tg=O.85 pm
/
Tg=3.42pm
o
2
4
6
8
Z [pm]
10
12
14
6
8
Z [pm]
10
12
14
Tg=1L71pm
a .40.5
o
/
z
Tg=0.85pm
0
2
4
Fig. 6.5. xz sections of the fine glomeruli and convoluted tubules structure in a mouse kidney sample
acquired with TFM without SLI, HiLo processed TFM with fringe period of 3.42pm, 1.711pm, 0.85pm,
respectively. The thickness of the imaged portion is 14pm. Intensity increases from purple to red. The
cross sectional intensity plot along the line indicated by the yellow arrow is also shown on the right side.
Further details on the sample can be found in http://products.invitrogen.com/ivgn/product/F24630
6.4. Contrast enhancement using SLI in TFM
In addition to the depth resolution improvement, SLI is beneficial to rejecting the out-offocus scattered emission photons and consequently improves the signal to background ratio or
image contrast. To simulate the various scattering conditions the sample was imaged through
280pm of immersion fluid and approximately 5ptm into the sample at 0% (Fig. 6.6(a) and
6.6(b)), 3% (Fig. 6.6(c) and 6.6(d)), 5% (Fig. 6.6(e) and 6.6(f)) Lipofundin concentration. The
sample is identical as the one used in Fig. 6.5. SI was imaged with the fringe period of 1.71 pm.
It is clear that the background rejection improves the contrast dramatically and fine structures are
more clearly visible in HiLo processed images. However, the fine features are progressively lost
as the concentration increases and this lost information cannot be recovered by the background
rejection as shown in red box in Fig. 6.6. This can be viewed in two ways. First, the axial
confinement of the excitation volume is broadened at the scattering condition as shown in Fig.
6.4 and this contributes to increased out-of-focus background signal. Although this background
signal can be removed by the HiLo processing, the shot noise generated by the background
signal still survives and the reduced signal to noise ratio ultimately limit the imaging depth in
113
turbid medium. Second, the fine features are lost by the scattering of the emission photons in
widefield imaging modes and this cannot be recovered by the HiLo processing.
Fig. 6.6. Unprocessed images in scattering conditions (a) 0% v/v, (c) 3% v/v, and (e) 5%v/v Lipofundin20. Processed images are (b), (d), and (f) for the same scattering conditions, respectively. Each image
represents a 90gm x 70gm.
6.5. Conclusion
In conclusion, we have demonstrated the first use of a HiLo based SLI in TFM. While
TFM has intrinsic optical sectioning, the presence of excitation and emission photon scattering
can compromise this ability. It is therefore useful to implement SLI to counter the broadening of
the axial extent of the excitation volume in a scattering medium as well as to reject the scattered
background photons, which improves the axial resolution and the contrast of the image. However,
the increased shot noise from the background signal in the turbid medium and the information
lost due to emission photon scattering cannot be avoided with the use of SLI.
Current implementation of SLI in TFM is not optimized in terms of imaging speed,
excitation efficiency and the sample exposure to the excitation light for minimizing the
photobleaching and phototoxicity effect. Here, we propose methods to improve the current
performance of SLI TFM. First, HiLo TFM is slower than TFM due to the fact that it requires
two images per z-section. Thus, the imaging speed can be improved by automating the switching
114
mechanism between UI and SI. For example, 10Hz imaging speed was demonstrated by using a
galvano-mirror to switch between UI and SI in case of GP SLI [24] or one can split the beam
into +1 and -1 order using a diffractive optical element and modulate one of the diffracted beam
using high speed laser beam shutter in case of FP SLI. This also reduces the power lost in the FP
SLI where the interferometer based FP SLI wastes half of the excitation beam power. Second,
the excitation efficiency of TFM is limited by the available laser power in a turbid medium. As
demonstrated by Cheng et al. [35], high peak regenerative amplifier can be used to increase the
excitation efficiency and thereby improving the imaging speed of the TFM. Imaging speed can
be further improved by increasing the repetition rate of the pulsed laser source using the pulse
multiplier. However, this works only in a regime where the two photon excitation probability is
low enough so that increasing repetition rate would not induce the saturation of fluorophores. In
addition to the increased signal level, the photo-bleaching effect was also shown to be reduced in
this way [36].
Lastly, this technique might be useful for improving the depth resolution of TFM based
endoscopy which has poor depth resolution due to limited NA of the GRIN lens.
References
1.
2.
3.
4.
5.
6.
D. Oron, E. Tal, and Y. Silberberg, "Scanningless depth-resolved microscopy," Opt. Express
13, 1468-1476 (2005).
G. H. Zhu, J. van Howe, M. Durst, W. Zipfel, and C. Xu, "Simultaneous spatial and
temporal focusing of femtosecond pulses," Opt. Express 13, 2153-2159 (2005).
A. Vaziri, J. Y. Tang, H. Shroff, and C. V. Shank, "Multilayer three-dimensional super
resolution imaging of thick biological samples," P. Natl. Acad. Sci. U.S.A. 105, 2022120226 (2008).
0. D. Therrien, B. Aube, S. Pages, P. De Koninck, and D. Cote, "Wide-field multiphoton
imaging of cellular dynamics in thick tissue by temporal focusing and patterned
illumination," Biomed. Opt. Express 2, 696-704 (2011).
H. Choi, D. S. Tzeranis, J. W. Cha, P. Clemenceau, S. J. de Jong, L. K. van Geest, J. H.
Moon, I. V. Yannas, and P. T. So, "3D-resolved fluorescence and phosphorescence lifetime
imaging using temporal focusing wide-field two-photon excitation," Opt. Express 20,
26219-26235 (2012).
B. K. Andrasfalvy, B. V. Zemelman, J. Y. Tang, and A. Vaziri, "Two-photon single-cell
optogenetic control of neuronal activity by sculpted light," P. Natl. Acad. Sci. U.S.A. 107,
11981-11986 (2010).
115
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
E. Papagiakoumou, F. Anselmi, A. Begue, V. de Sars, J. Gluckstad, E. Y. Isacoff, and V.
Emiliani, "Scanless two-photon excitation of channelrhodopsin-2," Nat. Methods 7, 848-854
(2010).
D. Kim and P. T. C. So, "High-throughput three-dimensional lithographic microfabrication,"
Opt. Lett. 35, 1602-1604 (2010).
Y. C. Li, L. C. Cheng, C. Y. Chang, C. H. Lien, P. J. Campagnola, and S. J. Chen, "Fast
multiphoton microfabrication of freeform polymer microstructures by spatiotemporal
focusing and patterned excitation," Opt. Express 20, 19030-19038 (2012).
E. Papagiakoumou, A. Begue, B. Leshem, 0. Schwartz, B. M. Stell, J. Bradley, D. Oron,
and V. Emiliani, "Functional patterned multiphoton excitation deep inside scattering tissue,"
Nature Photon. 7, 274-278 (2013).
A. Vaziri and C. V. Shank, "Ultrafast widefield optical sectioning microscopy by multifocal
temporal focusing," Opt. Express 18, 19645-19655 (2010).
H. Dana and S. Shoham, "Numerical evaluation of temporal focusing characteristics in
transparent and scattering media," Opt. Express 19, 4937-4948 (2011).
H. Dana, N. Kruger, A. Ellman, and S. Shoham, "Line temporal focusing characteristics in
transparent and scattering media," Opt. Express 21, 5677-5687 (2013).
K. H. Kim, C. Buehler, K. Bahlmann, T. Ragan, W. C. A. Lee, E. Nedivi, E. L. Heffer, S.
Fantini, and P. T. C. So, "Multifocal multiphoton microscopy based on multianode
photomultiplier tubes," Opt. Express 15, 11658-11678 (2007).
M. A. A. Neil, R. Juskaitis, and T. Wilson, "Method of obtaining optical sectioning by using
structured light in a conventional microscope," Opt. Lett. 22, 1905-1907 (1997).
D. Lim, K. K. Chu, and J. Mertz, "Wide-field fluorescence sectioning with hybrid speckle
and uniform-illumination microscopy," Opt. Lett. 33, 1819-1821 (2008).
A. Weigel, D. Schild, and A. Zeug, "Resolution in the ApoTome and the confocal laser
scanning microscope: comparison," J. Biomed. Opt. 14, 014022 (2009).
D. Lim, T. N. Ford, K. K. Chu, and J. Mertz, "Optically sectioned in vivo imaging with
speckle illumination HiLo microscopy," J. Biomed. Opt. 16, 016014 (2011).
J. Mertz and J. Kim, "Scanning light-sheet microscopy in the whole mouse brain with HiLo
background rejection," J. Biomed. Opt. 15, 016027 (2010).
D. Bhattacharya, V. R. Singh, C. Zhi, P. T. C. So, P. Matsudaira, and G. Barbastathis,
"Three dimensional HiLo-based structured illumination for a digital scanned laser sheet
microscopy (DSLM) in thick tissue imaging," Opt. Express 20, 27337-27347 (2012).
E. Y. S. Yew, H. J. Choi, D. Kim, and P. T. C. So, "Wide-field two-photon microscopy with
temporal focusing and HiLo background rejection," Proc. Spie 7903, 790310 (2011).
L. C. Cheng, C. Y. Chang, W. C. Yen, and S. J. Chen, "Spatiotemporal focusing-based
widefield multiphoton microscopy for fast optical sectioning of thick tissues," Proc. Spie
8520, 85200N (2012).
J. Michaelson, H. J. Choi, P. So, and H. D. Huang, "Depth-resolved cellular microrheology
using HiLo microscopy," Biomed. Opt. Express 3, 1241-1255 (2012).
T. N. Ford, D. Lim, and J. Mertz, "Fast optically sectioned fluorescence HiLo
endomicroscopy," J. Biomed. Opt. 17, 021105 (2012).
E. Papagiakoumou, V. de Sars, V. Emiliani, and D. Oron, "Temporal focusing with spatially
modulated excitation," Opt. Express 17, 5391-5401 (2009).
116
26. M. G. L. Gustafsson, L. Shao, P. M. Carlton, C. J. R. Wang, I. N. Golubovskaya, W. Z.
Cande, D. A. Agard, and J. W. Sedat, "Three-dimensional resolution doubling in wide-field
fluorescence microscopy by structured illumination," Biophys. J. 94, 4957-4970 (2008).
27. M. E. Durst, G. H. Zhu, and C. Xu, "Simultaneous spatial and temporal focusing for axial
scanning," Opt. Express 14, 12243-12254 (2006).
28. P. A. Stokseth, "Properties of a Defocused Optical System," J. Opt. Soc. Am. 59, 1314-1321
(1969).
29. D. Karadaglic and T. Wilson, "Image formation in structured illumination wide-field
fluorescence microscopy," Micron 39, 808-818 (2008).
30. C. J. R. Sheppard and M. Gu, "Image formation in two-photon fluorescence microscopy,"
Optik 86, 104-106 (1990).
31. H. J. van Staveren, C. J. Moes, J. van Marie, S. A. Prahl, and M. J. van Gemert, "Light
scattering in Intralipid-10% in the wavelength range of 400-1100 nm," Appl. Opt. 30, 45074514 (1991).
32. D. A. Boas, C. Pitris, and N. Ramanujam, eds., Handbook of biomedical optics (CRC Press,
2011), Chap. 5.
33. C. Y. Dong, K. Koenig, and P. So, "Characterizing point spread functions of two-photon
fluorescence microscopy in turbid medium," J. Biomed. Opt. 8, 450-459 (2003).
34. P. Sun and Y. Wang, "Measurements of optical parameters of phantom solution and bulk
animal tissues in vitro at 650 nm," Opt. Laser Technol. 42, 1-7 (2010).
35. L. C. Cheng, C. Y. Chang, C. Y. Lin, K. C. Cho, W. C. Yen, N. S. Chang, C. Xu, C. Y.
Dong, and S. J. Chen, "Spatiotemporal focusing-based widefield multiphoton microscopy
for fast optical sectioning," Opt. Express 20, 8939-8948 (2012).
36. N. Ji, J. C. Magee, and E. Betzig, "High-speed, low-photodamage nonlinear imaging using
passive pulse splitters," Nat. Methods 5, 197-202 (2008).
117
This page is intentionally left blank
118
Chapter 7
3D-resolved fluorescence and phosphorescence lifetime
imaging using temporal focusing wide-field two-photon
excitation
7.1 Introduction
Fluorescence
(FLIM)
and phosphorescence
(PLIM)
lifetime
microscopies
are
information-rich optical spectroscopic techniques [1-7]. A particularly important application of
FLIM is in the quantification of Fdrster resonance energy transfer (FRET), the preferred method
of quantifying intracellular
protein-protein interactions
in vivo. For example,
FRET
measurements have been used to measure intramolecular distances [8], and to observe dynamic
conformational changes in proteins [9] and RNA [10]. FLIM has been also applied in disease
diagnosis. Koenig and co-workers found that normal skin and cutaneous melanoma can be
differentiated by their morphological appearances in combination with their fluorescence lifetime
spectroscopic signatures [11]. A clinical trial across multiple centers in Europe is underway to
test the utility of multiphoton FLIM in the minimally invasive diagnosis of melanoma [12].
While PLIM is not as widely used as FLIM, this methodology nonetheless has several
important potential biomedical applications due to the availability of phosphorescence-based
oxygen sensors. Quenching of phosphorescence by oxygen affects the phosphorescence lifetime
of such sensors, which then enables measurement of oxygen partial pressure in vivo in tissues or
thick biological samples with high temporal and spatial resolution [13-15] . PLIM-based partial
oxygen measurements can be used to quantify the degree of hypoxic tissues or tumors, a critical
physiological parameter of solid tumors that determines tumor growth, gene expression [16],
metastatic potential [17], metabolism, prognosis [18-20], and response to therapy [21, 22]. The
use of PLIM-based oxygen sensors has also enabled the quantification of oxygen supply and
consumption in the brain, which is critical for understanding brain metabolism and cognitive
function [23, 24]. Today, PLIM is not widely used mostly due to the associated long lifetime
which entails typical image frame rate on the order of minutes to hours.
119
The implementations of FLIM and PLIM measurements in microscopy have different
challenges. For FLIM, the need for picoseconds level timing resolution of photon arrival time
requires fast electronics; for PLIM, the long lifetime (up to milliseconds) significantly lengthens
the required pixel residence time and slows the image frame rate. In general, FLIM and PLIM
measurements can be performed either in the time-domain or in the frequency-domain. In the
time domain, a light pulse of short duration excites the sample and the time delay of the emitted
photon is measured. A histogram of these time delays provides a direct measurement of the
fluorophore's lifetime. Typical fluorophores exhibit single exponential decay kinetics in a
homogeneous micro-environment. However, in practice, even for fluorophores with single
exponential decay dynamics, the experimentally measured time delay histogram is a convolution
of an unknown instrumentation response and the exponential decay of the fluorophore. In the
presence of more complex environments, multiple exponential decays must be resolved. The
frequency-domain approach is, mathematically, the Fourier equivalent of the time-domain. In
the frequency-domain, the excitation light is modulated at a frequency, (o. The intrinsic
fluorescence temporal response acts as a low-pass filter and the emitted fluorescent light is phase
shifted (AO) and modulated (M). For single exponential decay fluorophores, the modulation and
phase shift are related to the fluorescence lifetime as:
Arp = tan(cor) and M=
(7.1)
More complex decay mechanisms can be measured by measuring the modulation and
phase shift at multiple excitation frequencies of o. Importantly, the unknown instrumentation
response can be more readily isolated and removed in the frequency domain. In order to obtain
accurate phase shift and demodulation measurements, the excitation light modulation frequency
of o has to be of the same order of magnitude as the inverse of the lifetime (i.e. up to 108 Hz for
FLIM). Phase shift and demodulation are often measured indirectly by either homodyne or
heterodyne approaches. In the homodyne approach, measuring the steady state amplitudes at
different phases allows the recovery of the waveform. In the case of heterodyne measurement,
the signal is detected by a detector whose gain is modulated sinusoidally at a slightly increased
frequency of
Co
+ Ao. This electronic mixing process results in translating the phase and
demodulation information to an electronic signal at the cross correlation frequency of A~o that
can be readily isolated by low-pass filtering. The relative merits of time-domain vs. frequency120
domain approaches and homodyne vs. heterodyne detection schemes have been discussed
extensively in the literatures [25, 26].
High throughput FLIM has been the focus of technology development of several research
groups. The developed instruments fall into two categories. In the first category, gated cameras
are used for time-resolved full field imaging. These instruments have been implemented with
traditional wide-field fluorescence microcopy without depth resolution [27, 28] or with spinning
disk confocal microscope for 3D resolved imaging [29, 30]. Spectral-lifetime resolved imaging
using this approach has also been implemented [31]. Another approach to achieve 3D imaging is
based on using multifocal multiphoton microscopy (MMM), detecting in parallel the signal from
each foci using separate time-correlated single photon counting (TCSPC) circuitry [32]. The
next-generation instruments described in this report are built on the foundation established by
these prior works.
7.2 Temporal focusing wide-field Two-Photon FLIM and PLIM
This paper describes the design of a fast 3D FLIM and PLIM imaging system that is
based on combining two complementary technologies: (1) temporal focusing wide-field (TFWF)
two-photon microscopy, a method for efficiently exciting a single 3D resolved plane in a
translucent specimen, and (2) camera-based heterodyne frequency-domain lifetime measurement,
a method for highly parallelized wide-field imaging with picosecond lifetime resolution. Fig. 7.1
shows the optical and electronic design of the temporal focusing wide-field FLIM/PLIM system.
A titanium sapphire femtosecond pulsed laser (Tsunami, Spectra-Physics, Mountain View, CA)
pumped by a continuous-wave diode-pumped solid-state laser (Millennia V, Spectra-Physics,
Mountain View, CA) is used to provide a 80 MHz train of 100 fs pulses at a center wavelength
of 780 nm. The l/e 2 beam diameter of the ray is about 1-2 mm. The excitation light is intensity
modulated using an acousto-optical modulator (AOM, 1205C-2, ISOMET, Springfield, VA).
Excitation light is dispersed by a reflective diffraction grating with groove frequency of 600
grooves/mm (53004BK02-351R, Richardson Grating Lab, Rochester, NY) and is directed
towards a high numerical aperture objective (Fluar, 40X/1.30 NA Oil, Zeiss MicroImaging,
Thornwood, NY). The diffraction grating surface is placed at the conjugate plane of the object
plane resulting in an excitation area of approximately 100 x 100 ptm 2 . Away from the focal plane,
121
this arrangement guarantees that the different spectral components dispersed by the grating are
separated spatially, resulting in broader pulse width and lower two-photon excitation (TPE)
efficiency due to the constancy of the time-bandwidth product. At the focal plane, the different
spectral components are recombined, the pulse width is minimized, and TPE efficiency is
maximized. Volumetric imaging is accomplished by plane-by-plane excitation by translating the
objective using a piezo-positioner (MIPOS500, piezosystem Jena Inc., Hopedale, MA).
The fluorescence or phosphorescence images are acquired using a high speed phase
resolved camera (LI2CAM MD connected to a LIFA control unit, Lambert Instruments, Roden,
Netherlands) modified for heterodyne detection. The high speed phase resolved camera consists
of a high sensitivity proximity focused image intensifier. The gain of the intensifier can be
modulated by injecting an external analog gating signal at the intensifier cathode. The phosphor
output window of the intensifier is fiber optically coupled to a 1392x1040 pixel CCD camera,
used in 2x2 binned mode, to ensure maximum light detection efficiency.
In TFWF FLIM, the femtosecond titanium-sapphire laser pulse train provides the
required modulation frequencies at 80MHz and its harmonics. In TFWF PLIM, the AOM, driven
by a frequency synthesizer (2024, Marconi Instruments, discontinued, and DS345, Stanford
Research Systems, Sunnyvale, CA) and a digital modulation RF driver (522C-2, ISOMET,
Springfield, VA), provides modulation frequency of o from DC to about 1 MHz. The Lambert
Instruments camera has an internal high voltage amplifier that controls the gain of the intensifier;
this amplifier is driven by a second frequency synthesizer that provides a sinusoidal driving
frequency at the modulation frequency of co plus a small cross correlation frequency of Aco. The
CCD camera frame acquisition is triggered by a third frequency synthesizer operating at a
frequency equal to a quarter of the cross correlation frequency of ACo allowing the cross
correlation signal to be sampled four times per wave. For precise phase resolved measurement,
these three frequency synthesizers are phase locked to the output of the titanium laser pulse train
at 80 MHz. The typical cross correlation frequency of Aco is chosen to be 1 Hz since the current
version of the Lambert camera is not designed for heterodyne detection and has a maximum
frame rate of about 4 Hz. Significantly faster imaging (up to kHz level) should be possible by
using cameras of higher frame rate and by ensuring that the phosphor screen of the intensifier has
sufficiently fast response time. The development of an improved camera for high speed
heterodyne detection is underway under the Lambert Instrument/MIT collaboration.
122
Reflective Diffraction Grating
(a)
Wide field excitation
based on
temporal focusing
AAA
Ttanium-Spphire-... AOM
~Laser4{
D
Specimen
Synthesizer2
computer
Fig. 7.1. Temporal focusing wide-field (TFWF) FLIM/PLIM design. (a) Optical sub-system: temporal
focusing widefield multiphoton microscopy. The figure is adapted from the PhD thesis by Daekeun Kim
[8] (b) Electronic sub-system: frequency domain lifetime measurement via heterodyne detection.
7.3. Results and analysis
7.3.1 Fluorescence lifetime measurement performance and applications
The accuracy of the developed wide-field 3D lifetime imaging system was evaluated by
measuring the fluorescence lifetimes of Rhodamine B (79754, Sigma-Aldrich Products, St.
Louis, MO) solution in different solvents that have been carefully quantified in literature [33].
As a lifetime reference, fluorescein (R14782, Invitrogen, Grand Island, NY) was dissolved in
0.1M NaOH pH 8.0 buffer at a concentration of 50 pM. The lifetime of the fluorescein solution
is assumed to be a 4.0 ns single exponential. Rhodamine B was dissolved in deionized water or
spectroscopic grade methanol, or ethanol at a concentration of 50 pM. All fluorescein and
rhodamine solutions were enclosed in a hanging-drop slides covered and sealed with #1.5 cover
slip.
123
3D resolved wide-field images of these solutions were acquired 50 pm away from the
cover slip. The modulation frequency and the cross correlation frequency were set to 80 MHz
and 1 Hz respectively, while the lifetime camera readout rate was set at 4 Hz providing four
amplitude measurements of each cross correlation waveform. Results from twenty cross
correlated waveforms were integrated to generate a single dataset resulting in a total image
acquisition time of 20 seconds. The phase delay and demodulation of Rhodamine B solutions are
then quantified relative to the modulation and phase of a reference fluorescein specimen. Fitting
the measurements to a single exponential decay model allows the extraction of lifetimes
tabulated in Fig. 7.2 (a). The lifetime measurements obtained by TFWF FLIM were in good
agreement with literature values within about 0.05-0.1 ns. Note that the two ways to estimate the
fluorescence lifetime (either from phase data or from modulation data) provide very similar
results, which supports the choice of a single exponential decay model to fit Rhodamine B
lifetime data. When plotting the lifetime measurements of each pixel into a single polar plot (a
sine and cosine transform of temporal decay information into the spectral domain [34, 35]) the
lifetime of individual pixels cluster around a single location located on the universal circle as
expected for single exponential decay processes as shown in Fig. 7.2 (b). The locations of these
four distributions are consistent with the tabulated lifetimes obtained from averaging over the
whole image.
(b)
(a)
Tau
Tau
(Ph) (Mod)
Ta u
(Magde)
FL
4.01
4.18
4.00
RhBf
2.83
2.84
2.93
FL
/E
RhB/EtOH
hB/MeOH
Rhh/H
-
RhB/
MeOH
H20
2.22
2.36
2.46
1.75
1.79
1.68
0
h *11
Fig. 7.2 Demonstration of accurate measurement of fluorescence lifetime of Rhodamine B solutions in
different solvents by TFWF FLIM. Fluorescence in water was used as a reference. (a) Tabulated results of
estimated lifetime values of T extracted from either phase (Ph) or modulation (Mod) measurements.
Literature values are also included as a reference [33]. (b) Lifetime resolved data for each pixel from the
fluorescein (FL) and rhodamine solution images shown in polar plot format.
The high data acquisition speed of the developed instrument is demonstrated by FLIM
imaging of fixed fibroblasts loaded with conjugated polymer nanoparticles (CPNs) of high two124
photon cross section previously measured in excess of 15,000 GM [36]. Wide-field two-photon
lifetime resolved imaging is fundamentally limited by the need to simultaneously and optimally
excite fluorophores within a large area plane via two-photon absorption. Optimal excitation
corresponds to two-photon absorption probability on the order of 10% per laser pulse. In this
case fluorophores are excited without saturation and image resolution degradation [37]. The twophoton absorption probability per fluorophore per excitation photon pair per laser pulse can be
expressed as [38]:
pr oc
5
(7.2)
where N is the number of pixels (assuming diffraction-limited pixel size) measured in parallel, 6
is the fluorophore two-photon cross section, P is the average power per image,f is the laser pulse
repetition rate, and r is the laser pulse width. For typical titanium-sapphire femtosecond laser
sources and typical fluorophores with approximately 10 GM two-photon cross section, optimal
excitation occurs at approximately N = 100. For imaging at diffraction limited resolution of
about 0.5 4m, the maximum field of view is only about 10x 10 pm 2; an area too small for typical
biological imaging. A recent study has shown that this limitation can be remedied by the use of
high peak power sources such as regenerative amplifiers [39]. This limitation is less critical for
fluorophores of high two-photon cross section such as quantum dots or CPNs where image size
can be extended to about 100 x100 pm 2 . Imaging CPN-labeled specimens by TFWF FLIM was
used to demonstrate the data acquisition speed of TFWF FLIM and to ensure that this experiment
was not limited by inefficient wide-field two-photon excitation. A representative intensity scaled
lifetime image and the associated polar plot of pixel lifetime distribution of the specimen is
shown in Fig. 7.3. The non-symmetric, off universal circle distribution of pixel values in the
polar plot indicates non-single exponential decay of CPNs. This is consistent with the different
mean lifetimes measured from phase and modulation data. It is important to note that consistent
pixel fluorescence lifetime measurements can be obtained with integration time as short as 5 ms
(measurement uncertainty on the order of 0.1-0.2 ns). When the integration time is further
reduced to 2 ms, phase-based lifetime measurement still provides a reasonable lifetime estimate
(with less than 0.5 ns error) while modulation-based lifetime estimates start to deviate from
values obtained with longer average time (> 0.5 ns).
125
(b)
(a)
I.M.2
1fe1
1.7M0.2
1.70.2
Fig. 7.3 (a) An intensity scaled mean lifetime image of fixed fibroblasts with vacuoles loaded with
endocytosed conjugated polymer nanoparticles of high two-photon absorption cross section. Color scale
represents pixel lifetime values corresponding to the color bar with units of seconds. Image brightness
represents pixel intensity values. Black regions are ignored in analysis corresponding to locations with
intensity below 500 photons that are mostly outside the boundary of this cell. (b) Representative polar
plot of pixel lifetime values for 10 ms data acquisition time. (c) Tabulated mean lifetime values and their
standard deviations are estimated from the modulation or the phase data for four different image
acquisition rates.
The application of temporal focusing wide-field two-photon imaging in lifetime resolved
biological imaging is demonstrated in a thin 6tm thick histological cross section of a formalin
fixed injured rat sciatic nerve. After transecting the nerve, the two resulting nerve stumps were
connected with a collagen tube [40, 41]. In order to quantify the degree of induced regeneration,
myelin sheaths surrounding the newly-grown axons were visualized by labeling using
FluroMyelin Green (F34651, Grand Island, NY). Lifetime imaging by the TFWF FLIM system
was conducted using identical imaging parameters with the ones used in rhodamine solutions
imaging. Fig. 7.4 (a) shows a zoom-in cropped image of 20pmx20ptm field of view displayed as
an intensity scaled lifetime image, where the color of each pixel represents its mean lifetime and
the brightness of each pixel is proportional to the number of photons acquired. Pixel lifetime
information can also be displayed in a polar plot format as shown in Fig. 7.4 (b). Unlike the
rhodamine and fluorescein solutions, the distribution of pixel lifetimes is broader and does not
fall directly on the universal circle indicating either that the fluorescence lifetime of FluroMyelin
Green is not single-exponential or that its lifetime may be affected by the local tissue
126
microenvironment. The broadening of the lifetime distribution may also be caused by the
presence of autofluorescence signal emitted by the fixed nerve specimen. Since the fluorescence
lifetime of FluoroMyelin Green has not been adequately quantified in the literature, we evaluated
the accuracy of our TFWF lifetime measurement by imaging the identical specimen using a
point-scanning two-photon microscopy equipped with a Becker and Hickl SPC730 TCSPC
module for time-domain lifetime measurement as shown in Fig. 7.4 (c). The total pixel residence
time was 4.8ms and the integration time for a single lifetime resolved image was about 100 sec.
The mean lifetimes for the labeled myelin sheath are in good agreement at approximately 2.5ns.
The long lifetime blobs in the TCSPC correspond to nuclei separately labeled with DAPI with a
typical lifetime of approximately 4ns.
(a)
(b)
(c)
Fig. 7.4 Fluorescence lifetime imaging of an ex vivo histological sample by TFWF FLIM. (a) Intensityscaled fluorescence lifetime-resolved image of a regenerated ex vivo rat sciatic nerve after injury stained
with FluroMyelin Green. The image field of view is approximately 20x20 pm 2 (a) and
IOOX
100 pm 2(c).
Scale bar has units of seconds. (b) Polar plot representation of the pixel lifetime estimated based on the
phase data. (c) A larger view of the same sample acquired using a point scanning two-photon microscope
equipped with TCSPC lifetime resolved imaging system. The scale bar has unit of nanoseconds. The
lifetime measurements for both systems are in excellent agreement.
Fast lifetime-resolved imaging with 3D resolution is demonstrated by imaging
fluorescent beads of 15pm (F8844, Invitrogen Inc, Grand Island, NY) and 4pm diameter
(F8858). Fig. 7.5 (a) shows the intensity-scaled lifetime images taken at 5pm axial steps. The
depth resolution of the system is on the order of 2pm. The 4gm bead is uniformly labeled with
fluorophores while the 15p~m is only labeled at the surface while its interior is not impregnated
with fluorophore. It is interesting to note that while the 15pm and 4pm beads are emitting in the
yellow-green and red spectral ranges respectively, these beads have essentially the same
lifetimes. The associated polar plot of pixel lifetimes acquired at z=15pm depth suggests an
almost single exponential decay for these fluorophores as shown in Fig. 7.5 (b).
127
(b)
(a)
?,.WN
Fig. 7.5 Demonstration of fast 3D-resolved lifetime imaging by TFWF FLIM system. (a) Intensity scaled
lifetime images of 15 gm yellow-green beads and 4 pm red beads imaged at different axial plane
locations z. Pixel lifetimes were estimated from modulation data. The scale bar represents 5 gm in length.
(b) Representative polar plot of pixel lifetime information at z=1 5 gm.
The 3D resolved imaging ability of the developed system in living biological specimens
was demonstrated by imaging live human dermal fibroblasts interacting with a porous collagen
scaffold, similar to the one used to induce nerve regeneration in Fig. 7.4 [42, 43]. Fibroblasts
were labeled by either Calcein AM (a marker of cytoplasm and nucleus), or Syto 13 (a nucleic
acid marker) or both.
3D-resolved lifetime imaging was performed using the same imaging
parameters as described in Fig. 7.3-5. A representative image of a cell labeled with both markers
is shown in Fig. 7.6 (a). The lifetime of Calcein AM was estimated to be 3.5 ns and 3.1 ns using
phase or modulation data respectively. These two lifetime rate estimates are in good agreement
with the published TCSPC measurements [44]. The fluorescence lifetime of Syto 13 was
estimated to be 2.9 ns and 1.8 ns using phase and modulation data respectively. The large
difference between phase and modulation results indicates the non-single exponential nature of
the decay. Similar non-single exponential decay has been observed when Syto 13 was used in
bacterial cultures, where a similarly broad distribution of lifetime from 1.5-2.5 ns were observed
suggesting different lifetimes depending on whether the dye binds to DNA or RNA [45]. The
polar plot of the pixel lifetimes for specimens labeled only with Calcein AM shows a tight
distribution of pixel lifetimes close to the unit circle, which indicates a single exponential decay
(Fig. 7.6 (b)). On the contrary, the polar plot for specimen labeled with only Syto 13 shows
broad distribution of pixel lifetime rates that suggests a non-single exponential nature of
fluorescence decay (Fig. 7.6 (d)). As expected, the polar plot in specimens labeled with both
128
markers has a pixel distribution at the algebraic mean between the tight distribution
of Calcein
AM close to the unit circle and the broad distribution of Syto 13 (Fig. 7.6 (c)).
000+"M
Fig. 7.6 Fluorescence lifetime imaging of human fibroblasts seeded inside a collagen scaffold
doublestained with Calcein AM and Syto 13 by TFWF FLIM. (a) Intensity scaled lifetime images. (b) Polar
plot
of fibroblasts labeled only with Calcein AM. (c) Polar plot of fibroblasts labeled with both Calcein
AM
and Syto 13. (d) Polar plot of fibroblasts labeled only with Syto 13.
7.3.2 Demonstration of 3D phosphorescence and fluorescence lifetime-resolved imaging in tissue
engineering scaffolds
Wide-field lifetime-resolved imaging systems enable not only efficient 3D fluorescence
lifetime imaging, but also fast quantification of phosphorescence lifetime. Phosphorescence
lifetime measurement has significant applications in biophysics including background-free
labeling, and monitoring the binding of high molecular weight proteins. However, the most
important biomedical use of molecular phosphorescence is monitoring oxygen concentration in
biological systems. Minimally invasive measurements of partial oxygen pressure can be used, for
example, to quantify hypoxia in tumors, an important determinant of tumor physiology [16, 17]
and its response to therapy [22, 46]. Improved methods to quantify the 3D distribution of
molecular oxygen partial pressure P0 2 with cellular resolution inside solid tumors in animal
models could provide novel information for understanding how cancer physiology affects
129
therapeutic responses. Oxygen sensors based on phosphorescence quenching offer a powerful
method to measure PO2 distribution in vivo. Phosphorescence measurement is necessary since
tumor pO2 concentration is low and the chromophores must have an excited state lifetime longer
than the characteristic time of chromophore-oxygen interaction. Both phosphorescence intensity
and lifetime are sensitive functions of PO2. Intensity-based measurements are undesirable since
they depend on the local concentration of the phosphorescence probe in the tissue, which cannot
be readily quantified. In contrast, phosphorescence lifetime provides a concentrationindependent
probe
for
measuring
oxygen
quenching
rate.
The
lifetime
of typical
phosphorescence sensors for PO2 range between hundreds of ns to several hundred ps. In simple
biological samples that do not require 3D imaging, single-photon wide field phosphorescence
lifetime imaging has been implemented efficiently [47-50]. However, 3D PLIM using point
scanning confocal or multiphoton excitation is always slow since the required pixel residence
time must be substantially longer than the phosphorescence lifetime (as long as a fraction of a
millisecond). The estimated necessary duration for high-resolution 3D mapping of oxygen
distribution in a tumor can exceed days using palladium based probes (lifetime approaches one
millisecond). While these very long lifetime probes are sensitive to oxygen concentration and
suitable for hypoxic environments, such as the interior of a solid tumor, their long "turn-around"
time greatly limits the speed of PLIM. Importantly, unlike FLIM measurements, PLIM
measurement speed cannot be easily improved using conventional speed-enhancement methods
such as spinning-disk confocal, since the long phosphorescence lifetime will result in significant
signal loss due to the motion of the pinholes unless disk rotation rate is kept sufficiently low. On
the other hand, the parallelized nature of the proposed TFWF two-photon lifetime resolved
imaging system shortens PLIM acquisition time by several orders of magnitude with frame rate
approaching phosphorescence lifetimes.
In order to demonstrate the accuracy of the developed TFWF PLIM system, we
quantified oxygen concentration in solutions of 1 mM Tris (2,2'-bipyridyl) dichlororuthenium(II)
hexahydrate (TDRT, 544981, Sigma Aldrich Products, St. Louis, MO) in PBS equilibrated with
several independently calibrated oxygen/nitrogen gas mixtures of 02 mass fraction of 0, 4, 8, and
21%. The phosphorescence lifetime of the ruthenium solution equilibrated with each mixture
was found to be single exponential as indicated by the distribution of the estimated pixel lifetime
that lies on the universal circle of the polar plot (Fig. 7.7 (a)). The measured phosphorescence
130
lifetime ranges from about 400 to 600 ns.
There is an inverse dependence of the
phosphorescence lifetime on oxygen partial pressure in agreement with a Stern-Volmer
relationship (Fig. 7.7 (b), Fig. 7.7 (c)):
1
-OC
(a)
p0
(7.3)
2
(b)
S
(c)
LS
%P02
%PO2
Fig. 7.7 Fast measurement of partial oxygen pressure by TFWF phosphorescence lifetime imaging. (a)
Polar plot of phosphorescence lifetime of 1 mM Tris(2,2'-bipyridyl)dichlororuthenium(II) hexahydrate
solutions equilibrated with 0, 4, 8, and 21% P0 2 gas mixtures. Inverse phosphorescence lifetimes ((b),
from modulation data; (c), from phase data) are plotted against 02 concentration demonstrating SternVolmer dependence with R2 values of 0.99 and 0.95 respectively for linear regression.
The accuracy of TFWF two-photon PLIM method is a function of integration time of
each image frame. Specifically, measurement accuracy comparable to the one shown in Fig. 7.7
can be obtained when the integration time is on the order of several seconds. It is important to
note that this relative long integration time is due to the relatively low two-photon cross section
of TDRT, the limited number of phosphorescence photons generated, and the limited excitation
laser power (about 3W maximum output power for typical Ti-Sapphire laser sources) since the
power of about 0.7W is distributed over a 100xlOO pm 2 area. The uncertainty of ruthenium
phosphorescence lifetime measurements at integration times of 1.2, 1.5, 2.4, 3.0, 3.6, 5.5 and 6.0
seconds are shown in Fig. 7.8 (a) and the representative polar plots at the integration times of 1.5,
3.0 and 6.0 seconds are shown in Fig. 7.8 (b). It is clear that as integration time decreases, the
distribution of the polar plot broadens indicating greater uncertainty of lifetime measurement.
The uncertainty in the pixel lifetime measurements of an image for given integration time, as
quantified by the full-width-at-half-maximum of the pixel lifetime values in the polar plot, can
be related to the average number of photons per pixel. When plotted in a log-log format, linear
regression gives a slope of approximately 0.5 indicating that the lifetime measurement error is
dominated by photon shot noise.
131
A1.g Lifgimwif
491
m9e._J.
(a)
-720
-7.8
.0 7
2A
2.9
3
3.1
32
3.3
3 sec
6 sec
3A
if
1.5 sec
Fig. 7.8 Uncertainty in estimating the lifetime of 1mM TDRT solutions depends on the image integration
time. (a) TDRT phosphorescence lifetime measurement uncertainties (FWHM) for images acquired with
different integration times are plotted against the average number of photons contained in these images. (b)
Representative polar plots of TDRT solution pixel phosphorescence lifetime measurements at integration
times of 1.5, 3, and 6 sec.
The application of TFWF imaging in fast sequential phosphorescence and fluorescence
lifetime 3D imaging is demonstrated in an in vitro 3D cell-matrix system that can be used to
study how oxygenation affects cell-matrix interactions inside tissue engineering devices. The
system consists of live human dermal fibroblasts stained with Rhodamine DHPE (the dye is soon
endocytosed and primarily localized in intracellular vesicles) seeded inside a porous collagen
scaffold similar to Fig. 7.6. In situ monitoring of the oxygenation state of cells was accomplished
by adding 1mM TDRT in the cell culture buffer (PBS). 3D-resolved fluorescence and
phosphorescence lifetime imaging of the cell-seeded scaffolds was acquired sequentially using
modulation frequency (co) of 80 MHz and 300 kHz respectively. Representative images are
shown in Fig. 7.9. In PLIM imaging cells can be distinguished based on the increased lifetime
(less
oxygen)
observed
intracellularly
compared to the
surrounding medium pixels.
Representative intensity scaled lifetime-resolved images are shown in Fig. 7.10. At each pixel,
the relative intensity contributions from fluorescence and phosphorescence components can be
132
readily separated based on the pixel location within the polar plot. The distributions of the
fluorescence and phosphorescence components are shown in Fig. 7.11.
As expected, the
fluorescence emission of Rhodamine DHPE is localized in cellular vesicles, while the
phosphorescence emission of TDRT is predominately localized in the medium surrounding the
cells and the scaffold.
Fig. 7.9 Fast 3D-resolved TFWF PLIM in sample consisting of human fibroblasts stained with
Rhodamine DHPE, seeded inside a collagen matrix and treated, in PBS buffer containing 1 mM TDRT
ruthenium-based oxygen sensor. PLIM Images were acquired at 300 kHz modulation frequency. In
sequence from left to right: intensity image, phosphorescence lifetime-resolved image from phase data,
phosphorescence lifetime-resolved image from modulation data. On the far right is the polar plot of the
pixel phosphorescence lifetime-resolved measurements. The pixel lifetime data in the polar plot
distributes between the ruthenium phosphorescence lifetime that lies close to the universal circle and the
rhodamine fluorescence lifetime at the lower right hand corner corresponding to an effective zero lifetime
for 300 kHz modulation frequency. PLIM Movie sequences corresponding to phase and modulation
lifetime measurements throughout a 3D matrix are included (Supplementary material Mov 1, Mov 2).
Fig. 7.10 Representative single plane intensity scaled
frequency of 300 kHz.
133
lifetime-resolved image measured at a modulation
Fig. 7.11 Fast sequential 3D-resolved TFWF FLIM and PLIM imaging of human dermal fibroblasts
seeded in a collagen scaffold. Representative images of the fluorescence component emitted by
Rhodamine DHPE (top), and the phosphorescence component, emitted by TDRT ruthenium-based
oxygen sensor (bottom), of the signal emitted from a single 3D resolved plane. 3D resolved image stacks
of these two components can be seen in accompanying movie sequences (Mov 3, Mov 4)
7.4 Conclusion
We have developed an efficient fluorescence and phosphorescence lifetime resolved
imaging system. The system combines temporal focusing two-photon excitation for 3D resolved
wide-field excitation, and highly parallelized frequency domain lifetime resolved measurement
using a nanosecond gated imager. There are two primary limitations of the proposed system.
The first limitation stems from the fact that the peak power of typical titanium-sapphire
lasers is not sufficient to optimally excite many commonly available fluorophores over a large
area, therefore reducing the efficiency of this approach. This low excitation efficiency can be
compensated by using fluorophores of higher two-photon cross section (such as CPNs
demonstrated here). However, the most generalizable approach is using high peak power light
sources (such as regenerative amplifiers) for more efficient wide-field excitation.
In general,
sufficient laser flux should be provided to ensure fluorophores are effectively excited by each
laser pulse without excitation saturation, corresponding to two-photon excitation probability per
fluorophore per pulse of about 10%. Mathematically, two-photon excitation probability can be
expressed as in Eq.(6.2). This equation can be rewritten as to estimate the number of pixels that
can be simultaneously excited per excitation pulse:
134
N oc
Pr r
A0
(7.4)
f
We can further estimate imaging efficiency in terms of the number of pixels that can be excited
per unit time:
rN = Nf oc
p0
(7.5)
For given fluorophore cross section, laser average power and pulse width, the number of pixel
that can be simultaneously excited scales inversely with the laser repetition rate. This is the
reason that higher peak power, lower repetition rate regenerative amplifiers are superior for
simultaneous wide field imaging compared with lower peak power, higher repetition rate
oscillators. However, does wide field imaging (imaging many pixels simultaneously) provide an
imaging speed advantage compared with point scanning or multi-foci scanning methods ? Since
rN is independent of repetition rate, it would appear that it is just as efficient to image a single
pixel at a time as imaging many pixels at the same time. However, this conclusion must be
modified by taking into account of at least three additional time constants of the imaging system.
One of these time constants is the excited state lifetime of the chromophores. For efficient
excitation, the inverse of the laser repetition rate must be much longer in duration than the
fluorophore lifetime. Otherwise, depleting the molecular ground state will not only results in a
reduction of the efficiencies of the subsequent pulses but will also cause resolution degradation.
Given the nanosecond lifetime of most fluorescence probes, this is not a concern for fluorescence
lifetime-resolved imaging. However, for phosphorescence molecules with microsecond to
millisecond lifetimes, the laser repetition rate should be chosen to be sufficiently slow such that
these phosphorescence molecules can be relaxed prior to the next excitation pulse. Therefore,
wide field imaging is almost always more efficient for phosphorescence lifetime-resolved
imaging. Another time constant is the detector "dead time." Photon detectors, especially lifetime
resolved detectors, have a finite "dead time" between subsequent measurements. Depending on
the principles underlying these detectors, the "dead time" is typically on the order of
microseconds. The other relevant time constant is the scan speed of the imaging system.
Depending on the scanning mechanism, stage, galvanometric scanner, acousto-optical scanner,
scan speed quantifies how fast laser light can be directed across individual pixels (point
scanning) or a group of pixels (multi-foci scanning). Scan speed typically ranges from
135
microsecond to millisecond time scale. In general, image efficiency can be increased if pixel
integration time is made substantially longer than the detector "dead time" and the scan speed.
Taking these three time constants into account, more efficient imaging can often be achieved
with wide field imaging mode using a high peak power amplifier for both fluorescence and
phosphorescence modes.
The second major limitation of the current system is that the read-out rate of the camera
is limited to four frames per second. While we have demonstrated that lifetime-resolved images
can be acquired with integration time as short as several milliseconds, the slow read-out rate
limits the actual frame rate to about 4 Hz. This limitation results from the fact that the camera
used in this experiment is not specifically designed for frequency domain heterodyne lifetime
measurement but was custom modified for this experiment. Today, CCD or CMOS cameras can
be readily read out at a rate in excess of 1 GHz. By coupling a high speed gated microchannel
plate with a high frame rate, low noise camera will produce an ideal system that can perform
fluorescence lifetime-resolved measurements in excess of kHz. For phosphorescence imaging,
the integration time must be at least over an order of magnitude longer than the phosphorescence
lifetime of the probe. Nonetheless, phosphorescence lifetime imaging with sub-second frame rate
should be possible for even millisecond lifetime phosphorescence probes.
Finally, wide-field two-photon imaging uses significantly higher power than point
scanning; possible specimen photodamage should be considered. In the absence of one-photon
absorber and thermal damage, the two main mechanisms for photodamage are multiphoton
ionization and oxidative damage. The probability of multiphoton ionization is proportional to
laser flux. With a 3W titanium-sapphire laser, approximately 1W of power is eventually
delivered upon a 100 x 100 ptm 2 specimen surface. The laser flux is approximately 0.1 mW/m 2.
As a comparison, the laser flux of typical single focus scanning is on the order of 10-100
mW/m 2 . As discussed previously, a regenerative amplifier is a better laser source for wide-field
two-photon imaging providing higher laser flux such that the two-photon excitation probability
can reach about 10%. Since both wide-field and point-scanning approaches reach the same
excitation probability level at the same flux, they have the same potential for photodamage. In
terms of oxidative damage, since the amount of reactive oxygen generation is proportional to the
total fluorescence signal, the total dosage of reactive oxygen species generated is also the same
for these two approaches. Of course, since wide-field imaging has higher frame rate, the dose
136
rate is higher for the wide-field case. A priori, it is impossible to determine whether higher dose
rate results in significantly greater photodamage. Future studies are needed to carefully compare
the oxidative damage potentials of two-photon wide field imaging and point-scanning for
different biological systems under study.
References
1. P. I. Bastiaens, P. J. Bonants, F. Muller, and A. J. Visser, "Time-resolved fluorescence
spectroscopy of NADPH-cytochrome P-450 reductase: demonstration of energy transfer
between the two prosthetic groups," Biochemistry 28, 8416-8425 (1989).
2. W. Becker, A. Bergmann, M. A. Hink, K. Konig, K. Benndorf, and C. Biskup, "Fluorescence
lifetime imaging by time-correlated single-photon counting," Microsc. Res. Tech. 63, 58-66
(2004).
3. H. C. Gerritsen, J. M. Vroom, and C. J. de Grauw, "Combining two-photon excitation with
fluorescence lifetime imaging," IEEE Eng. Med. Biol. Mag. 18, 31-36 (1999).
4. E. Gratton, S. Breusegem, J. Sutin, Q. Ruan, and N. Barry, "Fluorescence lifetime imaging
for the two-photon microscope: time-domain and frequency-domain methods," J. Biomed.
Opt. 8, 381-390 (2003).
5. I. Gryczynski, A. Razynska, and J. R. Lakowicz, "Two-photon induced fluorescence of linear
alkanes; a possible intrinsic lipid probe," Biophys. Chem. 57, 291-295 (1996).
6. K. Konig, P. T. So, W. W. Mantulin, B. J. Tromberg, and E. Gratton, "Two-photon excited
lifetime imaging of autofluorescence in cells during UVA and NIR photostress," J. Microsc.
183 ( Pt 3), 197-204 (1996).
7. J. R. Lakowicz, "Emerging applications of fluorescence spectroscopy to cellular imaging:
lifetime imaging, metal-ligand probes, multi-photon excitation and light quenching,"
Scanning Microsc. Suppl. 10, 213-224 (1996).
8. R. M. Clegg, A. I. Murchie, and D. M. Lilley, "The four-way DNA junction: a fluorescence
resonance energy transfer study," Braz. J. Med. Biol. Res. 26, 405-416 (1993).
9. A. A. Deniz, T. A. Laurence, G. S. Beligere, M. Dahan, A. B. Martin, D. S. Chemla, P. E.
Dawson, P. G. Schultz, and S. Weiss, "Single-molecule protein folding: diffusion
fluorescence resonance energy transfer studies of the denaturation of chymotrypsin inhibitor
2," P. Natl. Acad. Sci. U.S.A. 97, 5179-5184 (2000).
10. X. Zhuang and M. Rief, "Single-molecule folding," Curr. Opin. Struct. Biol. 13, 88-97
(2003).
11. K. Konig, A. Ehlers, I. Riemann, S. Schenkl, R. Buckle, and M. Kaatz, "Clinical two-photon
microendoscopy," Microsc. Res. Tech. 70, 398-402 (2007).
12. E. Dimitrow, I. Riemann, A. Ehlers, M. J. Koehler, J. Norgauer, P. Elsner, K. Konig, and M.
Kaatz, "Spectral fluorescence lifetime detection and selective melanin imaging by
multiphoton laser tomography for melanoma diagnosis," Exp. Dermatol. 18, 509-515 (2009).
13. G. Helmlinger, F. Yuan, M. Dellian, and R. K. Jain, "Interstitial pH and p02 gradients in
solid tumors in vivo: high-resolution measurements reveal a lack of correlation," Nat. Med.
3, 177-182 (1997).
137
14. I. P. Torres Filho, M. Leunig, F. Yuan, M. Intaglietta, and R. K. Jain, "Noninvasive
measurement of microvascular and interstitial oxygen profiles in a human tumor in SCID
mice," P. Natl. Acad. Sci. U.S.A. 91, 2081-2085 (1994).
15. L. S. Ziemer, W. M. Lee, S. A. Vinogradov, C. Sehgal, and D. F. Wilson, "Oxygen
distribution in murine tumors: characterization using oxygen-dependent quenching of
phosphorescence," J. Appl. Physiol. 98, 1503-1510 (2005).
16. A. L. Harris, "Hypoxia--a key regulatory factor in tumour growth," Nat. Rev. Cancer 2, 3847 (2002).
17. E. K. Rofstad, "Microenvironment-induced cancer metastasis," Int. J. Radiat. Biol. 76, 589605 (2000).
18. S. M. Evans and C. J. Koch, "Prognostic significance of tumor oxygenation in humans,"
Cancer Lett. 195, 1-16 (2003).
19. M. I. Koukourakis, A. Giatromanolaki, R. A. Brekken, E. Sivridis, K. C. Gatter, A. L. Harris,
and E. H. Sage, "Enhanced expression of SPARC/osteonectin in the tumor-associated stroma
of non-small cell lung cancer is correlated with markers of hypoxia/acidity and with poor
prognosis of patients," Cancer Res. 63, 5376-5380 (2003).
20. M. Weinmann, C. Belka, and L. Plasswilm, "Tumour hypoxia: impact on biology, prognosis
and treatment of solid malignant tumours," Onkologie 27, 83-90 (2004).
21. J. M. Brown and A. J. Giaccia, "The unique physiology of solid tumors: opportunities (and
problems) for cancer therapy," Cancer Res. 58, 1408-1416 (1998).
22. I. F. Tannock and D. Rotin, "Acid pH in tumors and its potential for therapeutic
exploitation," Cancer Res. 49, 4373-4384 (1989).
23. S. Sakadzic, E. Roussakis, M. A. Yaseen, E. T. Mandeville, V. J. Srinivasan, K. Arai, S.
Ruvinskaya, A. Devor, E. H. Lo, S. A. Vinogradov, and D. A. Boas, "Two-photon highresolution measurement of partial pressure of oxygen in cerebral vasculature and tissue," Nat.
Methods 7, 755-759 (2010).
24. Y. Wang, S. Hu, K. Maslov, Y. Zhang, Y. Xia, and L. V. Wang, "In vivo integrated
photoacoustic and confocal microscopy of hemoglobin oxygen saturation and oxygen partial
pressure," Opt. Lett. 36, 1029-1031 (2011).
25. W. Becker, Advanced Time-CorrelatedSingle Photon Counting Techniques, (Springer,
Berlin, 2005).
26. A. Periasamy and R. M. Clegg, eds., FLIMMicroscopy in Biology and Medicine, (Chapman
and Hall, Boca Raton, 2009).
27. K. Suhling, J. Siegel, P. M. Lanigan, S. Leveque-Fort, S. E. Webb, D. Phillips, D. M. Davis,
and P. M. French, "Time-resolved fluorescence anisotropy imaging applied to live cells,"
Opt. Lett. 29, 584-586 (2004).
28. J. Requejo-Isidro, J. McGinty, I. Munro, D. S. Elson, N. P. Galletly, M. J. Lever, M. A. Neil,
G. W. Stamp, P. M. French, P. A. Kellett, J. D. Hares, and A. K. Dymoke-Bradshaw, "Highspeed wide-field time-gated endoscopic fluorescence-lifetime imaging," Opt. Lett. 29, 22492251 (2004).
29. D. M. Grant, D. S. Elson, D. Schimpf, C. Dunsby, J. Requejo-Isidro, E. Auksorius, I. Munro,
M. A. Neil, P. M. French, E. Nye, G. Stamp, and P. Courtney, "Optically sectioned
fluorescence lifetime imaging using a Nipkow disk microscope and a tunable ultrafast
continuum excitation source," Opt. Lett. 30, 3353-3355 (2005).
138
30. J. McGinty, K. B. Tahir, R. Laine, C. B. Talbot, C. Dunsby, M. A. Neil, L. Quintana, J.
Swoger, J. Sharpe, and P. M. French, "Fluorescence lifetime optical projection tomography,"
J. Biophotonics 1, 390-394 (2008).
31. D. M. Owen, E. Auksorius, H. B. Manning, C. B. Talbot, P. A. de Beule, C. Dunsby, M. A.
Neil, and P. M. French, "Excitation-resolved hyperspectral fluorescence lifetime imaging
using a UV-extended supercontinuum source," Opt. Lett. 32, 3408-3410 (2007).
32. S. Kumar, C. Dunsby, P. A. De Beule, D. M. Owen, U. Anand, P. M. Lanigan, R. K.
Benninger, D. M. Davis, M. A. Neil, P. Anand, C. Benham, A. Naylor, and P. M. French,
"Multifocal multiphoton excitation and time correlated single photon counting detection for
3-D fluorescence lifetime imaging," Opt. Express 15, 12548-12561 (2007).
33. D. Magde, G. E. Rojas, and P. G. Seybold, "Solvent dependence of the fluorescence lifetimes
of xanthene dyes " Photochem. Photobiol. 70, 737-744 (1999).
34. G. I. Redford and R. M. Clegg, "Polar plot representation for frequency-domain analysis of
fluorescence lifetimes," J. Fluoresc. 15, 805-815 (2005).
35. M. A. Digman, V. R. Caiolfa, M. Zamai, and E. Gratton, "The phasor approach to
fluorescence lifetime imaging analysis," Biophys. J. 94, L14-16 (2008).
36. N. A. A. Rahim, W. McDaniel, K. Bardon, S. Srinivas, V. Vickerman, P. T. C. So, and J. H.
Moon, "Conjugated polymer nanoparticles for two-photon imaging of endothelial cells in a
tissue model," Adv. Mater. 21, 3492-3496 (2009).
37. A. Nagy, J. Wu, and K. M. Berland, "Observation volumes and {gamma} factors in twophoton fluorescence fluctuation spectroscopy," Biophys. J. 89, 2077-2090 (2005).
38. W. Denk, J. H. Strickler, and W. W. Webb, "Two-photon laser scanning fluorescence
microscopy," Science 248, 73-76 (1990).
39. L. C. Cheng, C. Y. Chang, C. Y. Lin, K. C. Cho, W. C. Yen, N. S. Chang, C. Xu, C. Y.
Dong, and S. J. Chen, "Spatiotemporal focusing-based widefield multiphoton microscopy for
fast optical sectioning," Opt. Express 20, 8939-8948 (2012).
40. E. C. Soller, D. S. Tzeranis, K. Miu, P. T. So, and I. V. Yannas, "Common features of
optimal collagen scaffolds that disrupt wound contraction and enhance regeneration both in
peripheral nerves and in skin," Biomaterials 33, 4783-4791 (2012).
41. L. J. Chamberlain, I. V. Yannas, H. P. Hsu, G. Strichartz, and M. Spector, "Collagen-GAG
substrate enhances the quality of nerve regeneration through collagen tubes up to level of
autograft," Exp. Neurol. 154, 315-329 (1998).
42. I. V. Yannas, J. F. Burke, D. P. Orgill, and E. M. Skrabut, "Wound tissue can utilize a
polymeric template to synthesize a functional extension of skin," Science 215, 174-176
(1982).
43. I. V. Yannas, Tissue and Organ Regeneration in Adults, (Springer, New York, 2001).
44. S. Pelet, M. J. Previte, L. H. Laiho, and P. T. So, "A fast global fitting algorithm for
fluorescence lifetime imaging microscopy based on image segmentation," Biophys. J. 87,
2807-2817 (2004).
45. P. Walczysko, U. Kuhlicke, S. Knappe, C. Cordes, and T. R. Neu, "In situ activity of
suspended and immobilized microbial communities as measured by fluorescence lifetime
imaging," Appl. Environ. Microbiol. 74, 294-299 (2008).
46. J. M. Brown, "Tumor microenvironment and the response to anticancer therapy," Cancer
Biol. Ther. 1, 453-458 (2002).
139
47. G. Mehta, K. Mehta, D. Sud, J. W. Song, T. Bersano-Begey, N. Futai, Y. S. Heo, M. A.
Mycek, J. J. Linderman, and S. Takayama, "Quantitative measurement and control of oxygen
levels in microfluidic poly(dimethylsiloxane) bioreactors during cell culture," Biomed.
Microdevices 9, 123-134 (2007).
48. D. Sud, G. Mehta, K. Mehta, J. Linderman, S. Takayama, and M. A. Mycek, "Optical
imaging in microfluidic bioreactors enables oxygen monitoring for continuous cell culture,"
J. Biomed. Opt. 11, 050504 (2006).
49. D. Sud and M. A. Mycek, "Calibration and validation of an optical sensor for intracellular
oxygen measurements," J. Biomed. Opt. 14, 020506 (2009).
50. D. Sud, W. Zhong, D. G. Beer, and M. A. Mycek, "Time-resolved optical imaging provides a
molecular snapshot of altered metabolic function in living human cancer cell models," Opt.
Express 14, 4412-4426 (2006).
140
Chapter 8
Conclusion
8.1 Thesis summary
In this thesis, we developed novel 3D optical microscope systems that greatly improved
the throughput of high volume biological assays. First, high throughput 3D cell based image
cytometer have been developed based on the widefield structured light illumination and the high
speed remote depth scanning. The sequence of the image acquisition processes are all automated
and enables the large area sample imaging at submicron resolution. Assuming that an 80%
confluency cell culture is imaged, this system can image adherent cells on the slide glass at 800
cell/sec in 3D. The statistical accuracy of this instrument is verified by quantitatively measuring
the rare cell populations down to 1:105 ratio. The developed system can potentially be applied to
image cytometry investigation to study cultured cell morphologies with statistical significance
comparable to the flow cytometer. Hyperspectral imaging is also possible based on the use of an
interferometric full field spectrometer, which enables the full spectrum measurement over the
whole field of view simultaneously. Spectral background rejection algorithm is mathematically
formulated and experimentally demonstrated. The depth resolved hyperspectral imaging would
allows 4D (x, y, z, k) imaging and is expected to provide a more accurate spectral signature of
the biological system.
Second, we developed a depth resolved widefield two photon endomicroscope for the
medical diagnosis based on the temporally focused widefield two photon microscopy. We
proposed an improved method for delivering a high energy pulse through the hollow core
photonic crystal fiber to increase two photon excitation efficiency. We have demonstrated that
the combined strategy of increasing the repetition rate with pulse splitter and reducing the peak
power increase excitation efficiency through the endomicroscope by eleven folds. A structured
illumination method for improving poor depth resolution caused by the low NA microlens used
in these endomicroscopes has demonstrated with resulting resolution comparable to conventional
point scanning multiphoton microscope. In addition, temporally focused widefield two photon
141
microscopy has been developed for high throughput depth resolved fluorescence and
phosphorescence measurement with millisecond level integration time.
8.2 Thesis contribution
Current methods of optical microscopy have limitations in terms of imaging deep into the
tissue, achievable resolution and its throughput. The imaging depth is fundamentally limited by
tissue scattering and the best achievable imaging depth is a couple of millimeter at best. The
resolution is limed by the diffraction of the light and the best achievable resolution is about half
the wavelength of light. The imaging throughput of optical microscopy is much lower compared
to many other medical imaging modalities such as MRI, CT or ultrasound although it provides
much higher resolution.
The current research in optical microscopy are conducted to overcome these limitations.
Tissue scattering problem is partly solved by using optical clearing, adaptive optics and solving
mathematically the light scattering problem in turbid tissue. Optical resolution limit set by the
diffraction of light is now mostly overcome by using various techniques such as stimulated
emission depletion microscopy (STED), photo activated localization microscopy (PALM),
statistical optical reconstruction microscopy (STORM). The throughput of the optical
microscopy especially for depth-resolved imaging has been increased by parallelizing the
imaging process using such techniques as multifocal multiphoton microscopy (MMM), spinning
disk confocal microscopy, structured light illumination microscopy and temporally focused
widefield microscopy. Along with these exciting research directions in the field, this thesis
contributed to improve the throughput of the optical imaging instruments by utilizing the
structured light illumination and the temporally focused microscopy and expanded the frontiers
of the biomedical applications.
142