Security Sphere: Panoramic Image Acquisition
by
Oluwamuyiwa Oluwagbemiga Olubuyide
Submitted to the Department of Electrical Engineering and Computer Science
in Partial Fulfillment of the Requirements for the Degrees of
Bachelor of Science in Electrical Science and Engineering
and Master of Engineering in Electrical Engineering and Computer Science
at the Massachusetts Institute of Technology
May 23, 2001 1
Copyright 2001 Oluwamuyiwa Oluwagbemiga Olubuyide. All rights reserved.
The author hereby grants to M.I.T. permission to reproduce and
distribute publicly paper and electronic copies of this thesis
and to grant others the right to do so.
BARKER
MASSACHUSETTS INSTITUTE
OF TECHNOLOGY
JUL 11 2001
LIBRARIES
Author
Department 4JElectrical Engineering and Computer Science
May 23, 2001
Certified by
Charles G. Sodini
Professor of Electrical Engineering
Thesis Supervisor
~~1
Accepted by
Arthur C. Smith
Chairman, Department Committee on Graduate Theses
Security Sphere: Panoramic Image Acquisition
by
Oluwamuyiwa Oluwagbemiga Olubuyide
Submitted to the Department of Electrical Engineering and Computer Science
on May 23, 2001, in partial fulfillment of the
requirements for the degrees of
Master of Engineering
and
Bachelor of Science
Abstract
The Security Sphere is a panoramic video surveillance device. It is a compact ball
with a spherical six and a half-inch radius whose nominal operating position is hanging
from the central apex of a visual field. The Security Sphere has four imaging locations in
the bottom hemisphere that are aligned to form one panoramic 1800 solid angle image.
Each of the four imagers on the Security Sphere sends 30 visual data frames per second.
Each frame has an image spatial resolution of 256 x 256 and an intensity resolution of 8
bits/pixel, leading to a combined data rate of 2 Mbytes/sec from each imager. The MIT
Technology being showcased in this project is the Complementary Metal Oxide
Semiconductor (CMOS) Differential Passive Pixel Image (DPPI) sensor. This thesis
focuses on the utilization of the CMOS DPPI sensor to implement the Panoramic
Acquisition for a Security Sphere.
The demonstration system was implemented with a single imager sending
imaging data to the display unit. The panoramic capability of the Security Sphere was
demonstrated by manually shifting the imaging location.
Thesis Supervisor: Charles G. Sodini
Title: Professor of Electrical Engineering
2
Acknowledgements
I would like to thank my father for the intense inspiration he has always been for
me throughout my life. In addition, I would like to thank my mother for her advice, and
the puff-puff she sends me to keep me "plump." In addition, I want to thank my
significant other, Natalie Smith, for being there for me through the inevitable ups and
down of a project. My deepest appreciation to Derrick Cordy, who seems to unerringly
know when I need encouragement most. Special thanks go to Professor Tayo Akinwande
for the numerous talks and invaluable advice on project management and planning. I
would also like to thank my group members, Jelena "Mad" Madic and Matthew
"Polishing Pole" Yarosz for their enjoyable company over the last two years. I doff my
hat to Jeremy Hui for his invaluable suggestions over the course of this thesis.
I am forever indebted to my advisor, Professor Sodini, for his constant guidance
and support from introducing me to the Prototyping Research Results class to his
significant assistance in my selection to join the ranks of MIT doctoral candidates. I
would also like to thank our Research Engineer, Souren "Sam" Lefian, who taught me the
importance of cleanup. I would like to thank Fred Cote who believed in the possibility of
making "real engineer" out of an Electrical Engineer. I would also like to thank
Professor James Bales for sharing his deep interest with optics with a fellow scholar.
My deepest appreciation to the Intel Masters of Engineering Fellowship and the
Masters of Engineering Thesis Project Program whose funding made much of this thesis
possible.
3
Dedication
To my father, the late Professor Olusegun Ayokanmi Olubuyide, the lighthouse that
never fails.
4
Contents
I
13
Introduction
2
...... 13
1.1
Motivation ...................................................
1.2
Objective ..........................................
1.3
Thesis Organization ....................................................
........
Background: Differential Passive Pixel Image (DPPI) Sensor
Pixel Design ..................................................
2.1
15
16
2.1.2
Selection and Construction of N-Well Pixel ....................
17
2.1.3
Pixel Readout Format ............................................
19
DPPI Sensor Architecture ..................................................
20
2.2.1
Charge to Voltage Conversion Operation and Circuitry......... 21
2.2.2
Differential Output Voltage Output .............................
22
Operation of the DPPI Sensor ...........................................
26
27
3.1
Lens A nalysis .................................................................
27
3.2
L ens Selection .................................................................
31
3.2.1
4
15
Efficiency of Charged Carrier Collection ........................
Panoramic Image Acquisition
3
14
2.1.1
2.2
2.3
13
T unnel E ffect ..........................................................
34
Electrical Components
4.1
33
Printed C ircuit B oard ..........................................................
5
34
4.2
5
Current Sources .....................................................
35
4.1.2
Analog to Digital Converter ......................................
36
4.1.3
Manual Board Modifications ......................................
41
FPG A B oard ....................................................................
42
44
Mechanical Components
5.1
Lens Mount Design .........................................................
44
5.2
Hemispherical Design .......................................................
45
5.2.1
Right Angle Arc Design ...........................................
46
5.2.2
Bent Right Angle Arc Design .......................................
47
5.2.3
Open Equilateral Triangular Rod Design ........................
48
Open Equilateral Triangular Rod Design Analysis ........................
48
5.3
6
4.1.1
5.3.1
Bottom Lens Mount Analysis .......................................
51
5.3.2
Final Radius of the Hemisphere ......................................
52
5.4
Additional Fabricated Components ........................................
52
5.5
Listing of Mechanical Parts and Tools ......................................
53
54
Labview: Image Acquisition
6.1
D ata F low ........................................................................
54
6.2
Labview Front Panel .........................................................
55
6.3
L abview C ode ..................................................................
57
6.3.1
D ata A cquisition ......................................................
57
6.3.2
Data Processing .....................................................
58
6
7
6.3.3
Image Display .......................................................
59
6.3.4
Intensity Graph vs 8-bit Draw Pixmap ...........................
61
Image Results and Discussion
63
7.1
Nominal Image Quality ......................................................
63
7.2
Images Quality vs. Acquisition Parameters ................................
64
7.3
7.2.1
Image Quality as a Function of Frequency and Lens Aperture . 66
7.2.2
Image Quality as a Function of Drawing Software
7.2.3
Image Quality as a Function of the Lens Physical Specification. 67
7.2.4
Image Quality as a Function of Thermal Effects .................. 67
.............. 66
Panoramic Acquisition .......................................................
68
D iscussion ............................................................
70
7.4
Final Project R esults ...........................................................
71
8
Conclusion
72
7.3.1
8.1
Summary .......................................................................
72
8.2
Future Work .....................................................................
73
Appendix A: Differential Passive Pixel Image Sensor Timing Diagram ...............
74
Differential Passive Pixel Image Sensor Pinout ...........................
76
PSpice Simulation Circuit ..................................................
78
Appendix B: Wide-Angle Lens Specification .............................................
79
7
Image of the H2616FICS-3 ...................................................
79
Dimensions of the H2616FICS-3 ............................................
80
Appendix C: VHDL Code for Xilinx XC401OXL PC 160 FPGA .......................
81
Matlab Code for modeling the THS0842 Input Circuitry ................
86
LM334SMX Pinout ............................................................
87
T HS0842 Pinout ...............................................................
87
Schematic Diagram of the Analog Portion of the DPPI Sensor Board
88
Schematic Diagram of the Digital Portion of the DPPI Sensor Board
89
Final Printed Circuit Board Layout of the DPPI Sensor Board .......... 89
Appendix D: Future Work [IMAQ Code and Cable Design] ..............................
IMAQ Cable Design ..........................................................
90
92
Appendix E: B udg et .............................................................................
94
Referen ces .........................................................................................
95
8
List of Figures
2-1
Hole-Electron Optical Generation ....................................................
15
2-2
Sample P-N Junction Diode ...........................................................
16
2-3
N-Well Pixel (Photodiode) ......................................................
17
2-4
Schematic Diagram of a CMOS charge output pixel ................................
19
2-5
DPPI Sensor Architecture and Physical Representation ...........................
20
2-6
Correlated Double Sampling Sequence ...........................................
21
2-7
Model of Output Circuitry ............................................................
23
2.8a
Pspice Simulation of Vouthi .......................................................
25
2-8b
PSpice simulation of Voutlo ..........................................................
25
3-1
Conceptual Model of Image Area ..................................................
27
3-2a
Panoramic Overlap ...................................................................
28
3-2b
Model of Panoramic Overlap .......................................................
28
3-3
Model of Image Hemisphere ..........................................................
29
3-4
Number of Required Lenses versus Lens Angle ...................................
30
3-5
Barrel Compression with Wide-Angle Lenses ....................................
31
3-5 Relevant Physical Specifications for the H2616FICS-3 Lens........................
31
3-6
Physical Representation of the Security Sphere ....................................
32
3-7a
Tunnel Effect without Image Corona ..............................................
33
3-7b
Tunnel Effect with Image Corona ..................................................
33
4-1
Block Diagram of the DPPI Sensor ...................................................
34
4-2
Configuration of the LM334SMX as a current source ..........................
35
4-3
DC-Coupled Differential Input Circuit ..............................................
37
9
4-4
Flow Representation of Scale and Shift of DPPI Sensor output voltage ......... 38
4-5
Common Mode Model of the THS0842 Input Circuitry ...........................
39
4-6
Representation of FPGA Board .....................................................
42
5-1a
Front Surface of Lens Mount ...........................................................
44
5-1b
Back Surface of Lens Mount ..........................................................
45
5-2
Right Angle Arc Design Model .....................................................
46
5-3
Bent Right Angle Arc Design Model ................................................
47
5-4
Open Equilateral Triangular Rod Model ..............................................
48
5-5
Geometrical Model for Analysis of the Top Three Lens Mounts ................. 49
5-6
Optimal Depth Placement of the Lens Mounts ....................................
50
5-7
Geometrical Model for Analysis of the Bottom Lens Mount .....................
51
6-1
Tracing the Datapath for Image Display ...........................................
54
6-2
Front Panel of Custom VI (ImageTrig.vi) for DPPI Sensor output ............... 55
6-3
Labview Data Acquisition Module ...................................................
57
6-4
Labview Data Processing Module ..................................................
58
6-5
Selecting Valid Data .................................................................
59
6-6
Interleave Function ...................................................................
60
6-7
Labview Image Display Module ...................................................
60
7-1
N om inal Im age Quality ..................................................................
63
7-2a
Image Acquisition at Tint = 131.07ms (3.814 Frames/Second) .................. 64
7-2b
Image Acquisition at Tint = 66.54ms (7.628 Frames/Second) .....................
64
7-2c
Image Acquisition at Tint = 32.77ms (15.256 Frames/Second) ..................
65
7-2d
Image Acquisition at Tint = 16.38ms (30.512 Frames/Second) ...................
65
10
7-2e
Image Acquisition at Tint = 43.69ms (11.442 Frames/Second) ...................
66
7-3a
First 120' Arc [Overlap shown by dotted lines] ......................................
68
7-3b
Second 1200 Arc [Overlap shown by dotted lines] .................................
69
7-3c
Third 1200 Arc [Overlap shown by dotted lines] ...................................
69
7-3d
Bottom Lens [Overlap shown by dotted lines] .......................................
70
11
List of Tables
2-1
Differential Voltage Range for the DPPI Sensor ................................
26
4-1
C urrent Source V alues ...............................................................
34
4-2
THS0842 Input Resistor Values ...................................................
38
4-3
THS0842 Input Parameter Values ................................................
38
5-1
Listing of Mechanical Parts and Tools ............................................
51
7.1
P roject R esults .........................................................................
69
12
Chapter 1
Introduction
1.1
Motivation
Today's portable device atmosphere and low power demands continue to increase
commercial interest in devices such as the CMOS Differential Passive Pixel Image
(DPPI) sensor. This imaging device offers a host of characteristics that are compatible
with today's digital camera market needs: a battery voltage range of 3.3 Volts, low power
consumption, small area, and low noise generation. Furthermore, as a CMOS imager, it
can be integrated with analog and digital functional blocks in a standard CMOS process - leading to significant cost reductions. Thus, the motivation for this project was to
demonstrate these characteristics by creating the Security Sphere, a portable, panoramic
visual surveillance device.
1.2
Objective
The Security Sphere consists of two units, an image acquisition unit and a display unit.
The image acquisition unit of the Security Sphere is a 6.5-inch radius hemisphere with
four mounted CMOS DPPI sensors. The four mounted sensors are aligned on the
Security Sphere to achieve the 1800 solid angle of view. The acquired images from the
sensors are then sent to the display unit. In the primary operational mode, the image
acquisition unit is hanging from the central apex of a visual field, with the image data
being acquired and then sent at 30.512 frames per second to the display unit. The
commercial applications of such a device are numerous, from being placed in movie
13
studios, to service centers, to factories, or even a baby crib-in each situation reliably
monitoring the visual status of the environment.
1.3
Thesis Organization
Chapter 2 gives the background into the Differential Passive Pixel Image (DPPI)
Sensor, including a detailed explanation of the Pixel Design, Architecture, and operation
of the chip. The panoramic application for multiple DPPI Sensors is described in Chapter
3. Chapter 4 captures the hardware and software components involved in wiring the
DPPI Sensor. Chapter 5 outlines the design and fabrication of the mechanical
components that were fabricated to couple the Lens to the Board and the Sphere. Chapter
6 captures the display of the image data from the DPPI Sensor. Chapter 7 displays and
discusses the quality of the images acquired from the DPPI Sensor. Chapter 8
summarizes the project and analyzes the possibility of future work with the Security
Sphere. Appendix A-E contains the physical and financial specifications for designing
and implementing the Security Sphere.
14
Chapter 2
Background: Differential Passive Pixel Image (DPPI) Sensor
2.1
Pixel Design
The most common method used for solid-state imaging consists of converting light into a
measurable electrical quantity such as voltage, current or charge. Although there exists
several different physical alternatives to achieve image sensing, this discussion focuses
on utilizing a photodiode or a reverse-biased p-n junction.
Light
oe
electron 0O
Figure 2-1: Hole-Electron Optical Generation
As shown in Figure 2-1, upon interacting with a semiconducting substrate, a beam of
light carrying energy greater than the bandgap energy generates charged hole-electron
pairs. For silicon, the specific photonic energy required for charge generation is 1.1 eV.
The amount of generated charge further depends on the photon flux, the
wavelength of the incoming light and the absorption coefficient of the silicon surface.
The energy of a photon can be described by:
E = hv;
(2.1)
where h is Planck's constant and v is the frequency of the photon flux. The photonic
energy with respect to the wavelength is:
15
(2.2)
E = hc/X
Setting the photonic energy equal to 1.1eV and solving for the maximum wavelength for
the incoming photon to generate a hole-electron pair gives a value of 1.1 3um. Once the
photon has been absorbed and the electron-hole pair has been generated, the electrons
must be separated from the holes. In addition, since the charge of a single electron is
much too small [~le-19 volts] to be converted into a measurable quantity, it is necessary
to collect the generated charge carriers over a period of time.
2.1.1
Efficiency of Charged Carrier Collection
Depletion
Region
NDope
P Doped
Figure 2-2: Sample P-N Junction Diode
In the sample photodiode drawn above, the separation is achieved with the help of the
electric field in the depletion region. This electric field forces electrons and holes in
opposite directions. Furthermore, since there is a significant loss of free electrons and
holes in the depletion region there is a lowered probability for recombination of the
electron-hole pair. Thus, it is advantageous to increase the width of the depletion region
in order to maximize the collection efficiency of the photodiode.
16
Lower doping concentrations lead to longer depletions regions. Longer depletion
regions yield a larger area for the separation of electrons and holes and a shorter distance
for the minority carriers to travel. The diffusion length, which is the average distance an
electron or hole will diffuse before recombining, also increases with a lower doping level.
Thus lower doping concentrations are advantageous in two ways: longer diffusion
lengths and wider depletion regions.
2.1.2
Selection and Construction of the N-Well Pixel
rowsel
col
n-well
P-Si
Figure 2-3: N-Well Pixel (Photodiode)
The n-well photodiode is a promising implementation for high performance imagers in
sophisticated device technologies for several salient reasons. The n-well implementation
has a lower doping concentration (-5 x 10 16cm-3 ) with a deeper junction than comparable
technologies such as the n-diffusion implementation. Due to the comparable values of
the n-well doping and the p-type substrate, the effective depletion width of the n-well is
doubled. This increase in the depletion region also fortuitously leads to a decrease in the
capacitance of the pixel. The advantage of having a lower capacitance is lower thermal
noise which is characterized by (kTC) 0 5 in the charge domain. The second advantage of
using a lower doped material for charge integration is the minority carrier diffusion
17
length. This value increases by approximately 2 orders of magnitude in the hole and
electron diffusion length for n+ diffusion and n-well photodiodes.
Another difference between the n-well and n-diffusion photodiodes is the depth of
the junction. The deeper n-well junction brings the edge of the depletion region closer to
the range of visible photon absorption, thus improving the quantum efficiency of the
pixel.
The risk of utilizing the n-well photodiode is that the n+ diffusion is necessary to
access the charge collected in the n-well. This n+ diffusion must make contact with the
n-well but it must also serve as the drain of the row select transistor. The n+ diffusion
must also extend outside far enough so as not to short out the row select transistor. This
irregular extension is not permitted under the standard design rules for the n-well process.
Furthermore the design rules for the n-well are stricter than they are for the n+ diffusion.
For instance, the minimum width for the n-diffusion is 3X compared with 1OA for the nwell. In addition the spacing of the adjacent cells of the n-diffusion is more flexible by a
factor of 3 than that of the n-well. As a result, the layout of a n-well photodiode leads to
an area penalty of almost 3 times that of a n-diffusion for comparable fill factors. This
tradeoff in area is well worth the gain in quantum efficiency for the n-well photodiode, a
significant benefit for imager performance in sophisticated CMOS technologies.
18
2.1.3 Pixel Readout Format
rowsel
Vcm
Vpix
Cfb
Cline
reset
....... ......................
.......................
.....................
Vout
Vo ut
Figure 2-4: Schematic Diagram of a CMOS passive pixel
The CMOS passive pixel shown in Figure 2-4 was chosen for this image sensor. This
selection was due to the strict design rules for the n-well photodiode, for which the
voltage and current output pixels would not provide an adequate fill factor for a given
area. In comparison, the passive readout pixel yields high fill factors for n-well pixels.
19
2.2 DPPI Sensor Architecture
D
e
7f7
v
e
r
r
_
sL
_
-
Mr___p
,
x,
D
SDummy Cells
Figure 2-5:
DPPI Sensor Architecture and Physical Representation
The Differential Passive Pixel Image sensor was implemented with a 256 x 256 random
access pixel array displayed in Figure 2-5. The main advantage of this architecture is the
ability to access individual pixels independently. This ability leads to the possibility of
reducing resolution and power by decreasing the number of rows and columns accessed.
Another option is to utilize the DPPI Sensor for a zoom or pan feature, by controlling the
sequential readout of the pixels.
20
2.2.1 Charge to Voltage Conversion Operation and Circuitry
The array shown in the previous section is readout in a fully differential mode. The
differential mode of imaging is helpful in eliminating effects due to substrate bounce and
other common mode disturbances that eventually lead to column-to-column variations.
The dummy cells in Figure 2-5 were placed on adjacent column as the sensing pixel to
minimize the substrate bounce. This minimization of substrate bounce was accomplished
through the use of a correlated double sampling circuit illustrated in Figure 2-6:
Figure 2-6: Correlated Double Sampling Sequence
In the first phase, the circuit elements are reset. During phase 2, the feedback capacitors
Cti and Cf 2 integrate charge from the parasitic currents (Ip1 and Ip2) as well as the pixel
21
and dummy signals, Qpix and Qdum. At the conclusion of phase 2, the charge on the
feedback circuit is:
Qcfbl =
Qpix CI/(C + CPx + QPi
Qcn2 = Qdum C12/(C 12 + Cdum) + Qp2
(2.3)
(2.4)
where Qpi and Qp2 are the charge due to the parasitic currents, Ipi and Ip2 respectively. In
phase 3, the op-amp and column lines are switched and reset. The charge from phase 2 is
still held on the feedback capacitors. During phase 4, which is of the same duration as
phase 2, the feedback capacitors integrate parasitic current only. The final charge is then:
Qcfi = Qpix CI/(C + Cpix) + Qp1 +Qp2
(2.5)
Qcm2 = Qdum C12/(C 12 + Cdum) + Qp2 + Qpi
(2.6)
Since the output voltage of the differential amplifier is proportional to the difference in
charge between Qctb1 and Qcfb2, the effect of the parasitic current is automatically
removed from the output voltage. If it is assumed that the charge transfer ratios (C11/(C 1
+ Cpix)) and (C12/(CI 2
+ Cdum))
are well matched, then the output voltage is:
Vout = [(Qpix - Qdum)/Cfb] [C1/(Ci + Cpix)]
(2.7)
This architecture requires that the array be staggered since pixels on adjacent columns
need to be addressed separately, to avoid capacitive coupling.
2.2.2 Differential Voltage Output
The pixel voltage in equation 2.7 is split into a differential signal, consisting of Vo+ and
Vo-. Vo+ swings from 1.65V to 2.3V and Vo- swings from 1.65V to 1V, with both
voltage swings being dependent on the incident light intensity on the DPPI sensor. These
two signals are then sent through a voltage buffer before being output as Vouthi and
22
Voutlo respectively. The schematic for this output datapath is shown below in Figure 27:
----------------------3.3V
50/0.6
Vo+
48/2
:48/2
:500.
Sf1
Vouthi
25/2
Sf2+
Is1 200/2
200u
200/2
-200/2
3.3V
__
33
5/250/0.6
--
Vo-
100uA
-Vu1
Sfi-S2
-
48/2
24/2
48/2
Input Source Follower Stage
Output Source Follower Stage
Figure 2-7: Model of Output Circuitry
As apparent in Figure 2-7, the output circuitry consists of two cascaded voltage buffers
with an intermediate pass stage between the buffers. Since the DPPI sensor was
implemented with a CMOS technology, the two voltage buffers were achieved using
common-drain amplifiers or source followers. There is one key difference between the
implementation of a source follower as a NMOS or PMOS device that was utilized in the
output circuitry. An NMOS source follower has a higher input voltage relative to the
output voltage. On the other hand, a PMOS source follower has a lower input voltage
relative to the output voltage. Thus, by alternating N/PMOS source followers, the output
voltage of a voltage buffer can be adjusted to approximately equal the input voltage.
This design strategy is reflected in Figure 2.7. For the Vo+ signal, the effective voltage
23
buffer is a cascaded NMOS and PMOS source follower. The Vo- signal, on the other
hand, passes through a cascaded PMOS and NMOS source follower. The rationale for
the reversal of source follower order is due to the differential swing of Vo+ and Vo- in
response to light intensity. Vo+ swings upward in response to light, hence it is
recommended that it should be utilized for an NMOS source follower. If it is utilized for
a PMOS source follower, there is high likelihood that the addition of the upward swing of
Vo+, to the intrinsic upward swing of the PMOS source follower will drive the current
source out of saturation. On the other hand, Vo- swings downward in response to light,
hence it makes sense that it is buffered with a PMOS source follower. If Vo- is buffered
with an NMOS source follower, the downward swing of Vo- added to the innate
downward swing of the NMOS source follower would certainly drive the current source
out of saturation. Thus, Vo+ and Vo- are buffered with NMOS and PMOS source
followers respectively. Following the initial source follower stage is the pass stage.
Fortunately, the pass stage serves as the mux that connects the input source follower to
the output source follower. Hence it does not significantly change the range of the
differential output. After the pass stage, the differential signal can now be readjusted to
roughly match the input voltage signal. Specifically, Vo+ goes through a PMOS source
follower in order to adjust Vouthi to approximately match Vo+. Similarly, Vo- goes
through an NMOS source follower in order to adjust Voutlo to approximately match Vo-.
The output source follower stage is also required to provide buffering of the signal
between the outside world and the mux, while the input source follower buffers the pixel
array from the mux. The biasing of the entire voltage buffer is accomplished through two
external current sources, Isf1 and Isf2. The first current source, Isfi, serves as the bias
24
for the first pair of source followers, Sfl+ and Sfl-. The second current source, Isf2,
serves as the bias for the second pair of source followers, Sf2+ and Sf2-. Finally, the
differences in bias currents and threshold voltages between the NMOS and PMOS source
followers lead to the small difference that is measured between the input and output
voltages. Figures 2-8a&b are a PSpice simulation of the behavior of the output circuitry
for the DPPI sensor. Table 2-1 gives a concise comparison between the specified ranges
for Vouthi and Voutlo and the simulated results.
vo)uth
II
I.~
~
~
~~~..
1...
..
I~
......
I~~.
V.L2
.
.J..~..l.,
.......
Figure 2-8b: PSpice simulation of Voutlo
Figure 2-8a: PSpice simulation of Vouthi
Each of the flat regions in the above figures for Vo+ and Vo- correspond to a pixel
readout period. The solid lines correspond to the time it takes the output circuit to shift to
the correct output voltage. The significant voltage shifting from pixel-to-pixel was
designed to display the reasonable response time of the output circuit for the DPPI
sensor. Although the PSpice simulation was implemented with significant voltage
shifting from pixel-to-pixel, it is unlikely that this would occur with the DPPI sensor,
since it would require a very high spatial resolution from the incident light source.
Therefore, the expected and verified output difference of the DPPI sensor from pixel-to-
25
pixel has a smaller slope relative to the simulation above. As expected, in Figures 28a&b, the voltage matching between the NMOS and PMOS source followers is not exact,
with a +/- 0.2V difference between the input and output voltages.
Differential Voltage Dark Light Signal Bright Light Signal Simulated Specification
2.3V
0.65V
0.65V
Vo+
1.65V
Vouthi
1.85V
2.54V
0.7V
0.69V
Vo1.65V
1V
0.65V
0.65V
0.71V
0.7V
1.52V
0.81V
Voutlo
Table 2-1: Differential Voltage range for the DPPI Sensor
As evident, the expected and simulated voltage ranges of Vouthi and Voutlo agree very
well, with only a slight shift of +/- lOmV for Voutlo and Vouthi respectively. The
PSpice simulation circuit can be found in Appendix A.
2.3
Timing Diagram for the DPPI Sensor
In order to utilize the DPPI Sensor, 27 control signals, four power levels, and three
current sources need to be supplied at the specified sequence and voltage levels. The
template for operating the DPPI Sensor at 30.512 Frames/second is shown in timing
diagrams that can be found in Appendix A. The pinout of the DPPI Sensor can also be
found in Appendix A.
26
Chapter 3
Panoramic Image Acquisition
3.1
Lens Analysis
The most important facet of acquiring a panoramic image is correct selection of the lens.
In order to get a panoramic image, there must be complete visual overlap with the
relevant lenses at the focal length of each lens. In order to ascertain the image overlap
region, first the image area of a lens must be determined. The image area ( V ) a lens
sees is a function of its focal length, angle of view, and curvature of the lens. The lens
curvature gives V the shape of an inverted bowl. This curvature was found to be
negligible (to within 1%) in visual experiments conducted to ascertain the in-plane lens
angle of view. In the following analysis, the curvature of the lens is ignored. Therefore,
the 3-dimensional curved nature of V can be flattened into the 2-dimensional circle that is
displayed in Figure 3-1.
Image Radius
Focal Length
Lens Angle
Figure 3-1: Conceptual Model of Image Area
With this simplification, the image area, Nf, is captured by the following equation:
27
N = n(F x tan (0/2))2
(3.1)
where F is the focal length and 0 is the angle of view of the lens. The overlap
combination of the image area of each lens make up the image area displayed in Figure 32a.
Figure 3.2b: Model of Panoramic Overlap
Figure 3.2a: Panoramic Overlap
Figure 3-2a displays the reflects the underlying assumption that the contiguous lenses are
assumed to be in the same image plane. As shown in Figure 3-2b, the area of each
overlap can be found by placing an inscribed square in each image circular area. The
area of this inscribed square is captured by the following equation:
6= 2(F x tan (0/2))2
(3.2)
where 8 is the area of the inscribed square. To model the complete panoramic image
area, it is assumed that the sides of each inscribed square touches the adjacent squares.
With the number of lenses constrained to be even, the equation for the panoramic image
area is:
p = 8 N + (Fx tan (0/2))2(n-2)(1 + N/4)]
28
(3.3)
where p is the total panoramic surface area, and N is the number of lenses required for
the panoramic acquisition. The additional factor to the right of 8 N is due to the outer
arcs outside the inscribed squares. Equation 3.3 is slightly cumbersome, so for practical,
conservative lower bound approximations to the panoramic image area, the following
equation is utilized:
(3.4)
p-N
The panoramic image area of the lenses must also match the image area corresponding to
the 180 solid angle boasted by the Security Sphere. This 1800 solid angle image area
can be modeled by Figure 3-3. In this model, it is assumed that the lenses are embedded
in the hollow hemisphere, so that the radius of the image hemisphere is the sum of the
focal length of the lens and the radius of the physical hemisphere.
Image Sphere
--
@ Focal Length
Figure 3-3: Model of Image Hemisphere
Therefore, to determine the number of lenses required to cover the surface area of the
image hemisphere, the panoramic image area was set equal to the hemisperical image
area. This methodology is shown in the following equations:
29
2N(F x tan (0/2))2 = 2n(F + Rs) 2
(3.5)
N = n[(F + Rs)/( F x tan (0/2))]2
(3.6)
where Rs is the hemispherical radius. As in evident in equation 3.6 and Figure 3-4, the
number of lenses required decreases tangentially with the angle of view.
Figure 3-4: Number of Required Lenses versus Lens Angle
As shown in Figure 3-4, at 400 angle of view, 130 lenses are required, but at 120' angle
of view, only four lenses are required for a six-inch radius hemisphere! The inevitable
tradeoff is that wide-angle lenses create image and distance distortions. The distance
distortion causes objects to appear farther away and smaller than in real life. As displayed
in Figure 3-5, the image distortion is particularly acute at the edges of the visual frame,
leading to an observable compression of the data.
30
Figure 3-5: Barrel Compression with Wide-Angle Lenses
3.2
Lens Selection
The final lens selected for the panoramic acquisition was the Computar H2616FICS-3.
This CS-Mount lens has a horizontal angle of view of 127.90 and a vertical angle of view
of 98.20. The relevant dimensions for the lens and the physical hemisphere are captured
in Figure 3-6:
Angle of top 3 lenses relative to
the horizontal
A
98.20
H2616FICS-3
127.90
Figure 3.6: Relevant Physical Specifications for the H2616FICS-3 Lens
31
Since a circle spans 3600, and each image has a horizontal area of 127.90, it is
immediately apparent that with three imagers, it is theoretically possible to achieve the
full panoramic capability of the Security Sphere. These three lenses would be spaced
apart at 1200 intervals. The lenses would also have to be inclined at greater than 450
angle, so as to ensure vertical panoramic alignment. After careful analysis it was realized
that three lenses would require a very precise, and accurate alignment process.
Moreover, it would not be a very robust implementation of the Security Sphere without
calibrated mechanical components that are not easily machined. An extra lens was
therefore purchased to alleviate these concerns and guarantee a seamless, panoramic
image at the focal length of the H2616FICS-3. The alignment of the four lenses is
discussed in Chapter 5. The final physical representation of the Security Sphere is shown
below in Figure 3-7:
Figure 3-7: Physical Representation of the Security Sphere
32
3.2.1 Tunnel Effect
A common practice with lenses is to ensure the image format dimensions are greater than
the pixel array dimensions. In the case of the H2616FICS-3, it is the pixel array
dimensions (8mm by 6mm) that are slightly greater than the image format dimension
(6.4mm by 4.8mm). It was assumed that this dimensional mismatch would lead to a
"porthole" or "tunnel" effect--shown in Figure 3-8a--for the images acquired by the DPPI
Sensor. Fortunately, in practice it was seen that the "tunnel" effect was much smaller due
to a light corona that provides enough focused light for the outlying pixels in the DPPI
Sensor array. The actual tunnel effect is displayed in Figure 3.8b.
Acceptable Focus
Image Area
Focus Image Area
Figure 3-8a: Tunnel Effect without Image Corona
Figure 3-8b: Tunnel Effect with Image Corona
The detailed specifications for the Computar H2616FICS-3 lens is attached in Appendix
B.
33
Chapter 4
Electrical Components
4.1
Printed Circuit Board
There are two major boards that were fabricated for controlling the DPPI Sensor. The
first board is the DPPI Sensor board and the second is the FPGA board. The block
diagram representation of the DPPI Sensor board is shown in Figure 4-1:
ICLKI
INPUT:
FPGA
Control
DPPI Sensor
Signals
Lens
A/D
Vouthi
8 bits/pixel
Converter
(THS0842)
VoUtlo
Power
A
AL
t Current Souirces
Figure 4-1: Block Diagram of the DPPI Sensor Board
The input FPGA Control Signals and Power enter the DPPI Sensor Board via two 20 pin
header connectors. The actual signals travel through twisted cables to prevent cross talk
of the high frequency digital signals. The FPGA also provides the clock into the Analog
34
to Digital Converter (ADC). By clocking the ADC with the FPGA, global
synchronization of all the inputs are guaranteed, and accurate selection of the sampling
time of the ADC can be implemented. This feature is important because during the
period a pixel is selected for readout (muxclk), the output operational amplifier integrates
the output charge that was stored in the pixel. It is therefore important to sample the
output voltage only near the end of muxclk, so as to provide adequate time for all the
pixel charge to be integrated. Since muxclk is one of the signals sent from the FPGA, it
is synchronized to the remaining control signals and it is easily assured that the rising
edge of the clock occurs right before the falling edge of muxclk.
4.1.1
Current Sources
The LM334 is 3-terminal adjustable current sources featuring a10,000: 1 range in
operating current, excellent current regulation and a wide dynamic voltage range of IV to
40V. Current is established with one external resistor and no other parts are required. The
initial current accuracy is ±3%. In this specific application of the LM334, it was used as a
basic current source. This configuration is displayed in Figure 4-2:
Vin
TRset
Iset
V
VR
Figure 4-2: Configuration of the LM334SMX as a current source
35
With
(4.1)
Iset = (18/17)*[VR/Rset]
In the DPPI Sensor Board, Rset was implemented with a 200 Q potentiometer so as to
ensure a more robust design. The specific currents for the DPPI Sensor are shown in
Table 4-1:
Current Source
Ibias
Specification (uA)
>250
Measured (uA)
-400
Potentiometer Value (Q)
150
Isfi
Isf2
>200
>100
-300
-300
200
200
Table 4-1: Current Source Values
The pinout of the LM334SMX is attached in Appendix C. The datasheet can be found at
http://www.national.com/ads-cgi/viewer.pl/ds/LM/LM134.pdf
4.1.2 Analog to Digital Converter
The THS0842 is a dual 8-bit 40MSPS high-speed A/D converter. All digital inputs and
outputs are 3.3 V TTL/CMOS compatible. Furthermore, the THS0842 has on-chip
references with a full-scale range of -0.5V to 0.5V or 1V peak-to-peak. The midpoint of
this internal reference voltage is determined by CML, a voltage midway between the
power supply and ground. The A/D converter was utilized in the DC-Coupled
differential input circuit configuration. This configuration is shown below in Figure 4-3.
36
THS0842
Vinp
Ainp
___\
2p
Vbias
3
Rim
Ainm
r
Vinm
2m
Vbias
CML
REFIF
-E-
REFB
Figure 4-3: DC-Coupled Differential Input Circuit
In Figure 4-3 there are five resistors that perform voltage scaling and shifting to match
input differential input to the midpoint of the internal reference circuitry. Normally, the
THS0842 is applied with midpoints of the two differential inputs matched. This situation
cannot be applied to the differential output of the DPPI Sensor. Specifically, the range of
differential output of the DPPI Sensor is: Vinp (1.85V - 2.54 V) and Vinm (0.81V 1.52V). Vinp and Vinm are the source follower outputs designated as Vouthi and Voutlo
in section 2.2.2. The midpoint for Vinp is 2.195V, and the midpoint for Vinm is 1.165V.
Therefore, the midpoints of these two waveforms had to be shifted. Another concern that
had to be addressed was the differential range of the DPPI Sensor output. The
differential output range from the DPPI sensor is from 0.33V to 1.73V or a 1.4V peak-to37
peak swing. Furthermore, the 0.33V corresponds to total darkness for the DPPI pixel,
and the 1.73V corresponds to maximal saturation of a DPPI pixel. For the THS0842, 0.5V corresponds to total darkness, and 0.5V corresponds to maximal saturation. Thus,
the five resistors have to perform two functions, the 1.4V peak-to-peak has to be scaled
by 0.7 to 1V peak-to-peak, while simultaneously shifting the differential input voltage
range of 0.33V and 1.73V to -0.5V and 0.5V respectively. The scaling and shifting of
the output differential voltage from the DPPI sensor is shown in Figure 4-4:
t
Vinp
Vinp
0.69V
1.52V
1.78V
0.48V
1.3V
1.85V
0.33V
yn
- 2.54V
Scale
Entire
0.23V
Shift Vinm by 0.23V
.
independent of Vinp
Diff.Range Vinm 0.5V
Vinm
by 0.7
0.71V
0.57V
0.81V
0.5V
1.78V
Vinp
Vinm
0.48V
Shift Entire
1.3V
Range by
0. 8 V
to match
ADC Input
Range
0.5V
-i
Vinp
Am
0.48V
0.02V
-1.28V
Am
Vinm 0.5V
-A-
THS0842
-0.48V
Figure 4-4: Flow Representation of Scale and Shift of DPPI Sensor output voltage
This relatively independent voltage shifting of Vinp and Vinm while maintaining the
same voltage scaling was a resourceful utilization of the three degrees of freedom given
by the five resistors comprising the input circuitry of the THS0842. It allowed the
38
differential output of the DPPI sensor, which only has a positive difference between Vinp
and Vinm, to utilize the full differential range of an analog to digital converter.
Nominally, with positive swing, a designer can only access half the binary values of the
THS0842. It was realized, though, that if the differential input is shifted correctly, the
designer could access the full differential range of the THS0842. The final configuration
of the THS0842 reflecting this voltage shifting capability was therefore a custom
application of this device. For the following analysis, the input circuitry of the THS0842
can be modeled by the following circuit diagram (Figure 4-5):
Rip
Vbias
R2p
R3
2m
Rim
Figure 4-5: Common Mode Model of the THS0842 Input Circuitry
Note: Differential inputs are shorted to ground
The differential voltage scaling is then represented by the following formula:
Scale =
R3 (R2m+ R2p)(4.2)
[R3|1 (R2m + R2p)] + Rip + R2p
The Input High and Input Low are modeled by respectively by:
Ainp =Vbias *
n
+Vi
R
LR2m+[Rip
R
R2m
( + Rp| R2p)]
Ainm=Vbias* 1 Rp(4.4)
1 R2p +[Rlp||(R3+ R~1| R2m)]
39
(4.3)
The Matlab code used to iteratively solve for the best-fit solution to the above scaling and
voltage shift is shown in Appendix C. The final values for the input resistors and
coupling parameters reflecting the appropriate voltage scaling and shifting are captured in
the following tables:
Resistor
Rip
Value
0
Units
KM
Rim
7.6
KQ
R2p
R2m
R3
8.37
5.23
93.7
K
K_
K_
Table 4-2: THS0842 Input Resistor Values
Parameter
Input Resistance
Scaling Factor
DPPI Differential Swing
THS Input Diff. Swing
Negative Input Voltage
Positive Input Voltage
Calculated
4.9741
0.6098
1.4
0.98
1.8922
0
Measured
5.01
0.61
1.6
0.98
i.92
0
Units
KQ
V
V
V
V
Table 4-3: THS0842 Input Parameter Values
In the final configuration, the THS0842 is always sampling and digitizing data. This is to
reduce the jitter and uncertainty associated with the sampling edge of the ADC.
Therefore, due to symmetry of data valid and invalid regions for this specific
configuration of the DPPI sensor pixel readout (see the Sensor Timing Diagram in
Appendix A), the THS0842 always outputs twice the required data per frame. Half of
this output data is valid and the other half is correspondingly invalid. Fortunately,
software was implemented to differentiate between the valid and invalid output regions of
the THS0842. This software implementation is discussed further in section 6.3.2. The
pinout of the THS0842 is attached in Appendix C. The detailed datasheet can be found
at:
40
http://www-s.ti.com/sc/psheets/slas246a/slas246a.pdf
4.1.3
Manual Board Modifications
Some portions of the board had to be manually rewired for the final configuration
discussed above. There were three major modifications that were made. The first
modification was due to a component switch between current sources. Initially the
LM334MX was to be utilized, but it became unavailable by the ordering time. Hence,
the board had to be modified to accommodate the LM334SMX. This component change
necessitated rewiring power, ground supplies, and the resistor input. The second design
change was the decision to utilize the internal reference circuitry of the THS0842 instead
of the external reference circuitry designed on the DPPI Sensor Board. This design
change was implemented to decrease the inevitable variability a designer incurs with
multiple components. This design change was relatively easy to implement because the
default state of the THS0842 is with an active internal reference circuitry. Furthermore,
this design modification led to cost savings, due to components that did not require
purchasing and timesavings for verifying device functionality. The final design
modification was a simple reroute of one of the FPGA control signals to the clock of the
THS0842.. This modification, discussed in more detail above in Section 4.1, provide a
global synchronization and hence control of all signals on the DPPI Sensor board.
41
4.2
FPGA Board
The Field Programmable Gate Array board representation is shown below in Figure 4.6.
The circular nature of the FPGA board was chosen to facilitate the coupling of this board
to the hemisphere.
Power
CLK
PROM
Xilinx XC4010XL
PC160
FPGA
L
20 Pin
B
~Header
F Power
Figure 4-6: Representation of FPGA Board
The four-inch radius board generates the control signals and power sources that are sent
to the four imagers on the physical hemisphere. Upon powerup, the PROM configures the
FPGA to generate the control signals for the DPPI Sensors. The duration of the control
signals is controlled by the clock, which runs at 32MIHz under normal operating
42
conditions. The generated signals are then sent to the two headers for transmission to the
four DPPI sensor boards. The L Header consists purely of control signals for the DPPI
sensors. The B Header consists of a mixture of control signals and power sources for the
DPPI sensor boards. Finally, grounding is accomplished through individual cables from
each imager board to the numerous ground outputs on the FPGA board.
43
Chapter 5
Mechanical Components
5.1
Lens Mount Design
The design of the lens mount was relatively straightforward. The first major specification
for the lens mount was to provide a stable coupling between the H2616FICS-3 wideangle lens and the Printed Circuit Board (PCB). The PCB was designed with four 4-40
mounting holes, one for each corner of the board, so the coupling to the lens mount was
accomplished with nuts and bolts. The second major specification for the lens mount was
to ensure that the lens could correctly focus on the DPPI sensor. The third specification
was to allow no ambient light besides the lens to illuminate the DPPI sensor. These three
specifications were satisfied with the design shown in Figure 5-1a and Figure 5.1b:
3.5"
3.5"
4.2"4.2
Figure 5-1a: Front surface of lens mount
Figure 5-1b: Back surface of lens mount
In Figure 5-1a, the four mounting holes couple the PCB to the lens mount, and the large
circle represents the tap drilled to CS-Mount the H2626FICS-3 Lens. In Figure 5-1b, the
44
backside of the lens mounts is displayed. There is a 0.8-inch depression for the fully
populated board. The depression also provides optical isolation of the DPPI sensor from
the ambient surroundings. Furthermore, there is an extra 0.2-inch horizontal spacing for
the twisted cables from the FPGA board to exit the lens mount. Moreover, leaving the
backside of the lens mount accessible was a design advantage because modifications can
be made to the ADC portion of the DPPI sensor board while the board is coupled to the
lens mount. This design strategy gave added flexibility in calibrating the coupling of the
DPPI sensor board and the lens mount because changes to the digital output can be
immediately displayed under normal operating conditions. The design of the lens mount
also provides the capability for the coarse focusing of the lens with the DPPI sensor.
This coarse focusing is accomplished with the use of spacers with increments of 0.1
inches. These spacers place the DPPI sensor board roughly in the focal length of the
H2616FICS-3 lens. The final fine focus adjustment is accomplished through the use of
manual focus knob on the H2616FICS-3 with a distance adjustment of +/- 0.1 inches.
The final portion of the lens mount is a hole drilled to mount to the curved wood piece
attached to the physical hemisphere. Each of the four holes solely featured in Figure 5.1a
represents all the variable locations of this mounting hole on each of the four lens
mounts.
5.2
Hemispherical Design
The physical hemisphere is constructed from a 6.5" radius 0.125" thick polycarbonate
sphere that was sawed in half at the horizontal spherical midpoint. The following is a
concise discussion of the three approaches considered for the mounting of the lens mount
onto the hemisphere.
45
5.2.1 Right Angle Arc Design
Figure 5-2: Right Angle Arc Design Model
The first idea was to utilize the four lenses spaced at 90' arcs from one another along the
middle of the hemisphere. This idea offered the advantage of having a much greater
image area overlap between the sensors. This advantage is noteworthy due to the image
compression that occurs at the edges of the image for the wide-angle lens. The greater
the image compression at the edges, the greater the difficulty of image alignment. The
disadvantage of this scheme is the vertical panoramic coverage. Again, the lens will have
to be inclined at an angle greater than 450 in order to achieve seamless panoramic
coverage in the vertical dimension. Due to the curvature of the hemisphere, the angle of
the spacers for the lens mount would be more acute than easily machined. The key
obstacle with this approach though is fitting all four lens mounts in the same circular
plane inside the hemisphere. With the dimensions of the lens mounts (4.2" by 3.5"), it
would require a 7.5" radius sphere, which would render the hemisphere cumbersome for
the boasted portability.
46
5.2.2 Bent Right Angle Arc Design
Figure 5-3: Bent Right Angle Arc Design Model
The second idea was incorporate the advantage of the right angle arc design for image
area overlap, while avoiding the key obstacle of increased size of the hemisphere. The
key idea for this approach was to have the lens mounts in different circular planes.
Specifically two lens mounts will be placed 1800 apart in the same circular plane, while
the remaining lens mounts will be placed at a lower circular plane in hemisphere, also
1800 apart, but shifted by 900 relative to the upper two lens mounts. Although this idea
seemed promising, due to the 3.5 inch height of the lens mount, it would still require a
significant vertical dimension in the hemisphere, that again renders it rather cumbersome
for the portable feature.
47
5.2.3 Open Equilateral Triangular Rod Design
Figure 5-4: Open Equilateral Triangular Rod Model
The design that was finally implemented was an Inserted Triangular Rod Design, in
which the first three lens formed the "faces" of a triangular rod, and the final lens served
as the bottom "face" of the rod. This design approach lent itself to the particular
strengths of the H2616FICS-3, which has a 127.90 horizontal angle of view, so that three
lenses are each placed in the center of three equilateral faces, they would cover a 383.70
horizontal circular plane. As discussed in Section 3.2, the extra 23.70 provides enough
angular overlap for the horizontal image alignment. Furthermore, the final lens at the
bottom of the rod alleviates the difficulties of vertical image alignment. The vertical
image area of the top 3 lenses will extensively intersect the horizontal image plane of the
last lens, providing the required panoramic alignment capability.
5.3
Open Equilateral Triangular Rod Design Analysis
This section proves a more detailed analysis of the final design of the hemisphere, and the
selection of the hemispherical size. A top down look at the first three lens mounts is
shown in Figure 5-5:
48
Distance From Lens Mount to Surface of Hemisphere
R
R
R
Radius
L
Length of lens mount
Figure
5-5: Geometrical model for analysis of the top three lens mounts
In Figure 5-5 above, the radius is a function of the depth into the hemisphere, since the
effective horizontal circles of the hemisphere decrease as the depth is increased. This
relationship is captured in the following equation:
R =
H=
2
(5.1)
&L=R
(5.2)
r2 _
2
where R is the effective radius, r is the radius of the Hemisphere, and y is the depth from
the top of the hemisphere. Combining these equations, the equation for the maximum
area of the total inscribed equilateral triangle was found to be:
33(r 2 _ Y2)
4
(5.3)
Hence, the equation for the maximal volume of an inserted Triangular Rod into
hemisphere as a function of the depth is:
49
3
4
3y x (r2 _
Y
2
)
(5.4)
A plot of equation 5.4 versus depth into the hemisphere (y) is shown in Figure 5-6:
Figure 5-6: Optimal depth placement of the lens mounts
When plotted, the maximum dimensions for an inserted triangular rod into the
hemisphere was found to be: x = 1.4231 * Hemispherical Radius and y = 0.58 *
hemispherical radius. Given the height (y) for the lens mount is 3.5", the minimal
hemispherical radius is 6.1 inches for the maximum possible area to place the three lens
mounts.
50
5.3.1 Bottom Lens Mount Analysis
The last lens mount caps the bottom of the modeled equilateral triangular rod. The major
constraint on this analysis is the required clearance between the top of this lens mount
and the bottom of the top three lens mounts. In order to ascertain this clearance, first the
relationship between the size of the hemisphere and the dimensions of the bottoms lens
mount must be analyzed. This relationship is captured below in Figure 5-7:
Diagonal of Rectangle = Diameter of Circle in Hemisphere
Figure 5-7: Geometrical model for analysis of the bottom lens mount
As shown in Figure 5-7, the greatest depth the last mount can fit is determined by the
point at which the effective horizontal diameter of the hemisphere is equal to the diagonal
length of the 4.2" x 3.5" lens mount. This relationship is captured in equation 5.5:
2x rA2-yA2 = 5.47
(5.5)
For a 6.5-inch radius sphere, the maximal depth occurs 5.9 inches. To allow for the 1.37
inch Lens protrusion, the depth of this last lens mount was decreased to 4.73 inches. This
also gives a clearance of 1.2 inches from the bottom of the 3 lens mounts to the top of the
last lens mount.
51
5.3.2 Final Radius of the Hemisphere
The radius of the hemisphere was chosen to be 6.5 inches. With a 6.5-inch radius, the
height of the lens mounts could be increased to 4.2 inches with adequate clearance. The
top 3 lenses have a clearance 3.25 inches or 1.63 inches greater than the 1.37 inches
required for the H2616FICS-3. The bottom lens also has at least 1.2-inch clearance from
the top lenses with the required 1.37 inches from the bottom of the hemisphere.
Furthermore, because of the inclination of the top three lenses, the final fabricated
hemisphere had a slightly greater clearance than analyzed above.
5.4
Additional Fabricated Components
The FPGA board was mounted onto the hemisphere by utilizing an acrylic circular board
with mounting holes at a four-inch and six-inch radius. The acrylic board also has a
circular 3.5" radius hole in the center. The central hole allows many access points for the
FPGA board to transmit and receive signals from the four internal DPPI sensors. Finally,
this acrylic frame allows for attaching hooks to connect the hemisphere to the ceiling of a
room. In the final design the FPGA board was not mounted directed onto the
hemisphere, but needed a vertical displacement of two inches to provide the required
clearance for the inclined top three lens mounts. This vertical displacement was
accomplished with a transparent 1/8" thick plastic piece. This plastic piece also provided
a reinforcing frame to attach three 2" x
"x
" rectangular metal bars. These bars were
drilled to serve as mounting points for the FPGA board. Finally, wooden pieces were
manufactured to couple the lens mounts to the hemisphere. The front sides of these
52
pieces were machined to fit the internal curvature of the hemisphere, while providing a
flat surface for the corresponding flat surface of the lens mounts. Holes were drilled in
both the lens mounts and hemisphere, and wooden screws attached from both ends. The
wooden pieces also provided the required inclination for the panoramic alignment
discussed in Chapter 3. The inclination was achieved by providing the vertical slope
(shown in Figure 3-6) to the curved surface of the wooden piece. Thus, for complete
contact across the wooden face with the hemisphere, the wooden piece and the attached
lens mount would also have to be correspondingly inclined.
5.5
Listing of Mechanical Parts and Tools
Mechanical Component
Lens Mount
Raw Materials
1.25" Polycarbonate
Tool Utilized
Band Saw
0.375" Spacer
Caliper
Nuts & Bolts
Drill Press
Milling Machine
Tap and Drill
Hemisphere
Band Saw
Caliper
Drill Press
Hand Drill
Hand Punch
0.125" thick Sphere
0.25" Acrylic
Wood Pieces
Wood Screws
Nuts & Bolts
Milling Machine
Table 5-1: Listing of Mechanical Parts and Tools
53
Chapter 6
Labview: Image Acquisition
6.1
Data Flow
Labview Hardware Acquisition & Image Display
Four DPPI
Sensors
Sending data
At 8 bits/pixel
/
32 bits/panoramic
pixel
Hardware Ac quisition
Labview
Digital
SCB-68
Input!
Breakout
Box
Output
Image Display
via
Labviw VI:
Board
(DIO-32HS)
Figure 6-1: Tracing the datapath for Image Display
Figure 6-1 shows the datapath from the hemisphere into the Display unit. The ribbon
cable output from the DPPI sensor board is first routed into the SCB-68 Breakout box.
The SCB-68 serves as an adaptor for the SCSI-68 cable into the DIO-32HS board. From
the Digital 1/0 Board, the data is run through a custom software application that performs
the image display function. The first processing module for this software application is
correct acquisition of the data. The second processing module for this program is then
interleaving of the odd and even fields. The final module is presenting the formatted data
to 2 different drawing functions, each with unique strengths and weaknesses. The two
54
images (one from each drawing function) is then displayed on the front panel of the
custom Labview Virtual Instrument (VI) [ImageTrig.vi] written for the DPPI sensor
output.
6.2
Labview Front Panel
A Labview VI contains two major components: A front panel that serves as the user
interface, and a block diagram that contains the graphical source code of the VI that
defines its functionality. The front panel for the custom VI ImageTrig.vi is displayed in
Figure 6-2:
Intens
ity graph
-1.0
new picture
device (1)
top
left
point(O. 0)
S Cloll
Tight foI
addlitonal triggel
clock s ource
clock frequency
time Ut
F
n sec
File
or Ins tructions s elect
VI Properties
>>
Documentation
INPUT INFO
trigger type
namber
of
pre-trigger points
port data ead
trigger condition
............
I . .............
................................. . . . . . . . . . . .
additional trigger parameters
data pattetn
picture
2
Figure 6-2: Front Panel of custom VI (ImageTrig.vi) for the DPPI sensor output
55
As apparent in Figure 6-2, there are two picture windows for image display. The one on
the left is for the Intensity Graph VI and picture window on the right is for the 8-bit Draw
VI. The relative merits of each drawing function are discussed below in Section 6.3.4.
Below the Intensity Graph are controls for the Image Acquisition. The port list notifies
the ImageTrig.vi of the port from which data is flowing. Data was nominally sent in
through Port 0. The Buffer size tells it the amount of data to read into its intermediate
buffer before passing the data to the next function. Although the amount of valid data for
each frame is 65536 bytes, the buffer size is set to 131072 bytes since in each frame, the
analog to digital converter samples twice the required data. The Clock Source defines
whether an internal clock from the DIO board will be utilized or an external clock is
preferred. An external clock is selected in this case, to provide synchronization with the
data valid region of the analog to digital converter. The time limit states the maximum
amount of time the Virtual Instrument should allocate for the trigger occurrence and data
acquisition. The following equation was utilized to determine the time limit in seconds:
TimeLimit = 65.536msx"
(6.1)
The final control terminal is the Trigger type. Due to the configuration of control signals
on the DPPI sensor board, the trigger type was selected as a Start Trigger on the falling
edge of the external clock. The other display indicator is the Input Info (Port Data Read).
This indicator displays the data read into the VI and stored in the intermediate buffer.
56
Labview Code
6.3
As mentioned above, there is another portion to a Labview VI, the Block diagram that
contains the graphical source code. The code for this specific Virtual Instrument was
segmented into three modules: Data Acquisition, Data Processing, and Image Display.
6.3.1
Data Acquisition
trigger"
trigger type Istart
compare Input lines to the pattern at all timesTwI
aita
pattern
120
vyno chang
number of pre-trigger points
butter
sie
port data read
13
clock freque ncy
device (1
[8
116
p
bc
ont-
ig
C
CO-FIG
4ys
Jin put
tort[time
CLAR
READ
llmilr
s
c
off
kpattern match
external"
T rigger T
Imeout
.T
rtern
r0/
0/
generation w/
external clock
connectrk
Prigger condition
s=ri
4
Figure 6-3: Labview Data Acquisition Module
As shown above in Figure 6-3, the ImageTrig Virtual Instrument begins by activating
Digital Trig Config, which configures the trigger condition for starting a digital pattern
generation operation. Once the trigger occurs, program control is passed to the DIO
Config VI. This VI creates the task identifier and allocates a buffer to hold the scans. It
also assigns a list of ports to the group, establishes the group's direction, and produces
57
Error
the task identifier. Finally, DIO Config VI calls the Digital Buffer Config VI to allocate a
buffer to hold the scans as they are read or the updates are written. Program Control is
then passed to the Digital Buffer Control which starts the input operation that is read into
the internal data buffer of the Digital 1/0 board. The Digital Buffer Read then returns
digital input data from the internal data buffer, to present to the subsequent modules of
ImageTrig VI. Finally, the DIO Clear, clears the digital I/O task associated with the
original task identifier created by the DIO Config VI. If continuous operation is selected,
Digital Trig Config is activated once more and the process repeats.
6.3.2 Data Processing
256
u128
in*
Figure 6-4: Labview Data Processing Module
58
There were two main considerations when programming the Data Processing module for
the ImageTrig.vi. The first consideration was that the ADC samples twice the required
data for a frame. As discussed in Section 4.1.2, this is to reduce the jitter and uncertainty
associated with the valid edge of the ADC converter. Under optimal conditions, utilizing
3 control signals would accommodate the inevitable excess data from the ADC: a frame
sync, a line sync, and a pixel sync signal. The Digital I/O board, unfortunately, supports
only 2 control signals. It was decided that frame sync and the pixel sync signals would
be utilized to latch data into the Digital I/O board, and software would be implemented
for the line sync function. Without the line sync, the Digital I/O Board latches invalid
output data during the Opamp setup time. The extra samples come from the invalid data
region displayed in Figure 6-5, occurring in 128-byte blocks that are separated by 128byte blocks of valid data. Thus, by sampling only the correct 128-byte block and then
skipping the next 128-byte block recursively, the valid data can be recovered from the
data frame. This is procedure is coded above in Figure 6-4 and displayed below in Figure
6-5:
- - -1
Invalid Data Block
Valid Data Block
Invalid Data Block
Valid Data Block
,k7 Shift and then Array Subset Function
Valid Data Block
Valid Data Block
Figure 6-5: Selecting Valid Data
Since the first 128-byte region is an invalid data region, the data valid index is shifted by
128 bytes to compensate and an additional five bits added for the THS0842 propagation
delay. The data is then inputted into the Array Subset VI. This VI returns a portion of
array starting at an input index and containing a user supplied length of elements. The
59
second consideration is that data is outputted from the DPPI sensor in an interlaced
format. The interlacing format is that the odd field of a pixel row is read out then the
even field of the pixel row. To correct the format of the input data, the solution was to
utilize the Interleave 1D Arrays VI. This VI interleaves corresponding elements from the
input arrays into a single output array. Thus, contiguous Odd and Even Valid Data
Blocks were output from two separate Array Subset Functions and input into the
Interleave ID Arrays VI. This procedure is outlined in Figure 6-6. The interleaved data
is then built into a 256 by 256 array by repeating the above steps in a 256-increment loop.
Even Field
Odd Field
I I
3 15
7
9
I
6
4
2
,,,
18
101,
Interleave ID Array VI
Interleaved Field
1
3
12
4
5 1
61 7 1 8 19
110 1
Figure 6-6: Interleave Function
6.3.3 Image Display
left
ear.
Itop
point(0,7 4
picture
2
new picture
FLA
---
Its
- ---
y graph
intensity graph
Co9r=b
FFFF
rl
60
Figure 6-7: Labview Image Display Module
The final module of ImageTrig.vi is the Display module shown above in Figure 6-7. The
first challenge was to create a 256-grayscale table for Labview display virtual instruments
to access. A 256-grayscale table was created by subtracting the value of black from
white, and then incrementing in 1/255 increments of this difference from the value of
black to the value of white. This table was fed to two different display functions, the
Intensity Graph and the 8-bit Draw Pixmap. These two display functions were utilized in
order to gain the strengths of both image display methodologies in the captured image. It
is worth having a quick discussion comparing both display Virtual Instruments.
6.3.4 Intensity Graph vs. 8-bit Draw Pixmap
The Intensity Graph is a virtual instrument with a 256 value range color table. This color
table can therefore correctly handle a 256 range of grayscale data. This 256 range of
values is compressed between between -1 and 1. Although, theoretically, this data
compression should not matter as far as the clarity of the image is concerned, in practice,
the picture from the Intensity Graph has slightly reduced resolution than the 8-bit Draw
Pixmap.vi. On the other hand, the Intensity Graph.vi has two major advantages over the
8-bit Draw. The 8-bit Draw function can only draw a fixed size bitmap, where every
pixel has a fixed size. The Intensity Graph, though, allows interactive manipulation of
the pixel size, so the size of the displayed image can be changed with ease. The second
advantage the Intensity Graph has relative to the 8-bit Draw lies in their respective
mechanisms for image display. When the 8-bit Draw Pixmap function acquires its data, it
61
immediately disposes of the old picture, while simultaneously drawing the new picture.
Unfortunately, at times the 8-bit Draw Pixmap function is not fast enough to prevent the
human eye from seeing the black background that is left after it disposes of a picture.
This leads to the appearance of randomly located black horizontal bars of varying sizes in
the transition from one picture to the next. This phenomenon was dubbed "flicker,"
somewhat incorrectly since the fluctuation in apparent illumination does not have a rate
comparable to the reciprocal of the period of persistence in vision. The Intensity Graph
on the other hand utilizes a slower, but more visually pleasing drawing mechanism. The
old picture is kept in the frame until the new picture is ready to pasted. Furthermore, the
new picture overlays the old picture and then the old picture is removed. This removes
the appearance of flicker, which is one of the listed properties of the Intensity Graph. In
conclusion, the 8-bit Draw Pixmap is the superior display function when capturing still
images where image size is not of significance. On the other hand, if image size is of
importance and/or images are being captured continuously, the Intensity Graph is the
superior display function.
62
Chapter 7
Image Results and Discussion
7.1 Nominal Image Quality
Figure 7.1 is the representation of the Image Quality from the DPPI sensor with
following normal operating conditions: a frame rate of 30.512 Frames/second, lens iris
fully dilated, and high ambient light surroundings. The picture on the right is acquired
using the Draw 8-bit Pixmap.VI and the one on the left with the Intensity Graph.VI.
tens Ity graph
-1.0
Figure 7-1: Nominal Image Quality
63
7.2 Images Quality vs. Acquisition Parameters
The Figures 7.2a - 7.2e are images from the DPPI sensor with varying integration time,
with constant illumination and environment (Tint represents the charge integration time).
ens ity
graph
1 .0
-0.0
-1.
Figure 7-2a: Image Acquisition at Tint = 131.07ms (3.814 Frames/Second)
-ens ity graph
-0.0
-1.0
= 65.54ms (7.628 Frames/Second)
64
ns ity graph
-1 .0
-0 .0
Figure 7-2c: Image Acquisition at Tint = 32.77ms (15.256 Frames/Second)
nslty graph
-1
.0
-0.0
-1
.0
Figure 7-2d: Image Acquisition at Tint = 16.38ms (30.512 Frames/Second)
65
ensIty graph
-1 .0
--0.0
Figure 7-2e: Image Acquisition at Tint
43.69ms (11.442 Frames/Second)
7.2.1 Image Quality as a function of Integration Time and Lens Aperture
In Figures 7.2a - 7.2e, it is clear that as the integration time is increased, the pixels
become saturated and image contrast disappears. In the above image acquisition, the iris
of the H2616FICS-3 was fully dilated, and hence the DPPI sensor was fully illuminated
with the ambient light. With this iris position and ambient condition, Figure 7.2e (Tint =
43.69ms) delivered the best contrast and resolution.
7.2.2 Image Quality as a function of Drawing Software
Although not immediately obvious, as discussed in Section 6.3.4, the image quality
displayed by the 8-bit Draw Pixmap is superior to that of the Intensity Graph. It is hard to
discern with the human eye, but the image quality of the 8-bit Draw Pixmap is crisper
than the Intensity Graph. This contrast in image quality is due to the drawing mechanism
of the Intensity Graph, which is an averaging function that eliminates much of the high
66
frequency components of an image. The 8-bit Draw Pixmap on the other hand, draws the
raw image data into a bitmap, preserving all the frequency components.
7.2.3
Image Quality as a function of the Lens physical specifications
The expected barrel compression shown in Figure 3.5 is also present in Figures 7.2a 7.2e. This barrel effect is particularly apparent in the curved representation of the straight
cabinet in all the above figures. On the other hand, the tunnel effect that was expected
due to the image format of the H2616FICS-3 is not present, because of the extensive light
corona that is existent in every non-coherent light source.
7.2.4 Image Quality as a function of Thermal Effects
If the DPPI sensor is kept continuously running thermal effects start becoming evident.
In the selection of polycarbonate as the base material for the lens mounts (see Table 5-1),
heat sinking was not considered as an important factor. Polycarbonate, unfortunately
does not dissipate heat very well, hence the DPPI sensor slowly heats up. Thermal
generation, as well as optical generation, of hole and electron pairs is quite feasible,
hence a fraction of the trapped heat goes into hole and electron pair generation. Thus, the
pixel output charge becomes the sum of both optically and thermally generated charge.
This extra charge initially shows up as slightly brighter lines in the image, that
progressively become more delineated. After approximately 10 hours, these lines
become fully saturated and the thermal effects appear as straight white lines on the
display unit.
67
7.3 Panoramic Acquisition
Figures 7.3a - 7.3d were taken with the full Security Sphere activated. The hemisphere
was hanging 7 feet from the ground for these images. For each pair of identical images,
the one on the left shows the panoramic overlap with the other pictures, and the one on
the right shows the image uncluttered with text and curves. Since the images taken in
Section 7.2 gave the sharpest resolution with an integration time of 43.69ms, the iris fully
dilated, and normal ambient light conditions, all the following pictures were also taken
under the same conditions.
Intensity graph
Overlap with Figure 7.2c
T"
r'
----
-3
'21.0
Figure 7-3a: First 120* Arc [Overlap shown by dotted lines]
68
insIty graph
- - 10
-0.0
-
-1
.0
Figure 7.2c
Figure 7-3b: Second 120' Arc [Overlap shown by dotted lines]
nsIty graph
i
T
-0 .0
h Figure 7.2a
7.2d
Figure 7-3c: Last 1200 Arc [Overlap shown by dotted lines]
69
ensity graph
.0
-1
ierlap with Figure 7.2a
_
:)verlap with Figure 7.2c
Overlap with Figure 7.2b
Figure 7-3d: Bottom Lens [Overlap shown with dotted Lines]
7.3.1
Discussion
As analyzed in Chapter 3, the Security Sphere does achieve the panoramic acquisition
feature. The images in Figures 7.3a - 7.3d have a total image area overlap ranging from
12.5% to 60% that yield a 180* solid angle of view. Furthermore, following the analysis
of Section 5.2.3, it is not surprising that the bottom lens has the greatest overlap image
area, since it accomplishes the vertical alignment of the three top lens. One of the
immediate advantages of the panoramic alignment displayed in Figures 7.3a - 7.3d is that
all the images have most of one image edge overlapping with another image. The fusing
of the images into a coherent whole will have to utilize the following algorithm:
"
Flattening out the image compression at the edges of the each image
*
Aligning the images into a fishbowl 3-dimensional curve
*
Flattening the 3-dimensional curved image into a 2 dimensional image
70
7.4 Final Project Results
Parameter
# of functional imagers
Frame Rate
Spatial Resolution
Intensity Resolution
Image Acquisition
Panoramic Coverage
Cost
Target Specification
4
30 frames/sec
256 x 256
8 bits/pixel
Simultaneous
1800 Solid Angle
$1666.66
Achieved
4
30.512 frames/sec
256 x 256
8 bits/pixel
Sequential
1800 Solid Angle
$1346.64
Table 7-1: Project Results
All the target specifications were met except simultaneous acquisition of images from all
four sensors. The main obstacle in achieving this specification was the Labview Digital
I/O Board that was utilized for data acquisition from the Hemispere. This board has a
bandwidth limitation of 8Mbytes/sec and for sustained operation, 4Mbytes/sec. Since
each DPPI sensor has a sustained data rate of 2Mbytes/second, it was decided to only
display the images from the DPPI sensors individually. If the image captured by another
DPPI Sensor is required for viewing, the input connector to the SCB-68 breakout box is
manually transferred from the current DPPI sensor to the new sensor. With regards to
future work, Labview offers another hardware and software package known as the
National Instruments Image Acquisition (IMAQ) Board that is expressly designed for
displaying images from standard and custom cameras. This hardware board also has a
sustained input bandwidth of 100Mbytes/s. Furthermore, this hardware and software unit
has been recently acquired, and the code to integrate the Security Sphere with IMAQ has
been written. The code is attached in Appendix D. IMAQ will allow simultaneous
display of the panoramic video coverage of the full Security Sphere system.
71
Chapter 8
Conclusions
This thesis has described the design and implementation of a panoramic, video
surveillance device, dubbed the Security Sphere, that provides the hardware to serve as a
demonstration platform for the Microsystem Technology Laboratory's CMOS
Differential Passive Pixel Image sensor. This chapter summarizes the project and
presents suggestions for future work.
8.1 Summary
The CMOS Differential Passive Pixel Image sensor offers a host of characteristics that
are compatible with today's digital camera market needs: a battery voltage range of 3.3
Volts, low power consumption, small area, and low noise generation. Furthermore, as a
CMOS imager it can be integrated with analog and digital functional blocks in a standard
CMOS process leading to significant cost reductions. Due to ease of implementation and
time constraints, all of the additional circuitry for the Security Sphere system was built
with discrete, commercially available components. All the specific requirements for
demonstrating the CMOS DPPI sensor have been met. The only requirement left for
demonstrating the full Security Sphere system is simultaneous panoramic image
acquisition-which will be implemented with a new data acquisition device.
72
8.2 Future Work
Future work could be done to maximize use of the random access capability of the DPPI
sensor. This option can lead to a capability for panning and zooming with the Security
Sphere device. Utilizing a smaller FPGA board and DPPI sensor board can also shrink
the entire Security Sphere system. This particular avenue is particularly feasible due to
the excessive amount of unpopulated real estate on both boards. This system scaling
would theoretically allow the hemisphere to become a three-four inch radius ball, further
boosting the portability. Finally, simultaneous panoramic acquisition can be
accomplished with an adequate data acquisition device.
73
Appendix A: Differential Passive Pixel Image Sensor Timing Diagram
Operational Amplifier Setup Time and Pixel Integration Time:
Timing Diagram(0 - 32us)
Control Signal
FsI, Phip
Fs2
F216.5us
Fin
15usL
Fin4us
12us
0.5uss
Rstclk
3s
F1, Frt, Phirst
F2, FOT
4.5us
5.5us
Rsclk
Pixel Readout Period:
Control Signal
62.5ns
Timing Diagram(32us - 33.25us Period)
187.5ns
Muxclk
Muxenable
93- 75ns
.1
56.25s
MuxO
2500ns
2ns500ns
lus
Mux2
74
Ilus
Timing Diagram(32us - 64us Period)
Control Signal
Mux2
Mux3
Mux4
4us
4us
Mux5
8us
8us
Mux6
16us
16us
I
Readout of the 256 x 256 Pixel Array:
Timing Diagram (Ous - 512us Period)
Control Signal
16us
48us
Mux6
RsO
64us
I64us
128us
128us
Rsl
256us
256us
Rs2
Timing Diagram (Oms - 16.384ms Period)
Control Signal
512us
Rs3
1.024ms
Rs4
Rs5
Rs6
Rs7
2.048ms
2.048ms
4.096ms
.4.096ms
8.192ms
8.192ms
75
Timing Diagram (Oms - 32.768ms Period)
Control Signal
8.192ms
8.192ms
Rs7
16.384ms
Rs8
I
16.384ms
Differential Passive Pixel Image Sensor Pinout:
Pin Name
GND
GND
GND
GND
GND
GND
GND
GND
GND
GND
GND
Imager Pin #
3
18
53
57
67
72
74
76
78
115
120
Pin Location
Bi
Hi
N10
N12
K12
H11
H13
Gl
F13
B4
Al
VLL (ANALOG)
VLL (ANALOG)
VLL (ANALOG)
VLL (ANALOG)
16
70
77
118
G3
J12
G13
C4
Power:
Power:
Power:
Power:
1.65V
1.65V
1.65V
1.65V
VCM
VCM
VCM
VCM
(DIGITAL)
(DIGITAL)
(DIGITAL)
(DIGITAL)
55
69
79
116
M10
K13
F12
A3
Power:
Power:
Power:
Power:
1.65V
1.65V
1.65V
1.65V
VDD (ANALOG)
VDD (ANALOG)
VDD (ANALOG)
68
73
117
Ji
H12
A2
Power: 3.3V
Power: 3.3V
Power: 3.3V
VDD:DHI (DIGITAL)
VDD:DHI (DIGITAL)
VDD:DHI (DIGITAL)
VDD:DHI (DIGITAL)
VDD:DHI (DIGITAL)
17
71
75
80
119
Gi
J13
G12
Fli
B3
Power:
Power:
Power:
Power:
Power:
76
Type
Power: OV
Power: OV
Power: OV
Power: OV
Power: OV
Power: OV
Power: OV
Power: OV
Power: OV
Power: OV
Power: OV
3.3V
3.3V
3.3V
3.3V
3.3V
Pin Name
LPOPONB
OPONB
SFADD3
SFADD4
SFADD5
SFADD6
SFCLK
Imager Pin #
100
101
96
95
94
93
92
Pin Location
B9
A9
Al l
Bi1
Clo
A12
B12
Type
Static: OV
Static: OV
Static: OV
Static: OV
Static: OV
Static: OV
Static: OV
ALLRSTONB
SFALLON
13
91
F2
Cli
Static: 3.3V
Static: 3.3V
EVEN
Fl
F2
FIN
FOT
FRT
FRST
FS1
FS2
MUX
MUX1
MTUX2
MUX3
MUX4
MUX5
MUX6
PHIP
PHIRST
RSO
RS1
RS2
RS3
RS4
RS5
RS6
RS7
RS8
RSCLK
RSTO
RST1
RST2
RST3
RST4
RST5
RST6
RST7
99
103
102
107
104
105
106
108
109
83
84
85
86
87
88
89
19
15
21
22
23
24
25
26
27
28
29
20
12
11
10
9
8
7
6
5
A10
B8
C8
A7
A8
B7
C7
A6
B6
D13
Ell
D12
C13
B13
Dll
C12
H2
G2
J1
J2
Ki
J3
K2
Ll
Ml
K3
L2
H3
F3
Eli
E2
Dl
E3
D2
Cl
C2
NOT(RSO)
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
RSO
RS1
RS2
RS3
RS4
RS5
RS6
RS7
77
Pin Name
RST8
RSTCLK
Imager Pin #
4
14
Pin Location
D3
F1
Type
NOT(RS8)
Dynamic
IBIAS
ISF
ISF2
98
97
63
C9
B10
M13
Current Source
Current Source
Current Source
Pin Name
Voutlo
Vouthi
Imager Pin #
56
54
Pin Location
N11
L9
Type
Analog Output
Analog Output
PSpice Simulation Circuit:
3
+3,v3+
1I
MbreakND
M1
.O
Mbre@kE
Mbre4kt
.MbreaYD
*
MV
7~M16
Mbrea D
M
M
MbreakPO
Mbrek
+33v
3.3v
0
M
12MI
200u
MbreakND
D
~rA.
*
DreibrakND
-
Mbrea
Mbreak
+3,3v
M19
____
M1
Mbreak
M6
Mk
-
M18
MbreakPD
M
M
Mbrea
100u[-.
I
00
78
Mbr akND
M'
Mbr ak
M
d
Mbr a'
j
Appendix B: Wide-Angle Lens Specifications
H2616FICS-3
2.6mm F1.6
for 1/2" format cameras
CS-Mount
Model No.
Focal Length
Max. Aperture Ratio
Max. Image Format HxVxD
Iri
Operation Range
H2616FICS-3
2.6mm
1:1.6
6.4 x 4.8 x 8mm
F1.6 - 110
0.1 m - Inf.
40.9x23.1 cm
Focus
Object Dimension
at M.O.D.
D
H
Angle of View
V
(degrees)
Operating Temperature
Effective
Front
Lens Aperture Rear
Back Focal Length
Flange Back Length
Mount
Filter Size
Dimensions
Weight
__
1/2"
23.2mm(diameter)
7.5mm(diameter)
7.6mm
12.5mm
CS-Mount
34.5(diameter)x34.7mm
45g
152.8
127.9
98.2
-200 - +500
M.O.D.
Image of the H2616FICS-3
79
Minimum Object Distance
Dimensions of the H2616FICS-3
oil
..I
d
e
I__________
1.7
I-.
80
33.0
5,2
Appendix C: VHDL Code for Xilinx XC4010XL PC160 FPGA
Library synplify;
Use synplify. attributes.all;
Library IEEE;
Use IEEE.stdlogic1 164.all;
Use IEEE.stdlogicjunsigned.all;
Use IEEE.stdlogic arith.all;
Library xc4000;
Use xc4000.components.all;
Entity SphereBoard is
Port
Clk32MHz:
ADC2:
ADC3:
ADC4:
Framesync:
REQI:
REQ2:
fs1,
fs2,
fin 1,
frst,
fl,
f2,
rsclk,
rstclk:
muxclk,
muxenable:
Select Sequence
Co:
Ro:
Rs<0:8>
EVEN,
RST8:
FLIP:
muxen2:
latch:
rowlatch:
IN
IN
IN
IN
OUT
OUT
OUT
stdjlogic;
stdjlogic;
stdilogic;
stdlogic;
stdjlogic;
std~jogic;
stdjlogic;
--
-
Clock for implementing Colunm Select
Clock for A/D Converter 2
Clock for A/D Converter 3
Clock for A/D Converter 4
Syncs the FRAME for Labview
Request Valid for Data from ADC 1 & 2
Request Valid for Data from ADC 3 & 4
OUT
stdjlogic;
--
Signals for the opamp setup
OUT
stdjlogic;
--
Mux Clock and Enable for the Column
OUT
OUT
stdjlogic-vector(6 downto 0);
stdlogicvector(8 downto 0);
-----
--
OUT stdjlogic;
INOUT stdjlogic;
INOUT stdjlogic;
OUT stdjlogic;
OUT std-logic;
-----
81
-
Mux<0:6> signals
Row select signals,
Inverted Signals for Column Selected and
Pixel Reset
Signals change in SRAM status
Clock for AD Converter
Signal to latch the data into Labview
when the ADC data is validss
End Entity SphereBoard;
Architecture behave of SphereBoard is
Signal count: stdjlogic vector(10 downto 0); --integer range 0 to 2047;
--
count will track the Global
-- Clock
Signal count2: stdlogic-vector(9 downto 0); -- count will track Column Select
Signal count3: stdlogic-vector(9 downto 0):= "0000000000";
--
count2 will track Row
DPPI CONTROL
Process
Begin
Co <= count2(9 downto 3);
Ro <= count3(8 downto 0);
-- set the value of Ro to count2
EVEN <= (NOT count3(0));
--
set the value of Co to count
-
set the value of EVEN to NOT
RST8 <= (NOT count3(8));
-
set the value of RST8 to NOT RS8
RSO
IF count < 511 THEN
finI <= '1';
END IF;
Wait until falling-edge(Clk32MIHz);
-- initialize fini
count <= count + 1;
REQI <= (Clk27MHz and ADC2);
REQ2 <= (ADC3 and ADC4);
IF ((count3 = "0000000000" or count3 = "1000000000") and count < 511) THEN
Framesync <= 0';
ELSE
Framesync <= '1';
END IF;
IF ((count3 = "0000000000") and (count <= 1028)) THEN
muxen2 <= U';
ELSE
CASE count(1 downto 0) IS
WHEN "00" => muxen2
U'
WHEN "01" => muxen2
'1';
WHEN "10" => muxen2
'1';
WHEN "11" => muxen2
U';
END CASE;
END IF;
82
----
Goes hi and
samples at
count4 = 2 and 6
IF (count >= 1052 or ((NOT(count3 = "0000000000") and count <= 31))) THEN
IF count(3) = '1' THEN
rowlatch <= '1';
ELSE
rowlatch <= 0';
END IF;
IF count(2 downto 0) = "100" THEN
latch <= '';
ELSE
latch <= '1';
END IF;
ELSE
latch <= 0';
rowlatch <= 0';
END IF;
IF count >= 1023 THEN
-- check to see if in the output region, if so
count2 <= count2 + "0000000001";
Co <= count2(9 downto 3);
-- increment count
set the value of Co to count
IF count(2 downto 0) = "111" THEN--IF count4 = 0 THEN
-- checking to see if muxclk
muxclk <= '1';
-- should be enabled
ELSE
muxclk <= 0';
END IF;
IF count(2 downto 0)/= "110" AND
count(2 downto 0)/= "111" AND
count(2 downto 0)/= "000"THEN
THEN
muxenable <= '1';
ELSE
muxenable <= 0';
END IF;
IF (count4 = 5) OR (count4 =6) THEN
muxen2 <= 0';
ELSE
muxen2 <= '1';
END IF;
IF count4 = 7 THEN
count4 <= 0;
ELSE
count4 <= count4 + 1;
END IF;
END IF;
83
--
----
---
--
IF (count4 > 1) and
(count4 < 7)
check to see if
muxenable should be
enabled
latch the output
during the last 31.25ns of
muxenable being '1'
check to see if inner cycle of 250ns should reset
IF count < 31 THEN
fsI <= 0';
frst <= '';
fi <= 0';
END IF;
----
IF (count >= 31) and (count < 511) THEN
fsI <= '1';
END IF;
IF (count >= 31) and (count < 127) THEN
frst <= '1';
fi <= '1';
END IF;
IF count < 47 THEN
rstclk <= 0';
END IF;
IF (count >= 47) and (count < 63) THEN
rstclk <= '1';
END IF;
IF count >= 63 THEN
rstclk <= ';
END IF;
IF count >= 127 THEN
fl <= 0';
END IF;
IF (count >= 127) and (count < 527) THEN
frst <= 0';
END IF;
IF count < 143 THEN
f2 <= '0';
END IF;
IF count >= 143 THE?
f2 <= '1';
END IF;
IF count < 175 THEN
rsclk <= 0';
END IF;
IF (count >= 175) and (count < 191) THEN
rsclk <= '1';
END IF;
IF count >= 191 THEP
rsclk <= 0';
END IF;
IF count >= 511 THEP
fsl <= 0';
END IF;
IF (count >= 511) and (count < 639) THEN
84
below are the sequences in
which the control signals
will be generated
finI <= ';
END IF;
IF count < 527 THEN
fs2 <= 0';
END IF;
IF (count >= 527) and (count < 1007) THEN
fs2 <= '1';
END IF;
IF (count >= 527) and (count < 623) THEN
frst <= '1';
END IF;
IF count >= 623 THEN
frst <= '';
END IF;
IF count >= 639 THEN
finI <= '1';
END IF;
IF count >= 1007 THEN
fs2 <= ';
END IF;
IF ((count =0) or (count = 1)) THEN
FLIP <= ';
ELSE
FLIP <= '1';
END IF;
-- if end of 64us cycle, reinit all the vars except count3
IF count = 2047 THEN
IF (count3(8 downto 0) = "111111111") and
-- check to see if at the end of
count2 = "1111111111" THEN
count3 <= "1000000000";
-
a 32.768ms cycle
count3 <= count3 + "0000000001";
--
incrementing count3
Ro <= count3(8 downto 0);
<= (NOT count3(0));
EVEN
<= (NOT count3(8));
RST8
-
<= "00000000000";
<= "0000000000";
<= 0;
<= ';
<= 0';
<= '1';
--
increment or reinitialize the
value of Ro, EVEN,
RST8, count, count2
muxclk, muxenable,
and muxen2
ELSE
END IF;
count
count2
count4
muxclk
muxenable
muxen2
END IF;
-
-
END Process;
END behave;
85
Matlab Code for modeling the THS0842 Input Circuitry
function [Req] = par(Rl, R2)
% Function that outputs the parallel
% output resistance of two resistors
% Written by Oluwamuyiwa Olubuyide February 15th, 2001
Req = (Rl*R2./(Rl+R2));
% Matlab Script that models the following parameters
% for the THS0842 input circuitry:
% Compression, Equivalent Input Resistance,
% Positive Input bias, Negative Input bias
% Written by Oluwamuyiwa Olubuyide February 18th, 2001
% Required Input Values for the 5 resistors
Rlp=0;
R2p=8370;
Rlm=7600;
R2m=5230;
R3=93700;
% Formulas for Input Circuit Parameters
compress=(par(R3, (R2m+R2p)))./(par(R3, (R2m+R2p))+Rlp+R1m);
Reqm=R2m+par(R1m,(R3+par(R2p,Rlp)));
Reqp=R2p+par(Rlp,(R3+par(R2m,R1m)));
Ainm=3.3*(1-R2m/Reqm);
Ainp=3.3*(1-R2p/Reqp);
Req=par(Reqm,Reqp);
darkp=1.65+Ainp;
darkm=1.3+Ainm;
86
LM334SMX Pinout
NAME
V+
NO
3
Description
Input Voltage
V-
2
Output Voltage
R
1
Resistor Input. Determines Output Current.
4
NC
-
No connect
8
THS0842 Pinout
NAME
AVDD
NO.
27, 37, 41
Analog ground
BG
28, 36, 40,
46
29
0
CLK
47
I
CML
32
CouTr
26
NOT Cour
25
0
0
0
DB7 -DBO
4 -11
0
DRVDD
DRVss
1, 13
12, 24
I
I
DA7 -DAO
16-23
I
AVss
DVss
I1+
NC
45
43
39
38
2, 3, 14,
NOT OE
48
48
NOT
DVDD
33
PWDNREF
I_
Description
Analog supply voltage
1/0
I
Band gap reference voltage
Clock input. The input is sampled on
each rising edge of CLK
Common mode level. This voltage is equal
to (AVDD - AVSS)/2
Latch clock for the data outputs
Inverted latch clock for data outputs
Data outputs. D7 is MSB. This is the second bus.
Data is output from the Q channel when dual bus
output mode is selected. Pin SELB selects the output
mode.
Supply voltage for output drivers
Ground for digital output drivers
Data outputs. D7 is MSB. This is the primary bus.
Data from both input channels can be output on this
bus or data from the I channel only. Pin SELB selects
the output mode.
Digital supply voltage
I
Digital ground
I
Negative input for analog channel 0
I
Positive input for analog channel 0
I
No connect. Reserved for future use
15
I Output enable. A high on this terminal will disable the
bus
O Ioutput
Power down for internal reference voltages.
I A high on this terminal will disable the internal reference
_IIcircuit.
87
QQ+
NO.
35
34
1/0
I
I
REFB
30
I/O
REFT
31
I/0
NAME
44
SELB
STBY
Y
42
42
Description
Negative input for analog channel 1
Positive input for analog channel 1
Reference voltage bottom. The voltage at this
terminal defines the bottom reference voltage for the
ADC.
Reference voltage top. The voltage at this
terminal defines the top reference voltage for the ADC.
Selects either single bus data output or dual bus data
output.
I
A low selects dual bus data output
I Standby input. A high level on this terminal will power
Idown
the device.
Schematic Diagram of the Analog Portion of the DPPI Sensor Board:
88
Schematic Diagram of the Digital Portion of the DPPI Sensor Board:
Final Printed Circuit Board Layout of the DPPI Sensor Board:
89
Appendix D: Future Work [IMAQ Code and Cable Design]
CameraFile (4.10) {
Type (Digital)
Manufacturer (MIT)
Model (DPPI)
CreationDate (20010413110957)
InterfaceInfo (1424) {
AcquisitionWindow (0, 0, 256, 256)
BinaryThreshold (128)
BitDepth (8)
FrameTimeout (5000)
LUT (Normal)
LUTTransform (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
MaxImageSize (256, 256)
Scaling (0)
ScanType (Area)
DetectPClk (Yes)
HSyncBlanking (No)
OnBoardClock (20.000000)
Enables {
Mode (0x2)
HEnable (Ox0)
VEnable (Ox0)
}
Interlaced (Yes)
{
Sequential (Odd)
Mode (Frame)
Current (Yes)
}
SignalLevels {
DataLines (TTL)
EnableA (TTL, High)
EnableB (TTL, High)
EnableC (TTL, High)
EnableD (TTL, High)
MasterClock (RS422)
PGOutput (RS422)
PixelClock (TTL, High)
}
90
Tap (0) {
Start (Left, Top)
Position (QuadVerticall)
}
Tap (1)
Start (Left, Top)
Position (QuadVertical2)
}
Tap (2) {
Start (Left, Top)
Position (QuadVertical3)
}
Tap (3) {
Start (Left, Top)
Position (QuadVertical4)
}
}
Pattern {
Action (PG) {
Line (0) {
Start (High)
}
Line (1) {
Start (High)
}
Line (2) {
Start (High)
}
Line (3) {
Start (High)
}
Line (4) {
Start (High)
}
Line (5) {
Start (High)
91
I
Line (6) {
Start (High)
I
Line (7) {
Start (High)
I
RepeatSource (Fixed, 1000)
Timebase (1)
I
}
}
IMAQ Cable Design
The following information contains specifications for designing an IMAQ cable for the
Custom DPPI camera. The camera is assumed to be connected to a SCSI-100 connector
on the framegrabber.
PIXEL CLOCK
The Pixel Clock (PCLK) is the clock signal sent from the camera to the board. On each
rising edge of PCLK, one bit of data should be sent over each data line. The PCLK line
should be connected to pin 49.
DATA LINES
Each line is listed as D_<char><num><+/->, with <char> representing the tap, <num>
representing the data bit - 0 is the LSB and 8 is the MSB - and <+/-> representing
the positive and negative pins of the differential signal.
LINE
D_AO+
D_Al+
D_A2+
D_A3+
D_A4+
DA5+
DA6+
DA7+
PIN on SCSI 100 connector
1
3
5
7
9
11
13
15
92
LINE
DBO+
DB1+
DB2+
DB3+
DB4+
DB5+
DB6+
DB7+
DCO+
DC1+
DC2+
DC3+
DC4+
DC5+
DC6+
DC7+
DDO+
D_D1+
DD2+
D_D3+
DD4+
DD5+
D_D6+
D_D7+
PIN on SCSI 100 connector
17
19
21
23
25
27
29
31
51
53
55
57
59
61
63
65
67
69
71
73
75
77
79
-81
ENABLES
Line Enable (HENABLE) is a signal that designates that a line is in progress, and should
stay High during the entire time that line data is valid. The HENABLE line should be
connected to pin 43.
Frame Enable (VENABLE) is a signal that designates that a frame is in progress, and
should stay High during the entire time that frame data is valid. The VENABLE line
should be connected to pin 41.
Field (FLD) is a signal that designates which field of the interlaced image is currently
being read, and should stay High during the Odd fields. The FLD line should be
connected to pin 45.
93
Appendix E: Budget
Supplier
ADI Austin
Advanced Circuits
Digikey
Digikey
Digikey
Digikey
Digikey
Digikey
Digikey
Digikey
Digikey
Digikey
Digikey
Digikey
Digikey
Digikey
Digikey
MIT
MIT
MIT
MIT
MIT
MIT
MIT
MIT
MIT
National Instruments
National
Semiconductor
Rotocast Plastics
Xilinx
Xilinx
Order Quantity Unit Price $ Device Cost $
Part Description
462
115.5
4
Wide Angle Lens
360
36
10
DPPI Sensor Board
224.32
28.04
8
ZIF PGA Socket (13 x 13)
17.98
17.98
1
Connector
100 pin SCSI
22.9
4.58
5
Analog to Digital Converter
24.78
4.13
6
32MHz Clock
16.52
4.13
4
80MHz Clock
9.64
2.41
4
Inductor: 1uH
7.28
0.0364
200
Capacitor: 0.1uF
8.26
0.413
20
Pot: 200
8.26
0.413
20
Pot: 2K
0.9
0.09
10
Resistor: 750
1.8
0.09
20
Resistor: 2.5K
1.8
0.09
20
Resistor: 10K
1.6
0.08
20
Resistor: 750
1.8
0.09
20
Resistor: 370
1.8
0.09
20
Resistor: 250
0
0
11
DPPI Sensor
0
0
1
14" x 14" x 0.25" Acrylic Board
0
0
2
6" x 6" x 0.5" Wood Board
0
0
1
10" x 10" x 1.25" Polycarbonate
0
0
1
8" Diameter Delrin Rod
0
0
40
Nuts & Bolts
0
0
26
Spacers
0
0
20ft
Twine
0
0
20
Wood Screws
85
85
1
IMAQ Cable
0
0
20
Current Source
13" Diameter Globe
Field Programmable Gate Array
PROM
94
2
2
14
45
0
0
90
0
0
Total:
Budget:
1346.64
1666.66
Savings
320.02
References
[1]
A Differential Passive Pixel Image Sensor. Masters Thesis.
Fujimori, Iliana L.
Massachusetts Institute of Technology, 1997.
[2]
Passive Pixel Imager: Imager Pinout Specifications.
Fujimori, Iliana L.
Massachusetts Institute of Technology, March 27, 2000
[3]
Horenstein, Henry. Black and White Photography: A Basic Manual. Second
Edition. Little, Brown and Company. Boston, 1983
[4]
Sodini, Charles G., Iliana L. Fujimori, and Ching-Chun Wang. "A 256 x 256
CMOS Differential Passive Pixel Imager with FPN Reduction Techniques."
IEEE Journal of Solid-State Circuits, Vol. 35, No. 12, December 2000.
[5]
P-CAD PCB User's Guide, Accel Technologies, CA 1983
[6]
P-CAD Schematic Entry User's Guide, Accel Technologies, CA 1983
[7]
P-CAD Library Executive User's Guide, Accel Technologies, CA 1983
[8]
National Instruments Labview: Measurement Manual Texas, 1992
[9]
National Instruments Labview: User Manual Texas, 1992
[10]
Photography Dictionary can be found at www.computar.com
[11]
Armstrong, James R., F Gail Gray. VHDL Design Representation and Synthesis.
Second Edition, New York 2000.
95