Image Formation:
Optics and Imagers
Real world
Optics
Sensor
Acknowledgment: some figures by B. Curless, E. Hecht, W.J. Smith, B.K.P. Horn, and A. Theuwisse
Optics
• Pinhole camera
• Lenses
• Focus, aperture, distortion
Pinhole Camera
• “Camera obscura” – known since antiquity
Image plane
Image
Pinhole
Object
Pinhole camera
Pinhole Camera Limitations
• Aperture too big: blurry image
• Aperture too small: requires long exposure or
high intensity
• Aperture much too small: diffraction through
pinhole  blurry image
Lenses
• Focus a bundle of rays from a scene point
onto a single point on the imager
• Result: can make aperture bigger
Ideal Lenses
• Thin-lens approximation
• Gaussian lens law:
1/do + 1/di = 1/f
• Real lenses and systems of lenses may be
approximated by thin lenses if only paraxial
rays (near the optical axis) are considered
Camera Adjustments
• Iris?
– Changes aperture
• Focus?
– Changes di
• Zoom?
– Changes f and sometimes di
Zoom Lenses – Varifocal
Zoom Lenses – Parfocal
Focus and Depth of Field
• For a given di, “perfect” focus at only one do
• In practice, OK for some range of depths
– Circle of confusion smaller than a pixel
• Better depth of field with smaller apertures
– Better approximation to pinhole camera
Monochromatic Aberrations
• Real lenses do not follow thin lens
approximation because surfaces are
spherical (manufacturing constraints)
• Result: thin-lens approximation only valid iff
sin   
Monochromatic Aberrations
• Consider the next term in the Taylor series,
i.e. sin    - 3/3!
• “Third-order” theory – deviations from the
ideal thin-lens approximations
• Called primary or Seidel aberrations
Spherical Aberration
• Results in blurring of image, focus shifts when
aperture is stopped down
• Can vary with the way lenses are oriented
Coma
• Results in changes in magnification with
aperture
Coma
Distortion
• Pincushion or barrel radial distortion
• Varies with placement of aperture
Distortion
• Varies with placement of aperture
Distortion
• Varies with placement of aperture
Distortion
• Varies with placement of aperture
First-Order Radial Distortion
• Goal: mathematical formula for distortion
• If distortion is small, can be approximated by
“first-order” formula:
r’ = r (1 +  r2)
r = ideal distance to center of image
r’ = distorted distance to center of image
• Higher-order models possible
Correcting for Aberrations
• Compound lenses
use multiple
lens elements to
“cancel out”
aberrations
• Lenses of different
materials
• 5-15 elements, more for extreme wide angle
Other Limitations of Lenses
• Flare: light reflecting
(often multiple times)
from glass-air interface
– Results in ghost images or haziness
– Worse in multi-lens systems
– Ameliorated by optical coatings (thin-film
interference)
Other Limitations of Lenses
• Optical vignetting: less power per unit area
transferred for light at an oblique angle
– Transferred power falls off as cos4 
– Result: darkening of edges of image
• Mechanical vignetting: due to apertures
Sensors
• Film
• Vidicon
• CCD
• CMOS
Vidicon
• Best-known in family of “photoconductive
video cameras”
• Basically television in reverse
----
++++
Scanning Electron Beam
Electron Gun
Lens System
Photoconductive Plate
Digression: Gamma
• Vidicon tube naturally has signal that varies
with light intensity according to a power law:
Signal = Eg, g  1/2.5
• CRT (televisions) naturally obey a power law
with gamma  2.5
• Result: standard for video signals has
a gamma of 1/2.5
MOS Capacitors
• MOS = Metal Oxide Semiconductor
Gate (wire)
SiO2 (insulator)
p-type silicon
MOS Capacitors
• Voltage applied to gate repels positive “holes”
in the semiconductor
+10V
++++++
Depletion region
(electron “bucket”)
MOS Capacitors
• Photon striking the material creates
electron-hole pair
+10V
Photon
++++++
-
-
+
-
-
-
-
-
Charge Transfer
• Can move charge from one bucket to another
by manipulating voltages
Charge Transfer
• Various schemes (e.g. three-phase-clocking)
for transferring a series of charges along a
row of buckets
CCD Architectures
• Linear arrays
• 2D arrays
– Full frame
– Frame transfer (FT)
– Interline transfer (IT)
– Frame interline transfer (FIT)
Linear CCD
• Accumulate photons, then clock them out
• To prevent smear: first move charge to
opaque region, then clock it out
Full-Frame CCD
• Other arrangements to minimize smear
Frame Transfer CCD
Interline Transfer CCD
Frame Interline Transfer CCD
CMOS Imagers
• Recently, can manufacture chips that
combine photosensitive elements and
processing elements
• Benefits:
– Partial readout
– Signal processing
– Eliminate some supporting chips  low cost
Color
• 3-chip vs. 1-chip: quality vs. cost
Chromatic Aberration
• Due to dispersion in glass (focal length varies
with the wavelength of light)
• Result: color fringes near edges of image
• Correct by building lens systems with multiple
kinds of glass
Correcting Chromatic Aberration
• Simple way of partially correcting for residual
chromatic aberration after the fact: scale
R,G,B channels independently
Video
• Depending on the scene, pictures updated
at 15–70 Hz. perceived as “continuous”
• Most video cameras use a shutter, so they
are capturing for only part of a frame
– Short shutter: less light, have to open aperture
– Long shutter: more light, but motion blur
• Television uses interlaced video
Interlacing
These rows
transmitted
first
These rows
transmitted
1/60 sec later
Television
• US: NTSC standard
– Fields are 1/60 sec.
– 2 fields = 1 frame  frames are 1/30 sec.
– Each frame has 525 scanlines, of which
approximately 480 are visible
– No discrete pixels along scanlines, but if pixels
were square, there would be about 640 visible
Television
• NTSC standard
– Thus, an NTSC frame is about 640480
– Color at lower resolution than intensity
• PAL standard
– Lower rate: fields at 50 Hz. (frames at 25 Hz.)
– Higher resolution: about 768576