Image acquisition techniques, cameras
Václav Hlaváč
Czech Technical University in Prague
Center for Machine Perception (bridging groups of the)
Czech Institute of Informatics, Robotics and Cybernetics and
Faculty of Electrical Engineering, Department of Cybernetics
http://people.ciirc.cvut.cz/hlavac, hlavac@ciirc.cvut.cz
Courtesy: Vladimír Smutný.
Lenses and their parameters.
Illuminants.
Photoconversion, CCD, CMOS.
Cameras, user’s view.
Imaging systems.
Outline of the talk:
Miscellaneous.
Imaging systems
2/39
A look at an entire chain: from the observed property of interest through
radiance L and irradiance E to an electrical signal and finally to a digital image.
Direct observation – there is one-to-one correspondence between a point in
a 3D scene and its 2D image (e.g., a ray in projective transformation).
Two options of image acquisition:
Indirect observation – provides also a spatially dependent radiance L but
there is no one-to-one correspondence between 3D and 2D information (e.g.,
radar, tomography, spectral imaging techniques, magnetic resonance).
Light polarization (1)
Radiance is expressed as oscillating electrical and magnetic field in the
theory of electromagnetic field.
Vector fields describing the intensity of the electric field E and the intensity
of magnetic field B are solution to the system of Maxwell’s linear differential
equations.
The direction of vector E in 3D varies in general. E.g.
Sun due to short emission phenomena or incandescent
lamps provide mainly a random mixture of waves containing all E orientations, so called unpolarized light.
3/39
A polarization filter (polarizer) selects waves lying in a
single plane, so called linearly polarized light.
Light polarization (2)
A harmonic planar wave is a solution to Maxwell’s differential equations in a
free space (not taking into account electric potentials and currents).
The unpolarized light, coming e.g. from Sun, is polarized after passing
through a polarizing filter.
Nature provides a polarization filter in, e.g. Iceland spar (in Czech:
dvojlomný vápenec). Artificial polarization filters consists of parallel fibres of
elongated molecules oriented in one direction.
4/39
Examples: Polarized spectacles for fishermen. Polarized filter for a camera
lens.
Illuminants according to the emission (1)
5/39
Day light – no flickering, unstable in time and color, very good viewing colors.
Incandescent lamp – does not flicker, warms the device, big energy input, big
starting current, should be changed often, good for viewing colors.
Halide lamp – no flickering, should be changed often, good for viewing colors
(better than incadescent lamp), smaller than incadescent lamp.
Fluorescent lamp – flickering (it is possible to power it with high frequency
current or synchronize it), needs special power source, long life, bad for
viewing colors, close to surface source (in Czech plošný zdroj).
Illuminants according to the emission (2)
6/39
LED – Light Emitting Diode, modulated light, no warming, small size, low
power consumption, monochromatic (also infrared, white color), long life.
Laser – Light Amplification by Stimulated Emission of Radiation). Device
producing light of a single (pure) color = monochromatic. Can be
modulated, coherent (=same phase) ⇒ problems with interferences, low
power consumption, long life for semiconductor lasers.
Flash tubes – e.g. xenon lamps, used in applications in which big power is
needed, very expensive.
Illuminants according to the size (1)
7/39
Point sources – e.g. halid lamp, LED, laser. Emphasize the roughness of the
surface. Strong highlights.
Surface sources, diffuse – e.g. reflection from a white opaque wall, paper,
fluorescent lamp, illuminants with large focusing screen (in Czech matnice).
Suppress the roughness of the surface..
Back light diffuse – of advantage in the cases in which only the silhouette of
the object is of interest and the object is thin (metal sheet, animal skin, . . . ).
Very often used in applications as it simplifies segmentation to objects and
background significantly. Suitable for gauging (in Czech měření rozměrů).
Illuminants according to the size (2)
8/39
Blacklight, telecentric – illuminants with collimators. Can be used only for
small objects (up to diameter of the lens aperture), to be combined with
telecentric lenses. Suitable in cases in which silhouette of thin objects is of
interest.
Dark field – oblique illumination when rays are not directed to the lens, there is
a reflection from object to the lens.
Optical tricks
9/39
Monochromatic filter can suppress ambient light and decrease influence of
color abberations.
Polarized filter removes or enhances polarized image, e.g. reflection from glass
cover of the device).
Influence of the polarized filter
10/39
There is a clear glass positioned vertically in front of a camera and it is tilted
with respect to optical axis by about 45◦. A double refraction on the glass is
visible in both images.
Vertical polarization.
Window reflected in a glass.
Horizontal polarization.
Reflection suppressed. Visibility through.
Directional illumination
Irradiance of the opaque surface (ideal case: Lambertian) depends on the
surface direction. That is the reason why the tilt of the surface can be
measured (shape from shading). One of the first applications was in
measuring shape of planetary surfaces, e.g. Venus craters.
Shadows can generate edges in images which can be confused with object
boundaries..
11/39
Mirror component of reflectance causes highlights. If this the problem then
directional illumination is not suitable.
diffused ILLUMINATION
Natural day light with overcast sky, fog.
Solution in devices: circle from LEDs, semi-sphere from LEDs.
12/39
Useful for surfaces with significant mirror component of reflectivity.
Backlight illumination
Useful when the silhouette of the non-transparent object is sought.
Simplifies segmentation.
Useful also for semi-transparent objects where a range of interactions
between light and matter can be observed (refraction, absoption, diffusion of
light). Local inhomogeneities in the matter can be detected.
13/39
Examples: X-ray. Spectral analysis when absorption depends on frequency.
Light field illumination
Rays from the illuminant are directed to the camera.
Objects between illuminant and camera look darker due to refraction,
absorption or diffusion. Objects are dark on the light background.
14/39
Used for viewing small particles.
Dark field illumination
Rays from the illuminant are not directed to the camera.
Refraction, reflection, diffusion of light which falls to the camera is visible.
Objects are light on the dark background..
15/39
Used to visualize small particles, metallic surfaces in microscopy (e.g.,
aluminium conductors in microelectronics).
Telecentric illumination
A collimator secures parallel rays.
Lenses of big parameters have to be used if objects are large (often Fresnel
lenses = steps-like lens from concentric elements).
16/39
The measured gauge is invariant to the distance of the object from the lens.
Common lens
17/39
Distance to the object focal length.
Normal, wide-angle, telephoto lens.
Chip
Object
F
Lens
Sharpness of the image
18/39
open diaphragm
closed diaphragm
lens
chip
F
Microscopic lens
19/39
Image is increased, short working distance (approx. 1 mm), however can be also
bigger (approx. 100 mm).
Wide observation angle and small depth of focus.
Object
F
Lens
Chip
Telecentric lens
Only principal rays used, i.e. those parallel with optical axis.
The input lens has to have bigger diameter than measured object.
20/39
Useful when measured object changes its position or the object is ‘thick’.
Collimator
Chip
F
Light source
Object
Lens
Diaphragm
Parameters of lenses (1)
21/39
Focal length – fixed, adjustable (zoom) manually or motorized.
Working aperture, diaphragm (also speed of the lens) – the smallest
and the greatest aperture.
Diaphragm – fixed, adjustable manually or motorized..
C – the distance between the back of the lens and the chip is approx.
17 mm.
CS – approx 12 mm, the other parameters are the same.
Lens connecting
Lens for C mount can be adjusted to CS mount by an extension ring
5 mm thick, not possible in the other direction CS to C.
Parameters of lenses (2)
22/39
Focusing – Fix focus (e.g., web cameras, mobile phones), manual or motorized
focusing.
Distances in which object is in focus – can be changed by extension rings
in the expense of deteriorated optical properties.
Format – which is the biggest chip usable; 1”, 2/3”, 1/2”, 1/3”, 1/4”.
Thread for a filter – clear filter is used to protect the lens.
Radial distortion – is not given in technical sheets but it is important for
measurement applications. Lenses with short focal length have typically
bigger radial distortions (several pixels).
Principle of photoconversion in semiconductors
Incoming radiation (photons) in converted in the semiconductor mass into
charge couples, electron-hole.
The semiconductor is in a static electric field. The Electron-hole couples are
converted into a short current impulse.
23/39
The current impulse must be amplified and processed. E.g., in a CCD
element the impulse is used to charge a capacitor.
Photodiode and MOS structure
24/39
Cross cut of two main principles for current generation and storing the charge.
CCD architectures
25/39
CCD chip, properties of the technology
26/39
+ Linearity: CCD sensors explore conversion of a photon to the couple
electron-hole. The obtained charge is integrated in a capacitor.
+ Low noise: is given by the integral character of the measurement.
Uncooled chip with TV read-out has SNR approx. 60 dB.
+ Efficiency: Current sensors have hight energetic efficiency approx. 40%, i.e.
every third photon generates one couple electron-hole.
– Read-out: only from the whole chip at once.
– Limited range of intensities: is given by the maximal capacity of
individual capacitors..
CMOS chip, properties of the technology
27/39
http://www.ims-chips.de/products/vision/hdrc alt/hdrc ima.html
http://www.imec.be/bo
http://www.vector-international.be/C-Cam/cmosccd.html
+ Logarithmic sensitivity: CMOS sensors are based on the photo diode
principle. They measure a current in a read-out instance.
+ Read-out: possible in arbitrary order, e.g. only the region of interest can be
read-out.
+ Camera and processor on the same chip: CMOS technology is well
mastered (processors, memory). Smart cameras.
– Higher noise:
Cameras, user’s view (1)
Spatial resolution: number of pixels in a row and in a column. TV
CCIR/PAL 768×576. TV RS170/NTSC 640×484. Non-television cameras
also 2000×2000, keep increasing.
Resolution in intensity: given in bits for digital cameras, output typically
8 bits also 12 bits. For analog cameras – SNR, usually >50 dB.
Sensitivity: v lux. Should be recalculated according to used diaphragm and
AGC.
AGC: Automatic Gain Control; yes/no, can be switched off?, manual
control of gain.
28/39
Shutter: commonly from 1/50 s to 1/10000 s.
Cameras, user’s view (2)
Format: size of the photosensitive chip. Given either in inches of the
equivalent vidicon tube diameter or in mm. 1/2” corresponds to
4.8×6.5 mm.
Shape of a pixel: square pixel vs. non-square pixel.
Output for automatic diaphragm:
AWB: Automatic White Balance. Changes ratio of R and B with respect to
G.
Gama correction: fixed/adjustable. Direct signal γ = 1. Typicky
γ = 0, 45 (enhances black). Compensates intensity conversion function of
the CRT (Cathode Ray Tube) and adjusts it to the sensitivity of a human
eye.
29/39
Lens thread: C mount / CS mount.
Interlaced/non-interlaced scanning
30/39
1
1
768
1
1
2
2
3
3
4
4
574
574
575
575
576
576
Interlaced.
768
Non-interlaced.
Signal, interlaced/non-interlaced scanning
31/39
field
frame
frame
odd even odd even odd even odd even
1 3 5
…
575
~
2 4 6
Interlaced.
…
576
1 2 3
…
576
Non-interlaced.
Electronic shutter
32/39
Shortened exposition is used either if there is too much light or if fast events
have to be captured.
I
exposition
frame (noninterlaced)
field (interlaced)
t
Flash light and suppression of ambient light
ambient light intensity
The shutter time is shortened.
The instant of the flash is set when the
shutter is open.
I
The ration between the integral of ambient intensity during the shutter opening
and integral of the flash intensity gives
the influence of ambient light.
33/39
LED are often used as cheap ‘flash light’.
t
I
flash intensity
t
I
exposition
t
I
flash contribution
ambient light contribution
t
Types of CCD cameras
34/39
Line cameras: Used both in B/W and color modifications. Used often in
industrial applications, scanners, faxes and copying machines.
TDI A variant of a line camera used for synchronous capturing of moving scene
using more lines. Increased sensitivity.
Television cameras, black and white
CCIR – 50 Hz, 625 lines, 768x576;
RS-170 (EIA) – 60 Hz, 525 lines, 648x484.
Television cameras, color one PAL, SECAM – 50 Hz; NTSC – 60 Hz.
Progressive scan – non-interlaced.
Digital cameras contain A/D converter, there are high quality ones, prices
drop down both for industry or for multimedia.
Color cameras setups
Manual change of color filters in front of the monochromatic camera lens.
Three chip cameras – an incoming light is divided to a appropriate chip
using color filters and semitransparent mirrors.
35/39
One chip camera has filters directly on a chip. Spatial resolution in color
resolution is smaller than coresponds to the number of pixels.
Arrangement of color filters
in single chip cameras
36/39
R
G
R
G
C
Y
C
Y
G
B
G
B
M
G
M
G
R
G
R
G
C
Y
C
Y
G
B
G
B
M
G
M
G
Additive color model.
Subtractive color model.
Color scanner
37/39
scanned document
Illuminant
mirror
lens
chip
glass
movement direction
c
FIRE WIRE (i.Link by Sony)
A fast serial link. IEEE 1394.
38/39
2 types of transmission:
1. isochronous, e.g. images;
Used also for disks, cameras, interconnection between domestic electronics
pieces (e.g., audio system Sony).
Non-television camera. Example: color camera 1200× 1024, 15 snímků/s, 40
thousands Kč.
Two types of connectors. 4 pin and 6 pin one including power sourse.
Kabel max. 4.5 m. Repeaters
2. asynchronous, e.g. sending parameters to a device.
The younger competitor to fire wire (IEEE 1394) is USB 2.
A newer bus 1394b
Substantial innovation. Speeds up to 3.2 Gb/s.
Up to hundred meters 100 meters if transmitted on the optical cable.
39/39
Full backward compatibility with currently used specifications 1394-1995 and
1394a.