Laser range finder with CMOS image sensor in automotive field
Jih-Huah Wu*, Rong-Sen Chang
Institute of Optical Sciences, National Central University,
Chung-Li City, Tao-Yuan, Taiwan, R.O.C.
* E-mail address: jhwu@ios.ncu.edu.tw
ABSTRACT
A low cost prototype of a laser range finder using a CMOS image sensor is developed for the automotive field. The
system presented here is based on triangulation. The gravity of the infrared laser spot on CMOS image sensor is
converted into pixel coordinates proportional to the distance to be measured. Based on the experimental tests of the
system, it was found that the distance could be estimated with accuracy better than 5% within the range of 5 to 45 meters
at twilight and night. Experimental results are provided and possible improvements are discussed.
Keywords: laser range finder, CMOS image sensor, triangulation.
1. INTRODUCTION
It is important to maintain a suitable distance from the preceding vehicle on the freeway. However, it is very difficult
for a driver to judge the real distance by eye alone. In addition, adaptive cruise control (ACC)
1-2
systems for controlling
or measuring the headway distance from the host vehicle to the preceding vehicle are too expensive for most drivers.
Thus the finding of a feasible low cost system for measuring the distance between vehicles on the freeway is the main
goal of this paper.
Laser range finders can be divided technically into three categories or methods 3: time-of-flight, triangulation, and
interferometer. A combination of the vision system and the triangulation method is widely used for different kinds of
industrial applications
4-6
. Many algorithms have been developed for different applications, and there exist various
processors to speed up their execution. The range and resolution of these measurement systems are all dependent on the
system layout. The range covered can be from some centimeters to some meters, and the resolution achieved can be less
than 1μm. As a general rule, time-of-flight based systems are more appropriate for measuring long distances, while
triangulation based systems are more suitable for measuring short distances and small ranges with high accuracy 7.
Most automotive laser radar systems are based on the time of flight method
1,2,8,9
. A pulsed laser radar detection
9
system (LIDAR) for obstacle detection in the automotive field is proposed . This consists of an infrared (IR) scanning
system of the LIDAR type set up in a car. In addition, obstacle distance measurement can also performed by triangulation
for up to 30 meters. A laser range finder with a measurement range from 5 to 40 meters was proposed by Bernard and
Gaëlle10. It is based on optical power modulation, thus the name frequency modulation continue wave like (FMCW).
1
Laser triangulation is normally used in conjunction with position sensing detectors (PSDs) or with charge coupled
devices (CCDs), where the centroid position is found through intensity profile fitting to a similar Gaussian shaped laser
spot. In this paper, we adopt a CMOS image sensor to function as a one-dimension array position sensing detector, but
with an integrated pre-amplifier, analog to digital converter, and signal-processing system on one chip. Their
programmable features include electronic exposure control, continuous or single frame capturing, and progressive or
interlace scanning modes. The first one is very important for our experiments in automobile applications. This paper
describes the design of the distance measurement system and presents the results of tests conducted to verify the level of
performance that can be obtained.
2. PRINCIPLE OF MEASUREMENT
The basic principle of optical non-contact range-finding devices is to project a light signal onto an object and to
process the reflected or scattered light to determine the distance. A laser beam that provides a high spatial resolution is a
good choice. Our system adopts a simple structure based on triangulation; the geometrical layout is shown in Fig. 1. The
center of the sensor is approximately aligned to an object located about 15 meters away. The distance between the sensor
and the measured vehicle is evaluated by a simple triangulation principle. Using trigonometry, the target coordinates X
are mapped to the detection position d on the sensor, as shown in Fig. 1; the distance X is given by
X 
L
tan( o  tan 1 (d / f ))
(1)
,
where X > Q.
The angle in the second term, tan 1 (d / f ) , is positive when X < Q; θo is approximately equal to 3.05° in these
experiments.
Due to the small θo, X is almost equal to Z, and L is almost equal to L’, therefore, Eq. (1) can be simplified to
X Z 
fL
,
f tan  o  d
(2)
and we differentiate Eq. (2) to get
X  Z 
Z 2 d
fL
.
(3)
In a triangulation-based range finder, the farther distance the lower the measurement precision. In other words there is
a trade-off between the accuracy and the measurement range. In these experiments, the smallest resolvable distance
change that can be calculated by Eq. (3), with a pixel size of 8μm, L=800mm and f=16mm, thus the system has a
resolution of 1.3 meters at 45 meters, 1 meter at 40 meters, and so on. The centroid method has been widely used to
locate a light spot on image features of various types, to sub-pixel accuracy 6,11,12,13. By introducing sub-pixel processing,
we can increase the resolution. The intensity of the i’th pixel in a row of pixels on sensor is gi(x). By setting the threshold
level, a pixel with intensity below the threshold level would be neglected. The spot’s center X c is calculated as the
2
centroid of gi between pixels m and n. The centroid X c of the laser spot, on one dimension, can be derived by
n
Xc 
 xi g i
m
n
 gi
.
(4)
m
At twilight and at night, infrared radiation from the sun decreases tremendously. Thus a narrow band pass filter (BPF)
was mounted before the sensor to cut the visible light in order to increase the signal to noise (S/N) ratio. A spectral filter
is used in this experiment.
On the other hand, the automatic gain control built into the sensor is similar to a variable neutral density filter, and
can either be adjusted by the user or by programming it. The exposure time (ET) of the sensor can be adjusted to
different levels for different situations; this means that the S/N ratio can be improved by setting the gain level. By
decreasing the gain level, i.e., decreasing the ET, the gray level of a spot that exists in one frame will be unsaturated
except for the IR laser spot, taking a suitable threshold value above the lowest gray level. Thus the centroid position of
the spot can be found without using spatial zone selection.
Finally, in order to decrease the quantity of the data that the computer has to process, and to reduce the backlight
noise, the data is only picked out by zone selection. The data in a zone can be selected by a computer program thus
decreasing the amount of data to be calculated tremendously. It acts like a spatial filter, which is similar to adding a slit
before the sensor. These three processes in combination can solve the main problems involved in measuring the distance
to a preceding vehicle at twilight or night, including effects from the background light of street lamps or other vehicles,
the speed of the personal computer, and the overall cost of the system.
3. EXPERIMENTAL SETUP
The configuration of the prototype laser range finder, which consists of an IR laser diode (LD), a collimated lens, a
filter, a collective lens with focal length of 16 mm, and a sensor, is shown in Fig. 1. The IR LD has an operational
wavelength of 811nm, is set up on the bumper of a car. And the sensor was mounted on the rear-view mirror brace, as
shown in Fig.2. The VC++ language software program could quickly calculate the centroid position. The frame data was
captured by VFW (Video for Window), supported by Windows 98. If Windows 2000 is installed, the program can capture
the frame data via DIREXT.
The spectra of the IR laser and the taillights of the preceding vehicle were measured by spectrometer EPP2000
(manufactured by StellarNet Inc.) with a responsibility limitation of 1200nm. The spectrum of IR laser (typical
wavelength at 808nm), and the transmission of BPF are shown as curve 1 and curve 2 in Fig. 3, respectively. Curves 3
and 4 in Fig. 3 indicate the spectra of the taillights of a braking and an un-braking vehicle, respectively. At night, the
biggest noise source will be the taillights of the other vehicles; which are mostly made of tungsten. This means a broad
spectrum, from visible light to infrared 14. The total optical power of a vehicle taillight on the sensor is larger than that of
3
the IR laser when the vehicle brakes, however in most situations, the optical power of the taillight of a non-braking
vehicle is smaller than that of the IR laser, especially if the back of the preceding vehicle is not very dirty or decorated
with very low reflectance materials. See Fig. 4(a) and (b). The brightest point is the laser spot been rectified by an
aperture; it can be discriminated easily.
A CMOS image sensor HV7131D manufactured by Hynix Semiconductor Inc. with a 5.3*3.8 mm active area is used
to detect, with high accuracy and stability, the laser light spot along one axis. This sensor has continuous sensitivity
covering a wide dynamic range. The resolution of this sensor is 8 μm. The IC 2001B Driver is a low cost, single chip
driver for image processing, JPEG compression and with a USB Interface, Manufactured by the Jeilin Technology Corp.,
Ltd of Taiwan. The ET range can be adjusted from 0 to 99. An IR Laser diode model, LD-808-200G, manufactured by
AD technology, USA is also used.
4. EXPERIMENTAL RESULTS
At twilight, the signal of an IR laser is stronger than the background noise (the ET of the sensor was set at 20), but
the taillights of other vehicles may disturb the frame processing. This situation can be remedied by limiting the selection
zone, as shown in Fig. 4(a), or by choosing one linear array of pixels (as in this experiment).
At night, all vehicles turn on their taillights (without braking), consequently the ET of the sensor was set to 20. The
signal (IR laser spot) in the middle of the frame as shown in Fig. 4(b), is brighter. Due to the separation of the laser spot
and the taillight, choosing pixels for one linear dimension also allows us to pick out the laser spot. However, when the
laser spot overlaps with the taillight, measurement error will be produced. This condition will be discussed later.
The standard deviation (in pixels) of the system range from 5- to 45- meters is shown in Fig. 5. Due to the large
amount of engine vibration, the standard deviations vary greatly, as shown in Fig. 5, curves (2) and (3). The large curve
(3) variation was caused by variations in the intensity of the laser spot. Because the intensity of the laser spot on sensor
was not saturated and not uniform, when the laser spot is reflected or scattered from a distant preceding vehicle, the
shape can be easily separated by the different reflectance thus causing shifts in the calculated spot center to occur. This
situation can also be improved by increasing the ET value. And to avoid non-uniform reflectance from the preceding
vehicle, the laser spot should be focused and made as small as possible. It is worth pointing out that the laser beam
should be symmetrical along the optical axis; otherwise, a new error source may be introduced by changing the gain (ET)
of the sensor.
The maximum standard deviation for a static experiment is approximately 0.13 meters at a distance of 45 meters, and
0.1 meters at a distance of 40 meters, respectively, as shown in Fig. 6, curve (1). In the dynamic and normal case, the
maximum standard deviation is 1.3 meters at a distance of 45 meters, and 1 meter at the distance of 40 meters. In other
words, the distance could be estimated with an accuracy of better than 5% within the 5- to 45- meters range, as shown in
Fig. 6 curve (2). However the dynamic and worse case (our test vehicle has been used for more than 12 years), the
4
maximum standard deviation is 2.5 meters at a distance of 45 meters, and 1.4 meters at a distance of 40 meters, as shown
in Fig. 6 curve (3).
The system was tested during freeway driving to determine the detection range for vehicles traveling in the same lane
on straight stretches of the road. According to the ISO/DIS 15623.4.2, this prototype could match the Class I System
classification. The detection performance of Class I systems is specified as being effective for detecting obstacle vehicles
within the detection area along straight road segments. This system is not suitable for measuring distance on curve roads
as the laser beam may miss the preceding vehicle on the curves in some cases, making detection impossible. Subtracting
the distance, h, between the LD and the sensor from X, we can get the working distance. In this work, h is equal to 1.5
meters, and the working distance is in the range at 3.5- to 43.5- meters.
There are several sources that introduce errors into distance measurements. Some errors are due to the characteristics
of the laser, such as the speckle effect. It is shown that speckle noise 12 is a fundamental limit for the position sensing of a
laser spot centroid. However, the main limitation to distance resolution by the present prototype is the fact that the laser
beam does not come from an ideal spot, but rather a cylindrical or a circular shape (rectified by an aperture). This was
caused by the unsymmetrical beam divergence of the IR LD: the horizontal divergence angle of the LD is 9 degrees; the
vertical divergence angle of the LD is 30 degrees. The other main source error was the vibration of the engine of the host
vehicle. From the experimental results, we see that the less relative vibration between the sensor and the LD, the more
precise the system will be. Finally, the roughness of the road and the non-uniformity of the rear of the preceding vehicle
(this factor can be neglected for most automobiles, except trucks.) are other parameters that affect the measurement
accuracy, but these factors have not been considered here.
5. DISCUSSION
It is important to recognize that a tremendous eye damage hazard exists for lasers with wavelengths from 400nm to
1.4 microns 15. In this range finder prototype, the dominant wavelength of the laser diode is 808nm (typical). For safety,
the IR laser diode could radiate a pulsed laser beam having a pulse width of slightly more than 1/20 (for the VGA mode)
or 1/30 (for the CIF mode) second, with a peak power at around 145mW controlled by a pulse control circuit. This pulse
control circuit would be linked to a personal computer via a RS-232 or other interface card. The computer can
simultaneously catch the frame when the laser is turned on. The total energy would be approximately 7.25 or 5mJ, which
would be much safer. This specification falls within the laser safety Class 3a as defined in the ANSI Z136.1 standard. If
not pulsed, it matches the laser safety Class 3b. On the other hand, the extension of the laser beam is large, shaped like a
stripe or a circle with areas larger than 200cm2, meaning that the laser light that would penetrate to the eye of the
preceding driver or passenger would be far less than 5mJ, whether viewed directly or through intrabeam specular
reflection. Due to the inherent defects of the laser diodes, the laser spot shape varies and diverges with the distance. In
order to reduce the effect of a rough surface on the preceding vehicle, and to increase the S/N ratio, the laser beam has to
be made or rectified to be as small as possible. Using a laser diode with a small divergence angle or good optical system
5
to rectify the laser spot shape is a good choice.
Sometimes braking taillights from the preceding vehicle will cause incorrect operation of the range finder. To solve
this problem, one can employ a much narrower BPF with a FWHM (full width at half maximum) at 1-2 nm and a center
wavelength at 808nm, and control the laser power and temperature with a thermal electrical cooler, to achieve the major
wavelength (808nm). Another option is to discriminate the laser spot from the overlapping taillights by cross-checking
two frames to see whether the laser spot exists or not.
CONCLUSIONS
A prototype automotive laser range finder using CMOS image sensor has been designed specifically for 5- to
45-meters distance measurements at twilight or night. It presents the advantages of low cost and compactness, and it can
be integrated with other image processing system into single sensor. This type of range finder could assist the driver to
realize the real distance from the preceding vehicle, in order to reduce the chance of automobile collisions.
REFERENCES
[1]
K. Osugi, K. Miyauchi, N. Furui, and H. Miyakoshi, Development of the scanning laser radar for ACC system,
JSAE Review, 20, p. 549-554, 1999.
[2]
A. Higashimata, K. Adachi, T. Hashzume, and S. Tange, Design of a headway distance control system for ACC,
JSAE Review, 22, p.15-22, 2001.
[3]
M.-C. Amann, T. Bosch, R. Myllylä, and M. Rioux, Laser ranging: a critical review of usual techniques for
distance measurement, Optical Engineering 40(1), p.10-19, 2001.
[4]
I. Moring, T. Heikkinen, R. Myllylä, and A. Kilpelä, Acquisition of three-dimensional image data by a scanning
laser range finder, Optical engineering, Vol. 28 (8), p.897-902, 1989.
[5]
V. Sequeira, J.G.H. Goncalves, and M. Isabel Ribeiro, 3D environment modeling using laser range sensing,
Robotics and Autonomous Systems, Vol. 16, p.81-91, 1995.
[6]
M.A.G. Izquierdo, M.T. Sanchez, A. Ibañez, and L.G. Ullate, Sub-pixel measurement of 3D surfaces by laser
scanning, Sensors and Actuators, Vol. 76, p.1-8, 1999.
[7]
M.A. Kujoory, Real-time range and elevation finder, Proceedings of the IEEE, Vol. 72(12), p.1821-1822, 1984.
[8]
A. Najmi, A. Mahrane, D. Estève, G. Vialaret, and J.J. Simonne, Pulsed LIDAR for obstacle detection in the
automotive field: the measurement of reflectance range data in scene analysis, Sensors and Actuators A: physical
46-47, p.497-500, 1995.
[9]
M. Sekine, T. Senoo, I. Morita and H. Endo, Design method for an automotive laser radar system and future
prospects for laser radar, IEEE Int. Symposium on the Intelligent Vehicle, p.120-125, 1992.
[10] B. Journet and G. Bazin, A low-cost laser range finder based on an FMCW-like method, IEEE Trans. on
6
Instruments And Measurement, Vol. 49 (4), p.840-843, 2000.
[11] Y. Bar-Shalom, H.M. Shertukde, K.R. Pattipati, Use of measurements from an imaging sensor for precision target
tracking, IEEE Transactions on Aerospace and Electronic Systems, Vol. 25 (6) p.863-872, 1989.
[12] R.G. Dorsch, G. Häusler, and J.M. Herrmann, Laser triangulation: fundamental uncertainty in distance
measurement, Applied Optics, Vol. 33 (7) p.1306-1314, 1994.
[13] R.H. Stanton, J.W. Alexander, E.W. Dennison, T.A. Glavich, and L.F. Hovland, Optical tracking using
charge-coupled devices, Optical Engineering, Vol. 26 (9) p.930-938, 1987.
[14] L. Levi, Applied optics-A guide to optical system design, Vol. 1, p.194-196, Wiley, 1983.
[15] K.J. Kuhn, Laser engineering, p.444, Prentice-Hall1, New York, 1998.
Fig. 1.Configuration of a prototype laser range finder using a CMOS image sensor.
7
Fig. 2. Photography of CMOS image sensor and mounted on the rear-view mirror brace.
Fig. 3. Spectra of the IR laser (curve1), the transmission of a narrow band pass filter (curve2),
and the spectrum of the taillight of the vehicle (curve3 and 4).
8
Fig. 4. (a) Real test image on a freeway at twilight.
Fig. 4. (b) Real test image on a freeway at night.
4
standard deviation(pixels)
3.5
1: static, ET=20
3
2 : dynamic, ET=20
2.5
3 : dynamic, ET=02
2
1.5
1
0.5
0
5
Fig. 5.
10
15
20
25
30
object distance (M)
35
40
Experimental results, system standard deviation values (pixels)
are shown for three different measurements conditions.
9
45
3
1: Static
2.5
Standard Deviation(M)
2: Dynamic (normal car)
3: Dynamic (old car)
2
1.5
1
0.5
0
5
10
15
20
25
30
35
40
45
Object Distance (M)
Fig. 6. Experimental results, system standard deviation values (M)
are shown for three different vibration conditions.
10