Copyright © 2009 KSAE
1229−9138/2009/045−11
International Journal of Automotive Technology, Vol. 10, No. 2, pp. 219−228 (2009)
DOI 10.1007/s12239−009−0026−0
SENSOR FUSION-BASED LANE DETECTION FOR LKS+ACC SYSTEM
H. G. JUNG , Y. H. LEE , H. J. KANG and J. KIM
1,2)*
1)
1)
1)
2)
MANDO Global R&D H.Q., 413-5 Gomae-dong, Giheung-gu, Yongin-si, Gyeonggi 446-901, Korea
School of Electrical and Electronic Engineering, Yonsei University, Seoul 120-749, Korea
2)
(Received 20 May 2008; Revised 23 September 2008)
ABSTRACT−This paper discusses the market trends and advantages of a safety system integrating LKS (Lane Keeping
System) and ACC (Adaptive Cruise Control), referred to as the LKS+ACC system, and proposes a method utilizing the range
data from ACC for the sake of lane detection. The overall structure of lane detection is the same as the conventional method
using monocular vision: EDF (Edge Distribution Function)-based initialization, sub-ROI (Region Of Interest) for left/right and
distance-based layers, steerable filter-based feature extraction, and model fitting in each sub-ROI. The proposed method adds
only the system for confining lane detection ROI to free space that is established by range data. Experimental results indicate
that such a simple adaptive ROI can overcome occlusion of lane markings and disturbance of neighboring vehicles.
KEY WORDS : Lane detection, Sensor fusion, Lane keeping system, Adaptive cruise system
1. INTRODUCTION
CHAUFFEUR II is a European project that was completed in 2003 and aimed at developing truck platoon and
integration of LKS and ACC. In particular, the project
proposed a system integrating LKS and Smart Distance
Keeping (SDK), corresponding to ACC, and named it
CHAUFFEUR Assistance (Fritz et al., 2004).
1.1. Background: Popularization of the LKS+ACC System
Adaptive cruise control (ACC) is a driver convenience
system adding headway time control, which maintains
distance to the preceding vehicle within a preset headway
time, to conventional cruise control that maintains preset
speed if there is no preceding vehicle. The lane keeping
system (LKS) is a driver convenience system which assists
a vehicle in maintaining its driving lane. These two systems
have been developed as two separate systems (Bishop,
2005). However, as the adoption rate of ACC is rising and
various marketable embedded vision systems are emerging,
the LKS+ACC system integrating both functions have
attracted more interest.
Major Japanese automakers have already produced
LKS+ACC systems. LKS of Toyota (or Lexus) maintains
its driving lane only if ACC is operating. If ACC is not
operating, it will warn the driver of lane departure by a
torque pulse (Toyota, 2004; The Tundra Solutions, 2006).
Application vehicles include Lexus LS460 (Lexus, 2008)
and Crown Majesta (The Auto Channel, 2004). Nissan has
also developed a system integrating LKS and ACC (University of Twente, 2003), which it has applied to the Cima
(Nissan, 2001). Honda developed the Honda Intelligent
Driver Support System (HiDS) integrating Intelligent Highway Cruise Control (IHCC), corresponding to ACC, and
Lane Keeping Assist System (LKAS), corresponding to
LKS (Honda, 2006a). Application vehicles are the Accord
(Pintonhead, 2006), the Legend (Honda, 2006b), and the
Inspire (Honda, 2006c).
1.2. Advantages of the LKS+ACC System
1.2.1. Reduction of driver’s workload
A considerable portion of traffic accidents are caused by
driver carelessness and improper driving maneuvers. In
particular, the burden of long hours of driving causes
drivers to be fatigued, resulting in traffic accidents. Although
conventional ACC and LKS can relieve the driver’s workload, the LKS+ACC system is expected to provide greater
workload relief. Analyzing the effect of CHAUFFEUR
Assistance on the driver using a driving simulator certified
that driving stability was enhanced and the driver’s weariness was reduced compared with separate systems
(Hogema, 2003). The result of vehicle testing of the Honda
HiDS showed that 88% of test subjects felt their workload
was reduced. Eye gaze pattern analysis indicated that
drivers with HiDS observed a wider field of view (FOV)
(Bishop, 2008).
1.2.2. Increase of traffic system capacity
The results of analysis of CHAUFFEUR Assistance on the
driver showed that drivers tended to maintain smaller headway time and change lanes less (Hogema, 2003). It was
found that the LKS+ACC system gave a greater increase of
traffic capacity than either LKS or ACC alone. Experts
have predicted that the LKS+ACC system would provide a
remarkable increase of traffic capacity when the lane width
*Corresponding author. e-mail: hgjung@mando.com
219
220
H. G. JUNG, Y. H. LEE, H. J. KANG and J. KIM
is narrow (Arem and Schermers, 2003).
1.2.3. Enhancement of control performance
In regards to lane keeping control, if ACC is not operational, it is hard to predict time to cross (TTC). Contrarily,
if ACC controls vehicle speed, LKS can easily design and
follow the driving trajectory. As a result, the control performance will be enhanced (Cho
, 2006).
In regards to ACC, if preceding roadway information
acquired by LKS is provided, ACC can implement proper
speed control taking into consideration the shape of the
road. For example, speed control on curves realizes cruise
control that suits the driver’s preference by controlling the
speed according to the road’s curve shape. Speed control at
exits contributes to a reduction of the driver’s operating
load by controlling deceleration when a car enters an exit
lane (Denso, 2004a).
et al.
1.2.4. Enhancement of recognition performance
Using lane information acquired by LKS, ACC can recognize the preceding vehicle in a curved road. Preceding
vehicle detection by using only long range radar (LRR) is
supposed to be complicated by the need to eliminate noise
from vehicle movement and vibration. Radar-based obstacle
recognition can be enhanced by using image portion
corresponding to the obstacle’s position.
One of the major disturbances of lane detection is
occlusion by the preceding vehicle. Therefore, the position
information of the preceding vehicle makes the lane detection algorithm simpler and more robust. Otherwise, the
lane detection algorithm may be complicated by the need
to handle various cases including those in which the
preceding vehicle occludes lane markings.
three layers according to distance and again divided into
the left hand side (LHS) and the right hand side (RHS). In
these six regions, lane features are searched locally. Lane
feature pixels are detected by a steerable filter and are
approximated into a line or a parabola. The orientation of
the steerable filter is initialized by peak detection of the
edge distribution function (EDF), and then established
according to the lane feature state predicted by temporal
tracking. Regions of the lowest layer are fixed but regions
of the second and third layer are set dynamically.
The conventional lane detection system works well
when there is no obstacle in the vicinity. With the recent
adoption of the high dynamic range CMOS (HDRC)
camera, traditional problems such as driving against the
sun and tunnels have been overcome (Hoefflinger, 2007).
However, if the preceding vehicle occludes lane markings
or a vehicle in the adjacent lane approaches, lane features
become lost or too small. As a result, the edges of the
obstacle start to disturb lane detection. To overcome such
problems, ROI establishment based on precise trajectory
prediction using vehicle motion sensors and lane feature
verification-based outlier rejection are incorporated (McCall
and Trivedi, 2006).
Assuming that the disturbance of neighboring vehicles
occurs because the system has no knowledge about free
space, our previous paper proposed that simple confine-
1.2.5. Benefits of ECU integration
In order to enhance recognition performance of LKS and
ACC, low-level fusion between image information and
range information is essential. Low-level fusion between
separate LKS and ACC requires over-weighted traffic load
on the communication channel. In order to enhance control
performance, an extended vehicle model incorporating
lateral and longitudinal motion is needed and the vehicle
trajectory should be designed comprehensively. Therefore,
it is expected that high performance integrated electronic
control unit (ECU) implement one vehicle model. Denso
supplied LKS+ACC ECU to Toyota, who in turn developed the fusion ECU, which processes all sensor information
including the vision sensor, the radar sensor, and the Lidar
sensor and sends control commands to the active steering
system and active braking system (Denso, 2004a, 2004b).
1.3. Adaptive ROI-Based Lane Detection
The lane detection system proposed by this paper is
fundamentally based on the monocular vision-based lane
detection system published by McCall and Trivedi, (2006)
and Guo
(2006). The forward scene is divided into
et al.
Figure 1. Lane shape depends on road shape.
SENSOR FUSION-BASED LANE DETECTION FOR LKS+ACC SYSTEM
ment of ROI to free space can efficiently prevent the
disturbance of neighboring vehicles (Jung
, 2008). In
this paper, this idea is referred to as adaptive ROI-based
lane detection. Furthermore, in the case of the LKS+ACC
system, because a range sensor is already installed for the
sake of ACC function, lane detection performance can be
improved without sensor addition. Experimental results
confirm that the proposed method can detect lanes successfully, even in the case when conventional methods fail
because of neighboring vehicles. Compared with our previous paper, this paper adds an algorithm which can account
for traffic signs on the ground and quantitative evaluation.
In particular, our experimental results show not only
daytime performance but also nighttime performance.
et al.
2. CONVENTIONAL SYSTEM: MONOCULAR
VISION-BASED LANE DETECTION
and sub-ROIs of the second and third layers are established
using the lane detection results of their lower layer (Guo
., 2006). In other words, the detected lane of sub-ROI I
determines the location of sub-ROI II and the detected lane
of sub-ROI II determines the location of sub-ROI III.
et
al
2.2. Steerable Filtering
Lanes appear as slanted edge lines in the lane searching
region. If the slope of the lane feature is known
, the
steerable filter can detect lane features more efficiently
than general edge detection methods (McCall
, 2006;
Guo
, 2006).
The steerable filter is defined using the two dimensional
(2D) Gaussian function of equation (1). If the lane marking
is regarded as a line having width, then the second
derivative is used (McCall
, 2006). If the inner edge of
the lane marking is regarded as a lane feature, then the first
derivative is used (Guo
, 2006; Mineta, 2003). In our
research, first derivatives, defined as in equations (2) and
(3), are used. Equation (2) is the derivative of (1) in the xaxis direction (θ =0°) and (3) is the derivative of (1) in the
y-axis direction (θ =90°). It is noteworthy that the equations
(1) to (3) define 2D masks.
a priori
el at.
et al.
et al.
The basic lane detection algorithm is implemented based
on state of the art methods, McCall
(2006) and Guo
(2006).
et al.
221
et
al.
2.1. Three Layered ROI Structure
Lane markings have different shapes according to the shape
of the road, as shown in Figure 1. If the road is straight, as
in Figure 1(a), all lane markings, both near and far, can be
approximated as a straight line. If the road is curved, as in
Figure 1(b), lane markings at near and far distances should
be approximated as a straight line and a curve, respectively.
ROI should be established such that the searching area is
minimized but still contains the lane features. Desirable
ROI is expected to include lane features and exclude image
portion belonging to other objects. Considering the fact that
the lane becomes smaller as distance increases, the searching area is divided into three layers whose size decrease
gradually, and then divided into a LHS and a RHS. Consequently, six sub-ROIs are established. The height of the
available searching area changes according to camera configuration and the height of each layer is defined as the
ratio to the height of available searching area. Sub-ROI I
and IV near to the subject vehicle is established and fixed,
et al.
G ( x y ) e–
,
=
∂ –
G01 ----e
∂x
o
=
(x
∂ –
G901 ----e
∂y
o
=
(x
2
(x
2
+y
2
+y
2
+y
)
2
= –2
2
)
(1)
)
xe–
=–2
(x
ye–
2
(x
+y
2
2
+y
(2)
)
2
(3)
)
The first derivative of equation (1) in a specific direction
θ is defined using equations (2) and (3), as in equation (4)
(Guo
, 2006; Mineta, 2003). The filter defined in
equation (4) outputs a strong response to edges perpendicular to the specific direction θ and outputs a weaker
response as the angular difference increases. Therefore,
because the possibility that edges of the shadow and stain
have the same orientation as a lane feature is very low, a
steerable filter tuned using
known lane feature
direction can selectively detect lane features, i.e. lane
feature pixels. Figure 3(a) is an input image and (b) and (c)
are the outputs of the steerable filter tuned to −45o and 45o,
respectively.
(4)
Gθ1 ( θ ) ⋅ G01 ( θ ) ⋅ G901
et al.
a priori
o
=cos
o
+sin
2.3. Edge Distribution Function
The EDF is used to initialize the orientation parameter of
the steerable filter. The EDF is the histogram of edge pixel
directions with respect to angle (Guo
, 2006; Nishida
, 2005). Equation (5) defines the gradient of pixel ( ,
). denotes the intensity variation with respect to the xaxis and denotes the intensity variation with respect to
the y-axis. The gradient is approximated by the Sobel
operator. With and , edge direction at pixel ( , ) is
defined as in equation (6). After edge direction of all pixels
et al.
et al.
y
x
Dx
Dy
Figure 2. Three layered ROI structure.
Dx
Dy
x
y
222
H. G. JUNG, Y. H. LEE, H. J. KANG and J. KIM
Figure 4. EDF construction and peak detection.
After dividing the EDF into two regions with respect to
90o, the maximum peak of each region is detected as shown
in Figure 4(b). The left portion corresponds to the sub-ROI
I of Figure 2 and the right portion corresponds to sub-ROI
IV. As mentioned above, lane features in the lowest layer
can be approximated by a line and the angle of the detected
peak represents the direction of the lane feature of each
sub-ROI. Therefore, the angle corresponding to the detected peak is used for the initialization of the orientation
parameter of the steerable filter.
Figure 3. Lane feature pixels detected by tuned steerable
filter.
in ROI is calculated using equation (6), the EDF can be
constructed by accumulating the pixel occurrence with
respect to edge direction. Figure 4(a) shows the Sobel
operator result of Figure 3(a) and Figure 4(b) is the constructed EDF.
∂ I ----∂ I ⎞ T ≈ ( Dx
∇ I ( x y ) ⎛⎝ ----∂ x ∂ y⎠
Dy ⎞
⎛ -----θ(x y)
⎝ Dx-⎠
,
,
=
=tan
,
–1
,
Dy
)T
(5)
(6)
2.4. Lane Feature Detection
Lane feature detection consists of steerable filtering, Hough
transformation, inner edge point detection, and model fitting. Figure 5 presents the procedure of initial lane feature
detection. A steerable filter tuned to an
known lane
direction and binarization detects lane feature pixels. Using
the lane feature pixels, a Hough transform finds the lane
feature. The second column of Figure 5 shows the Hough
transform results; the horizontal and vertical axes represent
parameters of the linear lane model. In the case of the
lowest layer, the orientation of the steerable filter is set
using the EDF and in the case of the other layers, it is set by
lane feature state tracking, which is explained below.
The initial lane feature found using the Hough transform
a priori
SENSOR FUSION-BASED LANE DETECTION FOR LKS+ACC SYSTEM
223
Figure 5. Initial lane feature detection.
Figure 6. Detected inner edge points.
is a linear approximation of pixels showing strong response
to the steerable filter tuned to a specific direction using the
voting method. By searching the edge point from the lane
feature to the image center, the inner edge points are detected as shown in Figure 6 (McCall
, 2006).
The inner edge points detected in the first layer are fitted
to a line using least square (LS) linear regression. The line
is represented by two parameters, as shown in equation (7).
The horizontal image direction is the x-axis and the vertical
image direction is the y-axis. The cross-point of the line
and a border between the first and the second layer is used
as the center x coordinates of the second layer sub-ROI
(McCall
, 2006).
y=a x+b
(7)
In the second layer sub-ROI, lane feature pixels are
detected by the steerable filter and then the inner edge
points are detected. The detected inner edge points are
fitted to a curve using LS quadratic regression. The curve is
represented by the three parameters of a parabola as given
by equation (8). The intercept of the curve defined by the
quadratic fitting and a border between the second and the
third layer is used as the center x coordinates of the third
layer sub-ROI (McCall
, 2006).
Figure 7. Dynamically established second and third subROIs.
et al.
et al.
⋅
et al.
Figure 8. The tracked lane feature state is the output of lane
detection.
y=a ⋅ x
2
+b ⋅ x+c
(8)
2.5. Lane Feature Tracking
The left and right lines are determined by fitting inner edge
points detected in the three layers, respectively, for the
LHS and RHS of an image. Then, the orientation and offset
of the left lane and the orientation and offset of the right
224
H. G. JUNG, Y. H. LEE, H. J. KANG and J. KIM
lane are used as the lane feature state. The lane feature state
is tracked by a Kalman filter such that it is robust to
external disturbance (McCall
., 2006). The confidence
level of the detected lane feature is measured, and if it
drops below a pre-defined threshold, the EDF-based initialization procedure is called. Therefore, when the subject
vehicle changes lanes, the lane features can be detected in
spite of the abrupt state variable change.
The lane feature state is tracked in such a way to be used
for the setting of the direction parameter of the steerable
filter in the next frame. Furthermore, it is used as lane
information for lane keeping control and preceding vehicle
recognition. Using the lane feature state instead of the
instantaneous lane feature detected in each frame prevents
performance degradation in the case when the lane marking is disconnected or occluded by neighboring vehicles.
Figure 8 presents an example of a tracked lane feature state.
In our previous paper (Jung
, 2008), one of the open
problems was traffic signs on the ground, which were
drawn on the road and could not be distinguished by range
data. A horizontal edge count between the LHS and RHS
lane features enables the system to recognize the existence
of a traffic sign. As the time duration to pass the traffic sign
is short, the system can eliminate the effects of the traffic
et al
et al.
Figure 9. Narrowly established ROI to cope with traffic
signs on the ground.
sign by using the area about the previously recognized lane
as the new ROI. Figure 9(a) shows a correctly recognized
lane feature using the proposed algorithm and Figure 9(b)
shows the result without it.
3. RANGE DATA-BASED ROI
ESTABLISHMENT
According to a recently published survey concerning visionbased lane detection, vision-based lane detection generally
consists of five components: road marking extraction, postprocessing, road modeling, vehicle modeling, and position
tracking (McCall
, 2006).
Reviewing the development direction of each component, one common objective can be realized. The main
challenge of road marking extraction is overcoming external disturbances such as shadow and stain, and focusing
only on the lane feature. The steerable filter used in this
et al.
Figure 10. Acquired range data in the world coordinate
system and the image coordinate system.
SENSOR FUSION-BASED LANE DETECTION FOR LKS+ACC SYSTEM
225
an adjacent vehicle are almost parallel to lane markings,
they can be falsely recognized as lane features when the
adjacent vehicle approaches near the subject vehicle or an
incorrectly established ROI is used. The shadow of an adjacent vehicle causes many problems, even when the adjacent vehicle does not approach near the subject vehicle.
In particular, a cutting-in vehicle is an external disturbance
which is difficult to identify as it is related to the update
speed of lane feature tracking (i.e. the response time).
However, it has been found that once the road surface
covered by vehicles is rejected using range data, lane
detection can simply ignore all edges generated by the
appearance of neighboring objects. Furthermore, it is noteworthy that such a procedure can be implemented by
simple operation, which is finding the image position
corresponding to range data and masking off the area from
the ROI. Denoting the image pixel coordinates ( , ) and
world coordinates of range sensor by ( , , ), these
two coordinates are related by homography H as follows:
xi yi
Xw Yw Zw
Figure 11. Recognized free space.
paper is developed to improve lane detection performance
by focusing on edges having the expected orientation. Postprocessing is aimed at eliminating falsely detected lane
features caused by external disturbances using
knowledge regarding the road and lane. Road modeling,
vehicle modeling, and position tracking are aimed at efficiently narrowing the searching area by formularizing lane
marking shape, vehicle motion, and lane marking motion.
In other words, they are developed to establish the ROI
only at a region where the lane feature is expected to
appear in the next frame considering the current position of
the lane marking, vehicle motion, and lane marking structure. Consequently, external disturbances can be ignored
and lane detection performance can be improved. The common objective of component development is minimizing
the effect of external disturbances. We pay attention to the
fact that external disturbances are inevitable because they
are caused from dimension reduction from a 3D world to a
2D image. This means that once the external disturbance
can be identified in advance, complicated post-processing
and modeling can be simplified.
Assuming the most important external disturbance of
lane detection is neighboring objects including the preceding vehicle, adjacent vehicles, and guide rail, it can be expected that simply by confining the lane feature searching
area to a free space ensured by range data, lane detection
performance will be improved. When the preceding vehicle
approaches near to the subjective vehicle, it occludes lane
markings, and the edges of its appearance can be falsely
detected as a lane feature. Because the side surface edges of
a priori
Xb
xi
Yb = H ⋅ yi
Zb
1
(9)
Xw = Xb / Zb
Zw Yb / Zb
(10)
In order to acquire coordinates of the road surface, is
set to 0. The homography H of equation (9) is defined as in
equation (11). denotes the camera height and θ and ϕ
denote the yaw angle and tilt angle of camera, respectively
(Jung
., 2004).
Yw
hc
et al
H=
–hc ⋅ cos θ –hc ⋅ sin θ ⋅ sin ϕ f ⋅ cos ϕ ⋅ hc ⋅ sin θ
hc ⋅ cos θ –hc ⋅ cosθ ⋅ sin ϕ f ⋅ cos ϕ ⋅ hc ⋅ cos θ
0
cos ϕ
f ⋅ sin ϕ
(11)
Figure 10(a) shows range data acquired by a scanning
laser radar. The range data are acquired in the polar coordinate system and then transformed into the Cartesian coordinate system. Figure 10(b) indicates range data projected
onto the input image. It can be seen that the positions
where the vehicles and the guide rail meet, the road surface
is successfully detected. However, the range data are
disconnected in several positions and contains noise.
Clustering range data eliminates disconnected and added
noise. Scanning consecutive range data points, if two range
data points are far more than a threshold apart, e.g. 50 cm,
then they are recognized as a border between two range
data clusters. Among recognized clusters, clusters with too
small points or too short length are eliminated and then the
deleted region is interpolated by adjacent clusters. The area
below the border line consisting of recognized range data
clusters and the sky line is recognized as free space, to
which the lane feature searching region is confined. Figure
11 provides an example of recognized free space. Figure
226
H. G. JUNG, Y. H. LEE, H. J. KANG and J. KIM
Table 1. Detection performance of the proposed method
and conventional method.
Proposed
Conventional
method
method
Daytime
93.17%
92.13%
(2,491 frames)
Nighttime
99.04%
98.82%
(5,100 frames)
11(a) shows horizontal line-based clusters and range databased clusters. By combining these two kinds of clusters,
the free space is constructed, as shown in Figure 11(b).
Each of the six sub-ROIs is defined by a rectangle whose
four corners are established so as to be located in the free
space.
4. EXPERIMENTAL RESULTS
In order to verify the feasibility of the proposed range databased adaptive ROI establishment, we installed scanning
laser radar and a camera on the test vehicle and compared
lane detection performance of the proposed method with
the conventional method. A brief summary of the specifications of the scanning laser radar (SICK LD-OEM) is as
follows: the FOV is 360o, the angular resolution is 0.125o,
the range resolution is 3.9 mm, the maximum range is 250
m, the data interface is a controller area network (CAN),
and the laser class is 1 (eye-safe). The resolution of the
image is 640×480. Each image and range data was recorded with a speed of 10 frames per second. In total, 2,491
data images were recorded for daytime and 5,100 data
images for nighttime. Table 1 compares detection performance. It can be seen that the proposed method shows
better detection performance than the conventional method.
Although the difference in detection performance is small,
it is significant because situations related to the difference
are potentially dangerous, e.g. when there are closing
vehicles or when a vehicle cuts in suddenly.
Figure 12 shows that the proposed adaptive ROI can
overcome a disturbance from an adjacent vehicle. Figure
12(a) displays the input image and (b) and (d) show lane
feature pixels detected on the LHS of the input image by
the conventional method and the proposed method, respectively. It can be seen that the bottom edge of the adjacent
vehicle looks similar to the lane feature. Figure 12(c) and
(e) indicate the lane feature state detected by the conventional method and the proposed method, respectively.
They show that the problem that the left lane feature is
incorrectly detected by the conventional method in Figure
12(c) can be solved by the proposed method, as shown in
Figure 12(e). It is noteworthy that neighboring vehicles are
excluded in the free space, which is depicted in Figure
12(e).
Figure 13 and Figure 14 show examples when the
Figure 12. Comparison when adjacent vehicle approaches.
Figure 13. Comparison when the preceding vehicle
occludes left lane markings wholly.
SENSOR FUSION-BASED LANE DETECTION FOR LKS+ACC SYSTEM
227
Figure 15. Successful cases.
Figure 14. Comparison when little lane markings are
observable.
proposed method overcomes problems caused by the preceding vehicle. Figure 13 is a situation when lane markings
are disconnected at the current location and the preceding
vehicle occludes the remaining lane markings so that there
is no useful information available on the left lane feature.
The proposed method realizes that there is no useful
information and then maintains a tracked lane feature state
to output proper lane information. Figure 14 shows a
situation where there are few observable lane markings. As
the proposed method eliminates the image portion occupied by the preceding vehicle, it can focus on observable lane
markings. In contrast, the conventional method fails because
of the vehicle’s edges.
Figure 15 demonstrates that the proposed method can
successfully detect lanes in various situations. Figure 15(a)
shows a situation where there is wide free space in front of
the subjective vehicle. Figure 15(b) and (c) show situations
where there are a lot of shadows on the road surface. Figure
15(d) indicates a situation where the cutting-in vehicle
occludes right lane markings. In this case, although lane
markings in the near area are occluded, the tracked lane
feature state helps in finding the lane markings in the far
area.
Figure 16 shows results during nighttime; Figure 16(a)
and (c) shows the results of the conventional method and
Figure 16(b) and (d) shows the results of the proposed
Figure 16. The recognition results during nighttime.
method. As shown in Table 1, the detection performance
during nighttime is better than during daytime. This is
because lane markings become distinctive by narrow headlamp light-beams while other objects, which may disturb
the system during the daytime, can not be observed in the
dark. In other words, as the headlamp established an effective ROI for lane detection, the disturbance caused by
environmental objects was reduced without any additional
operations.
5. CONCLUSION
This paper proposes a method which prevents external
disturbance caused by neighboring vehicles by confining
lane detection ROI to free space confirmed by range data.
Experimental results show that the detection performance
of the proposed method is better than that of the conventional method. Although the difference between the
detection performances is small, the proposed method is
228
H. G. JUNG, Y. H. LEE, H. J. KANG and J. KIM
expected to significantly enhance the safety of the system,
as it correctly recognizes lane features even with an adjacent vehicle and a cutting-in vehicle. The main contribution of this paper is showing that a range sensor can
enhance lane detection performance and simplify the lane
detection algorithm. In particular, the proposed approach
confining the ROI based on range data can be implemented
by CAN communication even if ACC and LKS are implemented in separated ECUs, as in conventional implementation. Therefore, this approach requires only a minimal
change and may be easily adopted. Future studies will
focus on 1) real time implementation of the proposed
method on an embedded platform and 2) replacement of
the high angular resolution scanning laser radar used in this
paper with Lidar or Radar with low angular resolution.
REFERENCES
Arem, B. and Schemers, G. (2003). Exploration of the
traffic flow impacts of combined lateral and longitudinal
support. Available at http://www.aida.utwente.nl/en/
research/Seminar/slides_GS.ppt. Accessed on 17 May
2008.
Bishop, R. (2005). Intelligent Vehicle Technology and
Trends. Artech House Inc. Norwood. MA.
Bishop, R. (2008). Societal benefits of in-car technology.
Available
at
http://www.ivsource.net/public/ppt/
BishopDutchRttF.ppt. Accessed on 17 May 2008.
Cho, J. H., Nam, H. K. and Lee, W. S. (2006). Driver
Behavior with Adaptive Cruise Control. Int. J. Automotive
Technology 7, 5, 603−608.
Denso (2004 a). Sensing system. Available at http://www.
denso.co.jp/en/its/2004/products/pdf/sensingsystem_e.pdf.
Accessed on 17 May 2008.
Denso (2004 b). 11th ITS world cong. – Exhibited product
lineup. Available at http://www.denso.co.jp/en/its/2004/
products/safety2.html. Accessed on 17 May 2008.
Fritz, H., Gern, A., Schiemenz, H. and Bonnet, C. (2004).
CHAUFFEUR assistant: a driver assistance system for
commercial vehicles based on fusion of advanced ACC
and lane keeping. 2004 IEEE Intelligent Vehicle Symp.
495−500.
Guo, L., Li, K., Wang, J., and Lian, X. (2006). A robust
lane detection method using steerable filters. The 8th Int.
Symp. Advanced Vehicle Control, 2006.
Hoefflinger, B. (2007). High-Dynamic-Range (HDR) Vision.
Springer Berlin Heidelberg.
Hogema, J. (2003). Driving behavior effects of the chauffeur assistant. Available at http://www.aida.utwente.nl/
en/research/Seminar/slides_JH.ppt. Accessed on 17 May
2008.
Honda (2006 a). Safety for everyone in our mobile society.
Available at http://world.honda.com/CSR/pdf/CSR-067-Safety-Initiatives.pdf. Accessed on 17 May 2008.
Honda (2006 b). Honda introduces the all-new legend.
Available at http://world.honda.com/news/2004/4041007.
html. Accessed on 17 May 2008.
Honda (2006 c). Honda announces a full model change for
the inspire. Available at http://world.honda.com/news/
2003/4030618_2.html. Accessed on 17 May 2008.
Jung, C. R. and Kelber, C. R. (2004). A robust linearparabolic model for lane follow. Proc. XVII Brazilian
Symp. Computer Graphics and Image Processing, 10,
72−79.
Jung, H. G., Lee, Y. H., Yoon, P. J. and Kim, J. H. (2008).
Forward sensing system for LKS+ACC. SAE Paper No.
2008-01-0205.
Lexus (2008). The new LEXUS LS460. Available at http:/
/www.garagecordi.com/ls460.pdf. Accessed on 17 May
2008.
McCall, J. C. and Trivedi, M. M. (2006). Video-based lane
estimation and tracking for driver assistance: survey,
system, and evaluation. IEEE Trans. Intelligent Transportation Systems 7, 1, 20−37.
Mineta, K. (2003). Development of a lane mark recognition system for a lane keeping assist system. SAE Paper
No. 2003-01-0281.
Nishida, M., Kawakami, S. and Watanabe, A. (2005).
Development of lane recognition algorithm for steering
assistance system. SAE Paper No. 2005-01-1606.
Nissan (2001). Nissan releases all-new cima. Avaiable at
http://www.nissan-global.com/EN/NEWS/2001/_STORY/
010112-01.html. Accessed on 17 May 2008.
PistonHeads (2006). HONDA ADAS. Available at http://
www.pistonheads.com/doc.asp?c=52&i=15032. Accessed on 17 May 2008.
The Auto Channel (2004). Toyota crown majesta undergoes complete redesign. Available at http://www.
theautochannel.com/news/2004/07/07/202727.html.
Accessed on 17 May 2008.
The Tundra Solutions (2006). 4th-generation lexus flagship
luxury sedan features …. Available at http://
www.tundrasolutions.com/forums/toyota-scion-andlexus-news/58916-fourth-generation-lexus-flagshipluxury-sedan/. Accessed on 17 May 2008.
Toyota (2004). Environmental & social report 2004. Avaiable at http://www.toyota.co.jp/en/environmental_rep/
04/download/pdf/e_s_report_2004.pdf. Accessed on 17
May 2008.
University of Twente (2003). Summary_seminar: combination and/or integration of longitudinal and lateral support. Available at http://www.aida.utwente.nl/en/research/
Summary_seminar/. Accessed on 17 May 2008.