JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
1
Robust Lane Detection and Tracking based on
Cascaded Feature Extraction and Inter-frame
Similarity
Saumya Srivastava1 , Rina Maiti1 ,
1 Center for Product Design and Manufacturing, Indian Institute of Science, Bangalore, India
ssaumya@iisc.ac.in., rmaiti@iisc.ac.in
Abstract—Despite the development of several vision-based lane
detection methods in past decades, lane detection is still a challenging issue in the computer vision community. The presence of
shadows, obstacles, different symbols, and changing illumination
conditions makes the task of detecting the road demanding. This
paper proposes a robust framework for multilane (ego, right,
and left) detection and tracking system. Two algorithms are
proposed for extracting lane features, one takes into consideration
the edge information of the image whereas another algorithm
is based on extracting white and yellow lane marks using HSI
color space transformation. Straight Hough lines and curved lines
are plotted on the extracted lane mark using a first and second
algorithm, respectively. Proposed work utilizes Inverse perspective
mapping for the following three aspects: (1) Implementation of
Noise elimination module in the pre-processing stage (2) detection
of a pattern of broken lines (3) Curve fitting of the quadratic
polynomial model using least square error for both, continuous
and broken lane marks. (4) Execution of candidate line selection
Module. Kalman filter is used for continuous tracking of lane line
location and its corresponding lane parameters.
Index Terms—Hough Transform, Lane Detection, Advance drive
assistance system.
I. I NTRODUCTION
Multilane detection system plays a crucial role in many
advanced driver assistance modules, such as It gives
information about the ego lane as well as lane next to ego
lanes. Which could be further used for Lane Departure Warning
Systems (LDWS), Lane Keeping Assistance Systems (LKAS)
using single lane detection, and Lane Changing assistance Systems (LCAS). Lane detection primarily consists of three stages:
extracting lane features which are based on some visual clue,
model fitting, and then tracking. In the presence of different
sensing modalities in the market, such as lidar, radar, and GPS,
the Vision modality, being low-cost and correlated to the human
visual systems, is prominently used for gaining the perception
of the vehicle’s surroundings.
There are various sensing modalities used for road and lane
understanding, including vision (i.e. one video camera), stereo,
LIDAR, vehicle dynamics details procured from car odometry
A
Manuscript created October, 2024; This work was developed by the IEEE
Publication Technology Department. This work is distributed under the LATEX
Project Public License (LPPL) ( http://www.latex-project.org/ ) version 1.3. A
copy of the LPPL, version 1.3, is included in the base LATEX documentation of
all distributions of LATEX released 2003/12/01 or later. The opinions expressed
here are entirely that of the author. No warranty is expressed or implied. User
assumes all risk.
or Inertial Measurement Unit (IMU) with global positioning
information obtained GPS and digital maps. Vision is the most
prominent research area in lane and road detection due to the
fact that markings are made for human vision.
Advance driver assistance systems which either alert drivers
in a dangerous situation or take actives part in driving, are gradually being introduced into vehicles. So, there exists a bridge
between the driver and their perception about the surrounding
of the vehicle. The bottleneck to tackle the perception problem
has two elements: road and lane perception, and obstacle (i.e.,
vehicles and pedestrian) detection. Road and lane understanding
includes detecting the extent of the road, the number and
position of lanes, merging, splitting, and ending lanes and roads,
in urban, rural, and highway scenarios. Also, it is shown in
papers that by including vehicle tracking, that lane tracking
performance robustness, localization, and temporal response get
improved. Multilane detection is the indispensable area when
it comes to providing better perception of surroundings of the
vehicle.
However, these feature extractions using visual clues are
susceptible to varying illumination,
This paper is organized as follow: section 2 briefly reviews
related works on lane detection system. Section 3 describes
proposed lane detection system. Section 4 shows the details of
the experimental work, results along with the limitation of the
proposed method. Finally, section 5 concludes the paper with
inclusion of future work.
Lane detection comprises of three elements in its pipeline:
feature extraction, model fitting and tracking [1]. With the
intent to extract features, many studies explored the property
of white and yellow colors associated with longitudinal lane
marks [2], [3]. However, varying illumination and cast shadow
on road surface due to trees, vehicles, building etc. provide
hindrance in collating these color clues. To circumvent these
conditions, studies have taken color-space transformations into
consideration [4], [5], [6], [7], [8]. These color model disassociates the intensity constituent, the achromatic section, from
the color carry information, constituting the chromatic section.
Using the invariance property of lane color, different local or
global techniques such as histogram-based segmentation [3],otsu
thresholding [9] clustering method [8] adaptive thresholding
[10], are exploited to detect lane marks. However, these methods
are susceptible to conditions where during different illumination,
road surface is also classified into lane markings.
JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
Another well-known method to extract lane features are based
on the image gradient or edges. Some studies [25], [22] have
explored gradient properties by using oriented gaussian kernels,
which prove effective for detecting bright lane marks against
a dark background. However, these methods face challenges
in situations with varying illumination and instances where
noise, such as from certain parts of white cars, results in sharp
gradients. Other studies employ detectors like Canny [11], [12]
and Sobel detectors [13], known for their low computational
cost and ability to yield valuable structural information about
lane marks. Various improvements to canny edge detection has
been introduced in the literature [14], [15] to effectively address
noises in the road environment. Wu et al. [16] extracted lane
mark edges by using predefined local edge orientation range and
multi-adaptive thresholding algorithm on sub image level. In
[13], Kang et al. proposed a combination of local line extractor
and connected component algorithm to extract candidate lines of
road lanes in the sub-region. Another method for enhancing lane
feature is presented in the work by Yuan et al. [17], wherein
modified local adaptive threshold is applied to segment road
pavement. Additionally,a denoising operation with constraints
on both area and orientation, is applied to refine the results. Li
et al. [18] utilized HSV color transformation to extract white
lane marks, followed by edge detection of binarized image in
ROI area. For all these methods, noise removal is post requisite
step after edge detection. However, challenges arise during noise
eliminating such as (1) presence of misleading edges originating
from artifacts such as strong,aligned shadows of trees, buildings,
or other infrastructure, road marking patterns, cracks, and stains.
These elements exhibit a similar orientation to lane mark edges,
resulting in the failure of accurate lane edge extraction.(2) risk
of inadvertently wiping out small edges corresponding to broken
lane marks in the course of noise removal. (3) instances where
lane edges are embedded with clutter. In attempt to remove
noise, valuable information is often unintentionally erased and
alters the structural information of the lane edges. This exerts
a significant impact on the overall effectiveness of subsequent
processes.
Numerous studies have been presented on the lane detection
through the application of inverse perspective mapping [26],
[28], [29].This geometrical transformation aims to eliminate perspective effects by homogeneously remapping picture elements
to a new reference frame, thereby providing a top-down view of
the road scene in front of the camera. Consequently, this process
restores parallelization between lane markings, facilitating the
determination of road deflection. Combination of ridge features
and IPM has been used [30]. Though the method is independent
of camera calibration and can withstand changing pitch angle
but fails in handling roads with varying curvature. In [31], Jiang
et al. proposed a method where initially central lane is detected
by using perspective transformation. However, method fails did
not count for the scenarios where visual clue of central lane
are absent. In [32], proposed method uses conditional random
fields that works well for both, parallel and non-parallel lanes
situation. However, performance of the method degrades in the
anomaly of normal weather condition or in the presence of
leading vehicle which occludes lane markings. Zhaom et al.
[33] needs additional information such as lane width, vehicle
speed and direction in order to design a model. This additional
2
information requires higher costs. These approaches works
well under the assumption of flat road;however transformation
is vulnerable to vehicle vibrations. Additionally, the leading
vehicle can obstruct the lane mark, causing artifacts in the IPM
transformation.
II. R ELATED W ORK
Here, we discuss several existing methodologies aimed at
exploiting inter-frames similarity technique, highlighting their
effectiveness but also acknowledging their limitations in accurately detecting lanes across diverse environmental conditions.
A. Conventional tracking methods
In the domain of lane detection,tracking is often used to deal
with scenarios where lane markings are difficult to detect. [35],
[43], [27]. In [16], Wu et al. used kalman tracker to track end
coordinates of the lane marks. Other approaches involve the
use of hough transform-derived parameters, such as ρ and θ, to
define the state vector for the kalman filter [28], [42]. However,
kalman filter performs optimally only when the underlying noise
follows a gaussian distribution.
B. AROI, LBROI
In line with the concept of inter-frame similarity,numerous
studies define a region of interest (ROI) around the detected
lane markings.This ROI serves as a predictive space where lane
markings are expected to be located in the subsequent frame.
By constraining the search area, not only does it eliminates
background noise, but also reduces computational overhead.
Jung et al.[19] proposed the lane boundary region of interest
(LBROI), to search edge information about lane boundaries
in the next frame. Similarly, Yuan et al. [17] introduced the
adaptive region of interest (AROI),which dynamically adjusts
its size and position based on variations in lane curvature and
vehicle speed. While the method is effective in scenarios like
lane changes by re-identifying the starting point of the lane
for every 20th frame, it falls short in cases where consecutive
frames following the 20th frame produce false positive or
negative detection outcomes. This limitation has the potential to
propagate false AROIs until the next re-identification process is
executed.
C. Λ-ROI
In Lee et al. [40] work, apart from establishing Λ-ROI , interframe clustering is also employed, where aggregated statistical
information about slopes,lower intersection locations, vanishing
points and lane widths are clustered over a designated frame
sequence. The aggregated data is then tracked through the use
of a kalman filter, which plays a crucial role in handling missing
lane markings. Their methodology works well if noise is inconsistently present over frames. Λ-ROI with distorted trapezoid
shape and variable boundaries ensures continued detection of
lane markings despite changes in lane width or vehicle shifts
horizontal beyond the typical range. For detection part, their
methodology used scan-line test that is based on low-high-low
intensity profile of lane markings and their derivatives.However,
line segments from other vehicles, shadows, or buildings may
survive the slope filtering and the clustering if they are parallel
to the lane markings.
JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
D. left and the right lanes are tracked dependently
In the study by [41], a spatiotemporal image is constructed by
integrating pixel intensities along a scanline across the temporal
axis. Their methodology relies on the presumption of temporal
continuity in lane width. When one of the lanes (left or right)
is absent while the other remains intact, the method leverages
the temporal continuity of lane width for detection. However,
in scenarios where both left and right lanes are fully obscured
by obstacles, detection becomes unattainable.
E. similarity for location consistency and estimated vanishing
points
Yoo [34] utilized inter-frame similarity for location consistency and estimated vanishing points, introducing a probabilistic
voting function based on a line segment strength parameter that
quantifies the relevance of extracted line segments.
All these approaches relies on the assumption of a consistent
presence of lane markings at similar locations across consecutive frames. However, real-world driving scenarios often involve
non-linear events such as sudden change of ego lane and the
introduction new lane marks or discontinuation of existing lane
marks (e.g., parking marks). While highway lanes typically
maintain constant widths, urban roads often see fluctuations
in lane marking locations and widths, particularly around road
junctions.These dynamic driving conditions pose challenges to
the validity of the inter-frame similarity assumption, as they
lead to changes in parameters such as lane width, slope, and
position of the ego lane over time.**3) accuracy of tracking in
end depends upon accuracy if detection. If the dynamic of road
changes, then motion model also goes obselete.
motion model, lane -width, ROI
Didnot address the validity of new detection
latency is involve, which can be overcome if more sensor
information are fused together.
THOUGHT** For a established ROI, no lane marking is
detected, kalman will take it as a missing detection and predict
the lane marking.However, a different lane marking exist beyond
the ROI, therefore false positive will arrive.Similarly, with no
lane marking detected, for the next frame width of the ROI
will increase and detect a lane-like noise over persistent frame
and take it new lane marking.What happened to the results
obtained from kalman filter.!!! no proper reasoning/ explanation
is given; open ended;Discussion needs to be done; conflict
arises;ambuigity
In the proposed work, two modules are proposed. First
one deals with detection and second deals with inter-frame
similarity. For the detection of l,both local statistical and geometrical properties of lane marking are exploited using two stage
feature extraction. The first stage involves adaptive selection of
candidate threshold values on a sub-image level, considering
the geometrical constraint of pixel-width on longitudinal lane
marks.Threshold value is validated using contrast offset and
peak offset,with various margin values used to update final
threshold. However, erratic patches apart from longitudinal lane
marks with pixel value above threshold can surpass stage 1st .
To eliminate such noise, denoising operation is performed in
the second stage. Using vertical aggregation technique,peaks are
selected to define window between two valleys. Subsequently,
3
thresholding is performed on the window level and hough
transform is applied to select only aligned pixels. Straight line,
implemented with RANSAC is applied for curve fitting in cases
of both solid and broken lines. Once the detection is made,
inter-frame similarity module is introduced. For each frame in
a single span, deviation factor is calculated by comparing the
detection with previous estimated output of the kalman filter. If
the deviation factor falls within the tolerable pixel shift (TPS),
the kalman filter incorporates the detection to estimate the output
in the current frame; otherwise, it disregards the detection and
uses prediction for the current frame estimation. A dynamic area
of interest is established to search for the lane in the next frame.
However, to address scenarios such as lane change,introduction
of new lane marks, full frame detection mode is applied after
every 10th frame.Verification protocols are followed , where if
new detection is incompatible with results from the previous
DAOI, a flag is raised, prompting the system to switch to fullframe detection mode for the next frame. Flag is raised till the
detection result from full-frame becomes stable and DAOI is
re-established with re-initialization of kalman filter.
This work presents three primary contributions:(1) presents
the detailed insights into solution for the challenges of lane
detection in the real world scenarios like illumination variation,
noises, vehicle occlusion, lane change and introduction of new
lane marks (2) based on both local statistical and geometrical
properties of lane marking, our method iteratively prunes the
lane mark features,on both,sub-image level and window level
by using two stage feature extraction level. (3) In addition,
significant attention is paid to remove the noise signals from the
background. (4) This work establishes successful co-existence
by integrating detection with tracking and establishment of
DAOI.
III. P ROPOSED M ETHOD
In the proposed work, we deal with two modules: 1) Lane
detection comprising feature extraction and curve fitting 2)Interframe similarity involving DAOI establishment and the tracking
system. It begins with extracting the individual frames from the
video and transforming each frame to inverse perspective mapping to detect lane markers. To get the transform, it is imperative
to have the intrinsic (camera optics, skew factor) and extrinsic
(position, orientation) parameters of the camera sensor which
deals with camera orientation, location optics and tilt. These
matrices establish the relationship between camera coordinate
and image coordinate. Longitudinal lane marks typically present
as lines with yellow or white tint, appearing in continuous or
dashed patterns, offering high visibility and contrast against the
road surface. To extract this distinctive feature, the RGB color
values initially undergo conversion into the HSV color space.
This transformation not only aligns better with human vision but
also facilitates the separation of chromatic components (hue and
saturation) from achromatic intensity, enhancing the precision of
feature extraction.
A. Lane Detection
Despite the restoration of lane marking parallelism, challenges still persist in the feature extraction process. These
challenges include 1) instances of occlusion of lane marking by
JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
Fig. 1. Challenges associated with lane mark feature extraction after IPM transformation. First row displays the original image for reference. (a) ∼ (c)artifacts
arise due to leading vehicle and (d), saturation of image scene due to intense
ambient light, (d) ∼ (e)presence of severe shadow.
vehicles leading to artifacts in the IPM as depicted in Fig. 1(a),
(b). 2) Risk of misclassifiying objects as lane mark candidates
due to similarities in intensity and alignment, allowing them to
pass through thresholding process. As illustrated in Fig. 1(c), the
edges of the car exhibit similar orientation to that of the lane
marking. 3) The varying illumination across the road scene. For
instance, intense glare induced by background light or shadow
cast by trees, can alter the dark-bright-dark pattern of lane
markings against the road surface as shown in Fig. 1(d) and
Fig. 1(e), (f) respectively. To address these challenges, both
local statistical and geometrical properties of lane marking are
exploited in our work. Based on the skewness of histogram,
images in our proposed work methodology is classified as dark
or bright image. Furthermore, each IPM image is partitioned
into left half section and right half section using the central
demarcation line at xm as shown in Fig. 2(a) . Two stage feature
extraction is employed wherein first stage threshold values are
adaptively selected on sub-image level and in the second stage
denoising operation is performed on window level.
1) Adaptive Local Thresholding: To begin with, both section
of image is initially divided into sub-image is of size r ∗ c such
that r = 2c and for each sub-image, a histogram is constructed
for pixel range varies from [0, (L − 1)]. Since, the longitudinal
lane markings have distinguished property of being constant in
pixel width taken as w, cumulative distributive function of the
sub-image is then obtained such that cumulation satisfies the
following constraint:
( 0
!)
0
X
X
b∗w∗r
cdf(X) =
P (X) |
P (X) ≤
(1)
L−1
k=L−1
k=L−1
where P(X) is probability density function which is defined as
P (Xk ) = nk /N , L is the total number of gray levels in the
image, N = r ∗ c is total number of pixels and nk is the
cardinality at gray level k. The underlying assumption is that the
lane marking consistently exhibits a brighter contrast compared
to the road surface and typically occupies the right extrema of
the histogram. The pixel index at which the cumulation stops
as per the above-mentioned criteria is identified. Subsequently,
to increase adaptability to diverse conditions, a margin, denoted
as m1 and typically set to 20, is subtracted from the obtained
pixel index. The resulting value is defined as candidate threshold
index, th′ . It is validated only if it satisfies the following two
4
Fig. 2. Adaptive Local thresholding is performed over an image (a) sub-image
is chosen from left half section of IPM. The central demarcation line at xm
is depicted in orange (b) zoomed-in representation of sub-image (c) generated
histogram of the sub-image (d) zoomed-in image shows contrast offset and peak
offset (e) Accumulated pixels with red peaks beyond the candidate threshold th′
from(d) are highlighted, where sk′ represents the average cardinality for pixel
intensities ranging from [th′ , L − 1] (f) sub-image obtained after performing
Adaptive Local Thresholding (g) final image
conditions: 1) contrast offset should be greater than τ1 and 2)
peak offset greater than τ2 , where :
Contrast offset = max (k ′ ) − arg max (nk )
(2)
Peak offset = max (nk ) − sk′
(3)
k
where sk′ is the average cardinality for pixel intensity ranging
from [th′ , L − 1]
L
sk′ =
X
1
nk
′
th − L
′
(4)
k=th
and k ∈ [1, L − 1], k ′ ∈ [th′ , L − 1]. For a selected candidate
threshold value, high value of contrast offset signifies higher
contrast between foreground (lane marking) and background
(road surface). Similarly, high peak offset validates the presence
longitudinal lane marking with area w ∗ r embedded within a
sub-image of area r*c. Pixels surpassing the validated threshold
value are retained, while those falling below are assigned to
value 0. However, absence of a distinct peak and a flat histogram
indicate minimal contrast for a given sub-image. Consequently,
all pixels within such a sub-image are considered as background
and are assigned a value of 0. The outcome of local thresholding
with parameters b = 1/7, w = 7, τ1 = 30 and τ2 = 10 is
illustrated in Fig 2.
2) Window Denoising: It is evident from Fig. 3(c) that due
to varying illumination across the image, the application of
adaptive local thresholding yields an image with erratic patches
along with white longitudinal lane markings .To address this,
denoising operation is executed on window-level. To define
window, histogram detailing the vertical aggregation of pixels is
initially generated over resultant image. Under the assumption
that lane marks maintain a minimum separation of at least 40
pixels, significant peaks are identified, as shown in Fig.3 (d).
Subsequently, valleys located between these peaks are identified
to define windows across the image, thereby signifying the presence of lane markings within each specified window. As can be
seen in Fig. 3(e), two windows are designated across the rightside of IPM using vertical pixel aggregation.For each window,
the pixel with the highest intensity is selected and updated by
JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
5
have used tracking techniques. Kalman filter works well under
the assumption of gaussian noise. It predicts well for the
spurious detection. However, for the sudden change in the
driving maneuver, kalman filter will consider new location as
noise and doesn’t take it into account. In our proposed work
two kalman filters are used to track the detected lane in the
subsequent frames, one for left lane marking and other for right
lane marking. It consist of two stages, first stage predicts the
present state of the target as per equation described below:
Fig. 3. (a) Original image (b) Right half section of IPM (c) Image obtained
after applying adaptive local thresholding.The resultant image has erratic patches
along with lane marks (d) vertical pixel aggregation over the image (e) Hough
lines drawn on each window.
subtracting the margin m2 and subsequently identified as the
threshold. However, in the case of classified dark image with
reduced ambient light such as those captured in cloudy, dawn
and night scenes,has noise with intensity higher than that of
longitudinal marking. These noises readily bypass the adaptive
local thresholding in the preceding stage and results into sharp
trailing lines. To address such scenarios, threshold is determined
by selecting the index of arg maxk (nk ) with subtracted margin,
m3 . Hough lines are then laid over the resultant window which
not only removes further noise from the image, but also chooses
the aligned non-zero-pixels. Moreover, dilation is applied to
add pixels to object boundaries which helps in highlighting
the lane features.The images displayed in the first and second
rows of Fig. 4 represent the original and resultant IPM images,
respectively. The third row illustrates the image post-adaptive
local thresholding, while the fourth row displays the denoised
output image obtained with parameters m2 = 40 and m3 = 5.
Lane marks corresponding to ego lane is defined by farthest
peak and nearest peak in left half section and right half section
of image respectively. Second order polynomial function with
RANSAC is used for plotting curve in case of both, solid and
broken lines over each window.
B. Inter-frame Similarity
Inter-frame similarity in terms of temporal and spatial continuity
is a widely employed concept in various research endeavors. In [17], Yuan et al. defined adaptive region of interest
(AROI),wherein the position of lane marks in the previous frame
are used to predict their location in the subsequent frame. While
the method is effective in scenarios like lane changes by reidentifying the starting point of the lane for every 20th frame,
it falls short in cases where detection result in false positives or
negatives in the 21st frame. This limitation has the potential to
propagate false AROIs until the next re-identification process is
executed.
We make our system robust by utilizing the inter-frame
consistency in two aspects 1) for tracking lane mark and 2)
for defining the Dynamic Area of Interest (DAOI).
1) Tracking: Lane markings play a pivotal role in our
methodology; however, factors such as occlusion by leading
vehicles, degradation due to wear and tear, pronounced shadowing, noise, and fluctuating illumination conditions can hinder
their availability for extraction in every frame. To increase
robustness of the system, numerous studies in the literature
+
û−
t−1 = A × ût−2 + W
(5)
where ut is a vector containing the state , which in our case are
the parameter of lane. Measurement vector Z at any time t for
left and right lane marking consist of quadratic parameters :
Zl = [ α0 , α1 , α2 ]l
(6)
Zr = [ α0 , α1 , α2 ]r
(7)
While initialization, velocity component are set to zero and
A is a process transition matrix which establishes the relation
between previous and current state in dt time difference. It then
advances to estimate error covariance Pt−
−
Pt−1
= A × Pt−1 × At + Q
(8)
−
Where Pt−1
is a priori estimate error covariance and Q is
the process noise (white noise). Second stage plays role in
correction of the predicted state using the measurement. Kalman
gain K is computed to correct the estimated state as:
−
−
Kt−1 = Pt−1
× (Pt−1
+ Rt−1 )
−1
−
−
û+
t−1 = ût−1 + Kt−1 (zt−1 − H ût−1 )
(9)
(10)
It concludes the tracking stage by updating the error covariance
Pt− to Pt :
−
Pt−1 = (I − Kt−1 ) × Pt−1
(11)
where Pt is a posteriori estimate error covariance.Left and right
lane parameters are calculated from kalman state space equation
as:
Fig. 4. Row 1 depicts the original image, Row 2 demonstrates the Inverse
Perspective Mapping (IPM), Row 3 showcases the result post-application of
adaptive local thresholding on a sub-image level, and Row 4 exhibits the final
output after window denoising
JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
6
i
h
+
+
+
α
c0 t−1 , α
c1 t−1 , α
c2 t−1 = û+
t−1 [1 : 3]
(12)
i
h
+
+
+
α
c0 t−1 , α
c1 t−1 , α
c2 t−1 = û+
t−1 [1 : 3]
(13)
r
l
By using the estimated parameters of the kalman filter, bottom
and top x-coordinate of the fitted curve for left and right is
estimated as follows:
1 +
C +
l x̂t−1 , l x̂t−1
=
o
n
+
+
+
c0 t−1 , α
c1 t−1 , α
c2 t−1 , yϵ1, C
(14)
x y = fl x, α
1 +
C +
r x̂t−1 , r x̂t−1
=
o
n
+
+
+
(15)
c0 t−1 , α
c1 t−1 , α
c2 t−1 , yϵ1, C
x y = fr x, α
Deviation factor (DF) for both the left and right lane marking
is estimated by subtracting the detected x-coordinate of the
bottom-most and top-most point of the lane marking at time
t from its previous estimated value at time t − 1 as
DF l =
1
C
1 +
C +
l xt , l xt ) − (l x̂t−1 , l x̂t−1
(16)
DF r =
1
C
1 +
C +
r xt , r xt ) − (r x̂t−1 , r x̂t−1
(17)
To validate detection at time t, the deviation factor must fall
below the Tolerable Pixel Shift (TPS), which in our case is
taken a value of 15. TPS refers to the permissible deviation
in pixel positions between consecutive frames within a video
sequence. The Kalman filter incorporates measurements when
the deviation factor remains within this threshold. A high deviation factor indicates increased detection error and measurement
noise, prompting the Kalman filter to rely more on the existing
motion model than the measurement. In such case, for the
parameters α0 , α1 , α2 , the corresponding diagonal values of R
should be large. r1 = r2 = r3 = 10025
2) Dynamic Area of Interest: It is a demarcated area with
boundary x ∈ [xmin : xmax ] and y ∈ [1 : length(IP M )]
around the detected lane marking in the nth frame which
will act as potential lane marking search area for the next
th
(n + 1) f rame where
xmin = x̂+
t−1 − wd
(18)
xmax = x̂+
t−1 + wd
(19)
where wd = 20 is pixel width used to define DAOI. Not only
this reduces the processing time by limiting the scanning image
area for the next frame, but also improves the accuracy of the
proposed method by avoiding the noise in the entire scene.
Adaptive thresholding method with subsequent application of
hough lines and curve fitting as explained above is then applied
to DAOI and lane features are retrieved along with quadratic
parameters.
3) Integration of Detection, Tracking and DAOI: However,
DAOI relies on inter-frame consistency that says lane does-not
change substantially between two frames, proving inadequate to
handle scenarios where new lane markings emerge abruptly in
the road scene,such as parking lane marks appears for some
frame and then discontinues in Aly’s dataset, cordova1 and
cordova2 [25]. To overcome this limitation, entire image is
scanned for both right and left half section for every 10th frame.
Within span of 10 frames, detection result at time t is compared
with estimated result from t − 1 to calculate deviation factor.
Two distinct cases arises in such situation: 1) DF < T P C:
detection over entire image is compatible with previous frame
results and DAOI is established for the next t + 1 frame (2)
DF > T P C: detection is incompatible with results from DAOI,
kalman filter disregards the detection results and uses prediction
for the current frame estimation. Full-frame detection mode is
applied at intervals of every 10th frame in the video sequence.
Flag is raised to makes the system go detection in full-frame
detection mode for the next frame. If the detection 11th frame
is incompatible with results from the previous DAOI, a flag is
raised, prompting the system to switch to full-frame detection
mode for the next frame. Flag is raised till the detection result
from full-frame becomes stable and DAOI is re-established with
re-initialization of kalman filter.
To overcome this limitation,similarity span of n famres are
slected , over which DAOI
C. Lane Change
In the real life scenario, lane change is frequent driving
maneuver.To address such scenarios, For change in the lane to
left side of the ego-lane:
1 +
C +
1
C
l x̂t−1 , l x̂t−1 → r xt , r xt
(20)
For change in the lane to left side of the ego-lane:
1 +
C +
1
C
r x̂t−1 , r x̂t−1 → l xt , l xt
(21)
XXXXXXXXXXXXXXXXXXX
Once the lane change is confirmed, kalman filter is reinitialized
for both left and right side of image. For the next frame, new
demarcation, x′m is defined to partition the image into left and
right halves.If lane changes to rightwards,new demarcation is
defined as** It divides the images more from right side
x′m = 1l x̂+
t + lw
(22)
It shifts the midline to the right of centre such that both the
left and right lane marking detections are positioned on their
respective sides relative to the new demarcation . Similarly, if
lane changes to leftwards,new demarcation is defined as
x′m = 1r x̂+
t − lw
(23)
where lw denotes pixel space between left and right lane
marking .With empirical estimation, space pf minimum 30
pixels are chosen to define lane-width. Subsequently, a lane
change flag is triggered, and x′m is utilized until both the left and
right lane marking detections are positioned on their respective
sides relative to the previous midline demarcation, xm .
JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
7
Fig. 7. Experimental set-up
forward movement of traffic and indicates vehicle’s position
with respect to the road. These marking exhibits various
colors and pattern. While solid yellow or white lane markings
are relatively easy to detect, identifying them in worn or
damaged condition present significant challenges. Moreover,
other markings can act as source of noise that must be
mitigated to ensure accurate detection of lane markings. Our
proposed method is evaluated on two datasets:1) Our dataset
2) Caltech dataset. Our dataset and annotation is available on ref
https://drive.google.com/drive/folders/11LcEWYz9HNufgEz1bHgvc0jtTm4
To capture the dataset, camera is mounted over the top of
the vehicle and stabilized with the help of gimbal and rigs, as
illustrated in Fig. 7. The recording occurs at frame rate of 30 fps
on Indian roads. The experiments are conducted on PC equipped
with 3.20 GHz Intel Core i5 CPU and 8 GB RAM. Images are
scale to size 640 ∗ 340 and bonnet area is removed before IPM
transformation.
To verify the detection results, our evaluation methodology
aligns with criteria proposed in Aly’s and Yoo’s work [34], [25],
wherein detected line within every frame is compared to the
hand labelled ground truth. To determine whether two are the
same,distance is computed between N samples on detected line
{l1 , l2 , l3 ...lN } and their corresponding nearest N sample on
′
ground truth {g1′ , g2′ , g3′ , . . . gN
}, as shown in Fig.8. Mean and
median are derived as:
L
L
L
d¯L = mean d1 L
(24)
G , d2 G , d3 G . . . .dN G
L
L
L
d˜L = median d1 L
(25)
G , d2 G , d3 G . . . .dN G
Fig. 5. (a) lane change
Fig. 6. (a)The orange line represents the mid-line, which divides the image into
left and right sides. The green portion depicts the ego-lane for the driving car.
The transition of the ego-lane from time stamp t − 1 to t occurs as follows:
1 b+
before the transition,1l x
b+
t−1 < xm and r x
t−1 > xm and indicating a leftward
position of the lane; once the lane shifts to the right side, 1l x
b+
t−1 > xm and
+
1x
b
>
x
m
r t−1
IV. D ISCUSSION
On paved roads, markings over road provide direction for
separating the traffic flow in the same or opposite direction
and assistance to pedestrian and drivers. Road marking comes
in various types such as longitudinal, transverse, hazard, block,
arrow, directional and facility marking. Our proposed work
focuses on detection of longitudinal markings,that ensures the
Similarly, d¯G and d˜G is derived by computing the distance between N samples on ground truth {g1 , g2 , g3 ...gN }
and their corresponding nearest samples on ground truth
′
}. For both detection and ground truth to con{l1′ , l2′ , l3′ , . . . lN
cide, the following conditions have to be satisfied,
min d¯L , d¯G ≤ t1
min d˜L , d˜G ≤ t2
(26)
(27)
1) Our Dataset: Our dataset consist of 6427 images,
recorded under varying illumination condition that includes
cloudy weather, dusk, saturated ambient light and shadows.
Clip1 was recorded on cloudy day with minimal traffic on the
highway. There is enough contrast of black-white-black pattern
on road surface and lane markings are clearly visible. However,
clip2 and clip3 consist of traffic scenarios where location of
JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
8
lane marking is not prevalent due to presence of pedestrian
vehicle. To gauge the robustness of our proposed method, we
conducted a comparative analysis on our dataset against three
existing methods i.e, Aly [25], Xu[21], and Yoo [34]. Fig. ??
shows chart that proposed model surpasses the performance of
existing models in terms of detection accuracy, estimated our
dataset using t1 = 10 and t2 = 15. For our work, detection rate
is calculated as follows:
R% =
Cr + Cl
2T
(28)
where T is total number of frames and Cl and Cr represent the
counts of accurately detected left and right lanes, respectively.
As shown in the first column of Table I, the work by Xu [21]
yields very low detection rate. The methodology in Xu’s work
involves employing 95% of the maximum pixel values as the
threshold. However, single threshold is insufficient to handle the
varying illumination levels across our different video clips. To
address this limitation, we modified the method by empirically
selecting the threshold tailored to individual clips. While it has
improved detection rate in clip1, clip2, clip3, clip4, clip8, but the
overall performance remains unsatisfactory, as evidenced by the
results in the second column of Table II for clip5, clip6, clip7,
and clip8. There are several issues associated with the work
proposed by Xu [21]. First, lane marks gets easily removed by
global thresholding, if saturation over one portion has dominance over lane marking features. Additionally, formation of
ROI is susceptible to edges originating from vehicles, stains, or
aligned noise. Such noises have the potential to form significant
peaks during vertical aggregation and eventually form the strong
connected components, thereby reducing the accuracy of the
work.
Aly’s method explored gradient properties by using oriented
gaussian kernels, which prove effective for detecting bright
lane markings against a dark background, particularly on new
roads,as evidenced by detection results of 96.66%, 95.11%, and
91.28% for clip1, clip6, and clip8 respectively. 2D kernels,
configured with specific width are used to produce filtered
image.These filters are expected to produce high response to
lane markings. Once the filtered image is obtained,pixel values
above 97.5% quantile value are retained. However, with varying
lighting condition and noise (other markings,IPM artifacts etc.),
their methodology are prone to produce high false positive.In
the absence of temporal integration, their methodology proved
inadequate in scenarios where lane markings were obscured
by leading vehicles, as evidenced by the detection results
of 71.25%, and 70.57% for in clip2 and clip3 respectively.
However, method is ineffective in handling noises with similar
orientation that is in proximity to lane markings.
Yoo’s method demonstrates superior performance when compared to other approaches. However, detection performance
suffers in scenarios characterized by intense shadows, evidenced
by a detection rate of 35% in clip6. To make the system
robust, inter-frame similarity is used for vanishing point and
lane angle estimation. However, the method does not effectively
handle rapid changes in road dynamics, particularly during lane
transitions, where angle constraints on host lanes may fail. This
limitation becomes apparent with a low detection rate of 35%
and 56% in clip6 and clip7, respectively, both of which involve
lane changes. Additionally, a lower detection rate, notably in
clip2, clip3, and clip5, is due to the presence of line segments
collected from vehicles near the lane marks with orientations
similar to those of the lane marks. This similarity disrupts the
geometric relationships between line segments and the estimated
vanishing point. However, method is prone to false detection
in the presence of street writing, crosswalks, stop lines on
cross streets. In such scenarios, filters design to detect vertical
bright lines of a particular width may erroneously identify these
symbols as lane lines.
As can be seen in Table I, our work is presented in two
modes:Mode1 entails the application of our method in a fullframe detection mode, while Mode2 integrates inter-frame similarity module with detection, where DAOI is established and
tracking is applied. The proposed method outperforms other
approaches with the average detection rate of 99.38%. In traffic
scenarios where lane markings are occluded by leading vehicles,
our method achieves a detection rate of 87.33% and 87.71% for
clip2 and clip3, respectively, without the inter-frame similarity
module. Similarly, low detection rate of 83% is seen for clip6,
that deals with intense shadow and lane change scenarios.
However, upon integrating the inter-frame similarity module,
significant improvement over detection rate by 12.47%, 12.15%,
15.22% is observed for clip2, clip3, and clip6, respectively.
Our method works well to maintain the definition of ego lane,
TABLE I
C OMPARISON OF L ANE D ETECTION R ATE ON OUR DATASET
Clips Xu’s [21] Modified Aly’s Yoo’s[34] Our Method : Our Method:
Xu’s [21] [25]
Mode1
Mode2
Clip1 47.68
87.84 96.66 96.75
97.1
99.67
Fig. 8. Validation criteria. The distance is calculated by determining the
Euclidean distance between N sampled points on the detection line (in red)
and their corresponding nearest points sampled on the ground truth (in green).
Clip2
54.15
75.71
71.25
89.08
87.33
99.80
Clip3
49.71
74.57
70.57
91.71
87.71
99.86
Clip4
3.61
81.12
89.74
99.01
98.86
99.28
Clip5
54.35
59.46
85.91
38.68
98.85
99.01
Clip6
55.24
59.54
95.11
35.78
83.11
98.33
Clip7
49.43
52.71
84.00
56.29
92.42
99.14
Clip8
11.29
58.71
91.28
98.00
91.28
99.85
JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
9
Fig. 9. Lane detection results on our dataset samples: Row 1 - Aly’s method [25], Row 2 - Yoo’s method [34], Row 3 - Xu’s method [21], Row 4 - our proposed
method.
while handling the scenarios like lane change**. As can be seen
in Fig. 10, graph is plotted for the trajectory of ego lane location,
where x-axis denotes the frame numbers and y-axis represents
the bottom-most x-coordinate of the detected left and right lane
marking lines. Notably, the ego lane undergoes a rightward shift
around the 40th frame, indicated by abrupt positional transitions
for both left and right lane markings. At 40th frame number,
position of left lane mark transits from point (a) to point (b),
followed by the right lane mark transitioning from point (b) to
point (c). Similarly, around 265th frame,the ego lane undergoes
a leftward shift,where position of right lane mark shifts from
point(d) to point (e), followed by the left lane mark transitioning
from point (e) to point (f). Moreover, the figure provide insights
into tracking outcomes for clip7, wherein detection with large
deviation factor and measurement errors are also highlighted.
For trajectories of other clips please refer VI-A
2) Caltech Dataset: Our proposed method is also applied
on the Caltech Dataset [25] with sample results shown in Fig
12. Caltech Dataset consist of four clips: cordova1, cordova2,
washington1 and washington2. Caltech dataset which include
Fig. 10. lane change to right side is seen in sudden transition of left lane mark
point from (a)to (b) and right lane mark point(b) to (c). Smilarly,during lane
change to left side left lane mark point transits of points(d)to (e) and point(e)
to (f)
Fig. 11. Blue indicates detection result while green represents ground truth
lane markings. Adjusting thresholds to t1 = 10 and t2 = 15 resolves the issue
of false positives observed with thresholds of t1 = 15 and t2 = 20
marking like deflection arrows, word messages, bifurcation
arrows, rectangular blocks, zebra crossing marks. Indian roads
generally speaking does not have have much yellow markings.
So, in order to capture yellow road marking,we empirically set
the threshold band on H channel to 20◦ as lower bound and
65◦ as upper bound. Unlike previous approaches by Yoo [34]
and Aly [25] that used t1 = 15 and t2 = 20, we chose t1 = 10
and t2 = 15 for the Caltech dataset. This is done since, upon
assuming t1 = 15 and t2 = 20, a false positive detection (in
blue) with respect to ground truth (in green) is observed in Fig
11. The selection of t1 = 10 and t2 = 15 effectively addresses
this issue. Comparison of lane detection rate on caltech dataset
is shown in Table II.
Our method achieves a detection rate of 98.50% in mode2
and 93.75% in mode1, surpassing the rates of 91.60% and
89.40% achieved by Yoo’s and Aly’s methods, respectively, for
cordova1. Despite the reliance of Yoo’s method on inter-frame
similarity and Aly’s method detecting lanes in each frame independently, both exhibit a detection rate of 88.42% and 75.37%
in cordova2. Introduction and discontinuation of parking lane
marks in certain frames (from frame f00264 to f00314) result in
abrupt shifts in reference lane markings position used to define
ego lane, posing challenges for inter-frame similarity to address.
Our method outperforms their performance with detection result
of 93.84% in mode2, where mode1 is applied every 10th
frame until stable detection results are obtained in full-frame
JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
TABLE II
C OMPARISON OF L ANE D ETECTION R ATE ON C ALTECH DATASET
Clips
Aly’s [25] Yoo’s [34] Our method Our method
mode1
mode2
Cordova1
91.60
89.40
93.75
98.50
Cordova2
75.37
88.42
82.22
93.84
Washington1
92.43
83.97
91.10
95.76
Washington2
92.46
95.25
97.62
97.83
10
degree with Indian Institute of Science, India. Her research interests include image processing,pattern recognition and computer
vision.
As can be seen in Fig XXXX, lane markings features are
difficult to extract because of strong saturation. Comparing the
result with Aly[25], Yoo[34] and Xu’s[21] work, it is clear that
our proposed method is outperforming their method. Experiment
results in shadow, dark.
8) extraction of Lane marks considers both 1) their gray
values (ii) location of lane marking 9)used only soft thresholding, carefully pruning lane marking features10) Defining the
supposed even in the presence of vehicle ahead advances the
scope the vehicle detection in the particular trajectory width
consistency is also not the same, road can change the shape
and change needs to be accepted,we cannot correct always on
the basis of the other detected lane marking,new lane marking
could be introduced and then DAOI fails, keep checking if new
line is introduced.
R EFERENCES
Fig. 12. Lane detection results on caltech dataset samples: Row 1,2 - Aly’s
method [25], Row 3,4- Yoo’s method [34], Row 5,6- our proposed method.
mode, followed by re-establishment of DAOI with kalman filter
re-initialization. However, absence of lane marking clues on
the right side contributes to incorrect detection. Moreover, by
effectively addressing lane changes, our method achieves a high
detection rate of 95.50% as compared to 92.43% and 83.97%
in Yoo’s and Aly’s work, respectively, for washington1.
V. CONCLUSION
Unlike the XXXXXXXXXXXXXXXXX method where is
required, our method consider various innate properties of lane
markings like statistical characteristic, geometrical shape and
inter-frame similarity etc
3)Assumption is that lanes does not substantially change
between consecutive two frames and the lane in t frame is
estimated by using detecting lane in t-1 frame.
It opens up the scope where any object in the ego lane
Saumya Srivastava (S’14) received the B.Tech degree in
Instrumentation and Controls engineering from Uttar Pradesh
Technical University,India, in 2014 and the M.Tech degree from
IIT-BHU in 2017. She is currently working toward the Ph.D.
[1] Narote, S.P., Bhujbal, P.N., Narote, A.S. and Dhane, D.M., 2018. A review
of recent advances in lane detection and departure warning system. Pattern
Recognition, 73, pp.216-234.
[2] Chin, K. Y., and Lin, S. F. Lane detection using color-based segmentation.
In IEEE intelligent vehicles symposium, pp. 706–711, 2005.
[3] J. P. Gonzalez and U. Ozguner, “Lane detection using histogram-based
segmentation and decision trees,” in Proc. IEEE Intell. Transp. Syst., Oct.
2000, pp. 346–351
[4] Sun, T.Y., Tsai, S.J. and Chan, V., 2006, September. HSI color model
based lane-marking detection. In IEEE intelligent transportation systems
conference, pp. 1168-1172, 2006.
[5] Cheng, H.Y., Jeng, B.S., Tseng, P.T. and Fan, K.C., 2006. Lane detection
with moving vehicles in the traffic scenes. IEEE Transactions on intelligent
transportation systems, 7(4), pp.571-582.
[6] Katramados, I., Crumpler, S., Breckon, T.: Real-time traversable surface
detection by colour space fusion and temporal analysis. In: Computer Vision
Systems, 265–274, 2009.
[7] Alvarez, J., Lopez, A., Baldrich, R.: Shadow resistant road segmentation
from a mobile monocular system. In: Iberian conference on Pattern
Recognition and Image Analysis, (2007) II: 9–16.
[8] J. Son, H. Yoo, S. Kim, and K. Sohn, “Real-time illumination invariant
lane detection for lane departure warning system,” Expert Syst. Appl., vol.
42, pp. 1816–1824, Oct, 2014.
[9] Otsu, N., 1979. A threshold selection method from gray-level histograms.
IEEE transactions on systems, man, and cybernetics, 9(1), pp.62-66.
[10] T.-T. Tran, C.-S. Bae, Y.-N. Kim, H.-M. Cho, and S.-B. Cho, “An adaptive
method for lane marking detection based on hsi color model,” in Proc. Int.
Conf. Intell. Comput. Springer, pp. 304–311, 2010.
[11] Canny, J., 1986. A computational approach to edge detection. IEEE
Transactions on pattern analysis and machine intelligence, (6), pp.679-698.
[12] J.-G. Wang, C.-J. Lin, and S.-M. Chen, “Applying fuzzy method to visionbased lane detection and departure warning system,” Expert Systems with
Applications, vol. 37, no. 1, pp. 113–126, 2010.
[13] Sobel, noise cleaning] Kang, D.J. and Jung, M.H., 2003. Road lane segmentation using dynamic programming for active safety vehicles. Pattern
Recognition Letters, 24(16), pp.3177-3185.
[14] S. Srivastava, M. Lumb, and R. Singal, “Improved Lane detection using
hybrid median filter and modified hough transform,” International Journal
of Advanced Research in Computer Science and Software Engineering, vol.
4, no. 1, pp. 30–37, 2014.
[15] Chen, G.H., Zhou, W., Wang, F.J., Xiao, B.J., Dai, S.F., 2013. Lane Detection Based on Improved Canny Detector and Least Square Fitting. AMR
765–767, 2383–2387. https://doi.org/10.4028/www.scientific.net/amr.765767.2383
[16] Wu, P.C., Chang, C.Y. and Lin, C.H., 2014. Lane-mark extraction for automobiles under complex conditions. Pattern Recognition, 47(8), pp.27562767.
[17] Yuan, C., Chen, H., Liu, J., Zhu, D. and Xu, Y., 2018. Robust lane
detection for complicated road environment based on normal map. ieee
Access, 6, pp.49679-49689.
JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
[18] Li, M., Li, Y. and Jiang, M., 2018. Lane detection based on connection
of various feature extraction methods. Advances in Multimedia, 2018.
[19] Jung, C.R. and Kelber, C.R., 2005. Lane following and lane departure
using a linear-parabolic model. Image and Vision Computing, 23(13),
pp.1192-1202.
[20] S. Kwon, D. Ding, J. Yoo, J. Jung, and S. Jin, “Multi-lane detection and
tracking using dual parabolic model,”Bull. Netw., Comput., Syst., Softw. vol.
4, no. 1, pp. 65–68, 2015
[21] S. Xu, P. Ye, S. Han, H. Sun, and Q. Jia, “Road lane modeling based on
RANSAC algorithm and hyperbolic model,” in Proc. 3rd Int. Conf. Syst.
Inform. (ICSAI), Nov. 2016, pp. 97–101
[22] J. Deng and Y. Han, “A real-time system of lane detection and tracking
based on optimized RANSAC B-spline fitting,” in Proc. Res. Adapt.
Convergent Syst., 2013, pp. 157–164.
[23] Wang, Y., Shen, D. and Teoh, E.K., 2000. Lane detection using spline
model. Pattern Recognition Letters, 21(8), pp.677-689.
[24] M. Fischler and R. Bolles, “Random sample consensus: a paradigm for
model fitting with applications to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, 1981.
[25] Aly, M., 2008, June. Real time detection of lane markers in urban streets.
In 2008 IEEE Intelligent Vehicles Symposium, (pp. 7-12). IEEE.
[26] Bertozzi, M. and Broggi, A., 1998. GOLD: A parallel real-time stereo
vision system for generic obstacle and lane detection. IEEE transactions
on image processing, 7(1), pp.62-81.
[27] Wang, Y., Teoh, E.K. and Shen, D., 2004. Lane detection and tracking
using B-Snake. Image and Vision computing, 22(4), pp.269-280.
[28] Borkar, A., Hayes, M. and Smith, M.T., 2009, November. Robust lane
detection and tracking with ransac and kalman filter. In 2009 16th IEEE
International Conference on Image Processing (ICIP), (pp. 3261-3264).
IEEE.
[29] Nieto, M., Salgado, L., Jaureguizar, F. and Arróspide, J., 2008, October.
Robust multiple lane road modeling based on perspective analysis. In 2008
15th IEEE International Conference on Image Processing, (pp. 2396-2399).
IEEE.
[30] Kang, S.N., Lee, S., Hur, J. and Seo, S.W., 2014, June. Multi-lane detection
based on accurate geometric lane estimation in highway scenarios. In 2014
IEEE
[31] Jiang, Y., Gao, F. and Xu, G., 2010, April. Computer vision-based
multiple-lane detection on straight road and in a curve. In 2010 International Conference on Image Analysis and Signal Processing, (pp. 114-117).
IEEE.
[32] Hur, J., Kang, S.N. and Seo, S.W., 2013, June. Multi-lane detection in
urban driving environments using conditional random fields. In 2013 IEEE
Intelligent Vehicles Symposium (IV), (pp. 1297-1302). IEEE.
[33] K. Zhaom, M. Meuter, C. Nunn, D. Muller, S. Muller-Schneiders, andJ.
Pauli, “A novel multi-lane detection and tracking system,” Intelligent
Vehicles Symposium (IV), 2012 IEEE, pp.1084-1089, 3-7 June 2012
[34] Yoo, J.H., Lee, S.W., Park, S.K. and Kim, D.H., 2017. A robust lane
detection method based on vanishing point estimation using the relevance
of line segments. IEEE Transactions on Intelligent Transportation Systems,
18(12), pp.3254-3266.
[35] Kalman, R.E., 1960. A new approach to linear filtering and prediction
problems.
[36] Wang, Y., Dahnoun, N. and Achim, A., 2012. A novel system for robust
lane detection and tracking, Signal Processing, 92(2), pp.319- 334.
[37] Wang, Y., Dahnoun, N. and Achim, A., 2012. A novel system for robust
lane detection and tracking. Signal Processing, 92(2), pp.319- 334.
[38] Borkar, A., Hayes, M. and Smith, M.T., 2011. A novel lane detection system with efficient ground truth generation. IEEE Transactions on Intelligent
Transportation Systems, 13(1), pp.365-374.
[39] Yoo, H., Yang, U. and Sohn, K., 2013. Gradient-enhancing conversion
for illumination-robust lane detection. IEEE Transactions on Intelligent
Transportation Systems, 14(3), pp.1083-1094.
[40] Lee, C. and Moon, J.H., 2018. Robust lane detection and tracking for realtime applications. IEEE Transactions on Intelligent Transportation Systems,
19(12), pp.4043-4048.
[41] Jung, S., Youn, J. and Sull, S., 2015. Efficient lane detection based
on spatiotemporal images. IEEE transactions on intelligent transportation
systems, 17(1), pp.289-295.
[42] Liang, M., Zhou, Z. and Song, Q., 2017. Improved lane departure response
distortion warning method based on Hough transformation and Kalman
filter. Informatica, 41(3).
[43] Gehrig, S.K., Gern, A., Heinrich, S. and Woltermann, B., 2002, September.
Lane recognition on poorly structured roads-the bots dot problem in
California. In Proceedings. The IEEE 5th International Conference on
Intelligent Transportation Systems (pp. 67-71). IEEE.
11
Fig. 13. chart shows the detection comparision done on various clips on our
dataset
Fig. 14. lane change to left side is seen for clip6 in our dataset.Sudden
transition of left lane mark point from (a)to (b) and right lane mark point(b)
to (c). Similarly,during lane change to left side left lane mark point transits of
points(d)to (e) and point(e) to (f)
VI. A PPENDIX
A. Mathematical Typography and Why It Matters
Fig 14 shows the trajectory of ego vehicle over 450 frames in
clip6 (our dataset). Lane change embarks around 326th frame,
with sudden transition of left lane mark point from (a)to (b)
and right lane mark point(b) to (c). Similarly,ego lane shifts
again to left side with right lane mark point transits of points(d)
to (e) and left lane mark from point(e) to (f).Fig 16 shows
the trajectory of ego vehicle over 350 frames in washington1
(caltech dataset).Ego lane shifts to left side around 100th frame,
with sudden transition of left lane mark point from (b)to (c) and
right lane mark point(a) to (b).
B. Mathematical Typography and Why It Matters
Data: IPM Image, mode t , xm , IFs pan, Iode0 = 1
for t = 1, 2, . . . , T ( T is the number of frames)
do 1l xt , [α0 α1 α2 ]r , 1r xt
= Detection (I , mode,
xm )
C +
DF = Deviation Factor 1l xt , cl xt , 1l x̂+
,
x̂
if
mod
t−1 l t−1
(t/IF− span) == 0 OR mode t == 1 if DF < T P S
JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
Fig. 15. lane change to left side is seen for washington1 in caltech
dataset.Sudden transition of left lane mark point from (a)to (b) and right lane
mark point(b) to (c). Similarly,during lane change to left side left lane mark
point transits of points(d)to (e) and point(e) to (f)
Fig. 16. lane change to left side is seen for washington1 in caltech
dataset.Sudden transition of left lane mark point from (a)to (b) and right lane
mark point(b) to (c). Similarly,during lane change to left side left lane mark
point transits of points(d)to (e) and point(e) to (f)
then mode t+1
= 2/∗ DAOI established 1for cnext
frame
+
*/ 1l x̂+
,
x̂
←
Update
kalman
filter
x
,
x
else-if
t l t
l t l t
1
c
1
C
x
,
x
−
x̃
,
x̃
<
threshold
/*
stable
detection
l t l t
l t−5:t l t−5:t
+
∗/ mode t+1 = 2 1l x̂+
← Re-initialize
kalman filter
t , l x̂t
+
1
c
1 +
x
,
x
else
mode
=
1
x̂
,
x̂
←
kalman
filter (large
t+1
l t l t
l t l t
1
c
meas error) 1l x̃t−5:t , C
x̃
←
Update
MAWE
l t−5:t
l xt , l xt
end if mod(t/IF− span) ̸= 0 OR mode t == 2 then if
DF < T P S then 1: mode
for next
t+1 = 2 /* DAOI established
C +
1
c
frame ∗/ 2: 1l x̂+
,
x̂
←
Update
kalman
filter
x
,
t
t l t
l
l xt 3:
+
result = 1l x̂+
else 1: mode
t , l x̂t
t+1 = 2/∗ DAOI established
+
for next frame ∗/ 2: 1l x̂+
,
x̂
← kalman filter (large meas
t l t
C +
error) end return 1l x̂+
,
x̂
t l t
XXXXXXXXXXXXXXXXXXXXXXXXXXXX
12
Algorithm 1 An algorithm with caption
Require: n ≥ 0
Ensure: y = xn
y←1
X←x
N ←n
while N ̸= 0 do
if N is even then
X ←X ×X
▷ This is a comment
N ← N2
else if N is odd then
y ←y×X
N ←N −1
end if
end while
JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020
Data: IPM Image, modet , xm , IFspan , mode0 = 1
for t = 1, 2, ..., T (T is the number of frames) do
1
1
l xt , [α0 α1 α2 ]r , r xt
= Detection (I, mode, xm )
+
+
DF = Deviation Factor(1l xt , cl xt , 1l x̂t−1 , C
l x̂t−1 )
if mod (t/IFspan ) == 0 OR modet == 1
if DF < TPS then
modet+1 = 2 /∗ DAOI established f or next f rame*/
+
+
1
c
(1l x̂t , C
l x̂t ) ← U pdate kalman f ilter (l xt , l xt )
else-if (1l xt , cl xt − 1l x
et−5:t , C
et−5:t ) < threshold
l x
stable detection
modet+1 = 2
C +
1
c
(1l x̂+
t , l x̂t ) ← Re-initialize kalman f ilter (l xt , l xt )
else modet+1 = 1
+
+
(1l x̂t , C
l x̂t ) ← kalmanf ilter (large meas error)
1
et−5:t , C
et−5:t ) ←
lx
l x
U pdate M AW E (1l xt , cl xt )
end
if mod(t/IFspan ) ̸= 0 OR modet == 2 then
if DF < TPS then
1: modet+1 = 2 / ∗ DAOI established f or next f rame ∗
/
C +
1
c
2: (1l x̂+
t , l x̂t ) ← U pdatekalmanf ilter(l xt , l xt )
+
+
3: result = (1l x̂t , C
l x̂t )
else
1: modet+1 = 2 / ∗ DAOI established f or next f rame ∗
/
C +
2:(1l x̂+
t , l x̂t ) ← kalmanf ilter( large meas error)
end
return
C +
(1l x̂+
t , l x̂t )
13