From: AAAI Technical Report WS-93-04. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved.
Symbolic Traffic
Scene Analysis
Using Dynamic Belief
Networks
Tim Huang, Gary Ogasawara; and Stuart Russell
ComputerScience Division
University of California
Berkeley, CA94720
{tthuang,
garyo,russell
}@cs.berkeley,
edu
1
Introduction
An important task for progress in IVHS(Intelligent Vehicle HighwaySystems) is the development
of methods for real-time traffic scene analysis. All
three major applications of IVHS- ADIS(Advanced
Driver Information Systems), ATMS(Advanced Traffic Management Systems), and AVCS(Automated
Vehicle Control Systems) - could benefit from accurate, high-level descriptions of traffic situations. For
example, an ADIS and an ATMScould use information about traffic congestion and accidents to alert
drivers or to direct vehicles to alternate routes. An
ATMS
also could analyze local traffic at intersections
to identify those with higher risk of accidents. Finally, an AVCSwould need information about the
actions of neighboring vehicles and the condition of
traffic lanes ahead to control an automated car moving along a freeway [6].
Our work on traffic scene analysis is one of many
projects under the PATH(Partners for Advanced
Transit and Highways) program of the University
of California’s Institute for Transportation Studies.
The overall aim of PATHis to determine how automation and advanced technology can increase highway capacity and improve safety. To this end,
multi-disciplinary research groups at the University
of California are studying a wide range of issues
such as message exchange protocols between moving
platoons1 [2], critical safety concerns in a fully automated highway system [1], and highway surveillance
using machine vision [9].
Although simple sensor-based schemes can provide information about traffic densities and velocities, such information will be imprecise, since drivers
are constantly accelerating and decelerating, changing lanes, drifting, and turning. Moreover,traffic con-
trol decisions require information about lane blockages, stalled vehicles, accidents, etc. Webelieve that
the dynamic belief network, a new knowledge representation system, provides a flexible, theoretically
sound framework for traffic scene analysis because
it can easily model uncertainty in variables and because it can provide high-level, symbolic descriptions
by integrating low-level information from a variety
of sources over an extended history. It also provides
a natural framework for expressing knowledge about
typical traffic behavior, allowing more accurate analyses from a given sensor stream.
The rest of this extended abstract is organized as
follows: section 2 describes the operations that take
place from the low-level processing of a traffic scene
to its high-level description; section 3 explains what
dynamic belief networks are and some issues involved
in using them for scene analysis; section 4 describes
a small traffic situation and how a dynamic belief
network could represent it; and, section 5 concludes
with a short summaryof future work on this topic.
2
Traffic scene analysis
In general, traffic scene analysis proceeds from
low-level processing of road traffic images to highlevel descriptions of the traffic situation (which can
in turn be used to direct and disambiguate low-level
processing). The input to the low-level processor may
come from one or more cameras that may be fixed or
moving. For example, our work uses a model-based
low-level processor, described by Koller in [3] and [4],
that takes as input a sequence of monocular traffic
images recorded by a stationary video camera.
Koller’s system begins processing by employing
standard machine vision techniques to detect image
features (such as line segments) in each image, and
*Alsoaffiliated with Lockheed
AI Center, Palo Alto, CA
1 Platoonsare groupsof cars underautomaticcontrol with it focusses on those features that are moving over a
very short spacing.
sequence of camera images. After that, it looks for
Figure 1: The recorded image of a traffic scene at an Figure 2: The identifications of the vehicles as canonintersection,
ical vehicle types by the low-level processor.
groups of cohesively moving features that suggest a
moving object in the scene. The system employs a
three-dimensional parameterized vehicle model and a
motion model based on Kalmanfilters to help distinguish between actual moving objects and image noise,
as well as to identify specific vehicle types. For example, Figure 1 showsan image of a typical intersection,
and Figure 2 shows the outlines and trajectories of
vehicles identified by the low-level processor.
For each image, the low-level processor outputs
a list of movingobjects, together with each object’s
type, position, speed, direction, etc. The high-level
processor employs some mechanism, such as the dynamic belief network that we will use, to make inferences about this information. For example, it may
reason that a vehicle has broken downif it has slowed
to a halt and its hazard lights are blinking. Or, it may
reason that a vehicle is being driven by a drunk driver
if it shifts back and forth from one lane to another.
High-level processors also could be designed to
identify when:
¯
¯
¯
¯
¯
¯
¯ an object has obstructed part of the road
¯ a traffic slowdownis caused by rubber-neckers
or a slow-movingpolice car
3
Dynamic belief
networks
Belief networks are directed acyclic graphs where
nodes represent random variables and arcs represent
causal connections between nodes [8]. Associated
with each node is a probability table that provides
conditional probabilities of the node’s possible states
given each possible state of its parents. Given the observed values of any subset of the random variables,
the posterior probability distribution can be calculated for the unknownvariables.
Belief networks are powerful because they offer
a mathematically sound basis for making inferences
under uncertainty. The probability tables in a belief
network provide a natural way to represent uncertain
information, and the meaning of updated probabilities is well-defined. The presence or absence of arcs
a vehicle is entering or exiting the freeway
between variables expresses knowledgeof causal relaa vehicle gets a fiat tire
tionships amongthem. Furthermore, the conditional
independence relationships implied by the topology
a vehicle collides into one in front or to the side of the network allow exponentially fewer probabilities
of it
to be specified than the full joint probability distribution for all the variables in the network.
a vehicle’s speed is too fast or slow
Dynamic belief networks allow the addition and
a lane is blocked by an accident or stalled vehi- deletion of network nodes. Such networks are ideal
for changing domains where the emphasis is on reacle
soning over time, and have already been applied to
a person is running or walking across the road monitoring the location of moving factory robots [5]
74
and to aircraft inertial navigation [10]. In the standard framework, the entire network is copied for each
’time slice’, and nodes are connected to other nodes
in the same time slice and nodes in the previous slice.
The network structure
may change from one time
slice to the next, for example, when vehicles enter
and exit the scene being analyzed. Inferences about
future nodes can be made by adding time slices to the
network and making inferences based on the values
of the current network nodes.
Our work on applying dynamic belief networks
to a traffic simulation is progressing on a number of
fronts. Weare examining how well different network
designs reflect traffic situations, studying techniques
for efficiently updating dynamicbelief networks from
one time frame to the next, preferably without having
to copy the entire network, and studying methods for
organizing the background information necessary to
makeinferences about various traffic situations.
4
Example
by the sensor. This relationship is illustrated in Figure 3. The link from the ActualValue node to the
SensorValue node indicates the causal relationship
2between the values.
Since outside conditions can affect sensor accuracy and since sensors can fail, this also must be
modelled in the belief network. In Figure 4, the
SensorA¢curacy
node corresponds to the level of
confidence in the sensor’s accuracy. The Weather,
Terrain, and SensorFailure nodes correspond to
factors that mayaffect the sensor’s accuracy. The dynamic belief network maintains copies of every node
at each ’time slice,’ so all the nodes for time t are
replicated for time t+l. This allows the network to
reason over time. 3 For example, if the readings from
a sensor repeatedly contradict other sensor readings
over a period of time, the network will automatically
adjust its confidence in the accuracy of the sensor to
indicate that the sensor seems to have failed. If the
sensor accuracy node has possible values indicating
several different failure modes(each of which can be
modelled explicitly), then the network also will automatically diagnose the most likely failure mode.
network
The following example describes the structure of
a simple belief network that can be used to identify
blocked traffic lanes and near collisions. For each
time frame, somenodes refer to specific vehicles and
other nodes refer to the scene as a whole.
4.1
Modelling
sensors
Manynodes relevant to specific vehicles correspond to observational data that is typically obtained
at each time step and used to make and revise inferences. The values of these sensor nodes, which may
correspond to a simple speedometer reading or to a
value from a low-level vision system, are distinct from
the actual values that the sensors measure.
Figure 4: Portion of dynamic belief
models sensor accuracy.
~v~ I
%,,...,S
4.2
Single
vehicle
belief
network that
subnetwork
Figure 3: Portion of belief network showing nodes for
actual value (e.g., of velocity) and sensor value.
Figure 5 shows a belief network fragment, projected over one time step, for a single vehicle. The
Therefore, to model a sensor, we include two
nodes: one node corresponding to the real value that
the sensor is measuring (e.g., a vehicle’s velocity)
and one node corresponding to the value indicated
2Note that the direction of inference is usually to calculate
the posterior probability of the actual value given the sensor
value.
3Note the absence of an arc connecting the SensorValue
nodes at t and t÷l. This assumes no hysteresis in the sensor.
Hysteresis is modelled by adding an arc.
75
same type of subnetwork is created for each vehicle in a scene, and these subnetworks form parts of
the whole traffic scene belief network discussed below. For each vehicle, we assume that we can obtain sensor data from the low-level system about the
brake light (on, off), the turn signals (off, lett,
right), and the vehicle’s position and velocity. For
simplicity, we have omitted nodes to model sensor accuracy. Weassume that the roadway is divided into a
grid with X longitudinal segments and Y lanes (Figure 6), and we assign the vehicle’s position to one of
those segments. A more detailed network also would
represent the vehicle’s length and width, as well as
its position with respect to the center of its assigned
segment (or it would have a finer-grained grid representation).
Figure 6: The roadway is divided into a grid with X
horizontal segments and Y lanes.
hicle is stopped, i.e., StoppedVeh.t+l, and the node
indicating whether and where the vehicle is making a lane change, i.e.,
VehLaneChange.t+l. The
StoppedVeh nodes have as possible values the set S
of possible grid positions plus null if the vehicle is
not stopped, and the VehLaneChangenodes have as
possible values the set S × {left, right} (with the
convention that the bottom edge of each grid block
in S is the lane edge being crossed) plus null for
no lane change. The belief subnetwork determines
a probability distribution over the possible values for
StoppedVeh and VehLaneChange, and these variables
are used in the traffic scene belief network to make
inferences about general traffic conditions.
4.3
Traffic
scene
belief
network
Figure 7 showsa belief network for the whole traffic scene, with single vehicle subnetworks represented
by VehI nodes. For each grid position, we have the
following variables:
¯ LaneChangeIJ: This variable has four possible
values: no for no lane change, in for a lane
change into IJ segment, out for a lane change
out of IJ, and multi for multiple lane changes
into or out of IJ.
¯ StoppedSegIJ: This variable, either true or
false, is a "noisy-or" function of the relevant StoppedVeh nodes from the vehicle subnetworks.
Figure 5: Portion of dynamic belief network projected over one time slice for a single vehicle. The
nodes at time t, including sensor information such
as BrakeLightSensor.
t and PositionSensor.
t, are
used to make inferences regarding whether and where
the vehicle is stopped, i.e., StoppedVeh.t+l, and
whether and where the vehicle is making a lane
change, i.e., VehLaneChange.t+l.
There are two variables of interest to the traffic
scene: the node indicating whether and where the ve-
¯ ~earCollisionIJ: This variable, either true or
false, depends on the type of lane change at
IJ and whether or not IJ is a stopped road segment.
¯ BlockedLaneI: There are Y of these nodes with
value either true or false. A lane is considered
to be blocked if there is a series of stopped road
segments over a period of time.
Intelligent highway systems can make traffic management and control decisions based on information
76
Analysis
1989.
Methods" PATH Preprint,
UC Berkeley,
[2] A. Hsu, F. Eskafi, S. Sachs, and P. Varaiya. "The
Design of Platoon Maneuver Protocols for IVHS"
PATH Research Report,
UCB-ITS-PRR-91-6, UC
Berkeley, 1991.
[3] D. Koller, K. Daniilidis, and H.-H. Nagel. "ModelBased Object Tracking in Monocular Image Sequences of Road Traffic Scenes" International Journal of Computer Vision, June, 1992.
[4] D. Koller. "Moving Object Recognition and Classification Based on Recursive Shape Parameter Estimation" IEEE Conference on Computer Vision and
Pattern Recognition, 1993.
[5] A. Nicholson. "Qualitative Monitoring of a Robot
Vehicle using Dynamic Belief Networks" Uncertainty
in Artificial Intelligence, 1992.
[6] A. Niehaus and R. F. Stengel. "Rule-Based Guidance for Vehicle Highway Driving in the Presence of
Uncertainty." In Proceedings of the 1991 American
Control Conference, 1991.
[7] G. Ogasawara and S. Russell. "Planning Using Multiple Execution Architectures" Proceedings of the
Thirteenth International Joint Conference on Artificial Intelligence, to appear.
Figure 7: A dynamic belief
network schema for a
whole traffic scene, with single vehicle subnetworks
denoted as VehI.t nodes.
from the traffic
scene belief network. For example,
an ATMScould monitor the number of near collisions on road segments and identify high risk areas
that would benefit from extra safety measures. Or,
an AVCScould recognize blocked lanes and direct an
automated vehicle to a different lane.
5
Conclusions
Dynamic belief networks seem promising as a tool
for integrating
and analyzing low-level information
provided by traditional
systems, and for acting as a
substrate for real-time decision-making systems of the
kind described in [7]. Their use in traffic scene analysis has a number of applications
in monitoring and
control. Recent and projected improvements in sensing and interpretation
technology promise to allow
real intelligence
to be deployed in automated vehicles.
References
Ve[1] A. Hitchcock. "Specification of an Intelligent
hicle/Highway System for Investigation
of Hazard
77
[8] J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference Morgan Kaufmann Publishers, San Mateo, CA, 1988.
[9] B. Rao, A. Chatziioanou, D. Koller, C. MacCarley,
J. Malik, S. Russell, and J. Weber. "A Machine Vision Based Surveillance System for California Roads"
PATHRFP for 1992-1993.
[10] A. Runnals. Dynamic Belief Networks for Aircraft
Inertial Navigation University of Kent, Technical
Report prepared for the Royal Aircraft Research Establishment at Farnborough, 1992.