Chris Schwarz
National Advanced Driving Simulator

• Mid-America Transportation Center
– 1 year project to survey literature and report on state of the art in autonomous vehicles
– Co-PI: Prof. Geb Thomas
– Undergraduate students
• Kory Nelson
• Michael McCrary
• Mathew Powell
• Nicholas Schlarmann
– http://matc.unl.edu/research/research_projects.php?researchID=405
– https://www.zotero.org/groups/autonomous_vehicles/items

• Safety
– 32,000 people killed each year, 93% due to driver error, billions in property damage
– Autonomous vision is ‘crashless’
• Mobility
– Safely increase traffic density (x2)-(x3)
– Greater access for elderly, disabled, etc.
• Sustainability
– Fuel savings due to platooning (20%), eliminating traffic jams, reducing trip times, reducing ownership, reducing parking spaces

Academic Government
Private Military
A G
P M

An early experiment on automatic highways was conducted by RCA and the state of Nebraska on a 400 foot strip of public highway just outside Lincoln
(“Electronic Highway of the Future - Science Digest (Apr, 1958)” 2013)
A G
P M

• RALPH, ALVINN, YARF
• In 1995, RALPH drove NAVLAB 5 over 3000 miles from Pittsburgh to Washington, DC.
– Steered autonomously 96% of the way from
Pittsburgh, PA to Washington DC
Pomerleau, 1995, RALPH: Rapidly Adapting Lateral Position Handler,
IEEE Symposium on Intelligent Vehicles, September, 1995
A G
P M

1994-1997
A demonstration of the automated highway system in San Diego (1997). University of
California PATH Program
A G
P M

1997-2005
• Prevent driver distraction
• Facilitate accelerated deployment of crash avoidance systems
– Normal conditions
• IVIS
– Degraded condition
• Visibility, drowsiness
– Imminent crash
• Rear end, lane depart, intersection, ESC
Rada r
Vision
Curve speed
Warning
(CSW)
Lane-change/Merge
(LCM)
Lateral Drift
Warning (LDW)
Forward Crash
Warning (FCW)
Multiple ADAS system. Image from IVBSS materials, courtesy of UMTRI
A G
P M

Grand Challenge:
2004 – no winner
2005 – Stanley (Stanford)
Urban Grand Challenge
2007 – Boss (CMU) A G
P M

2004-present
• DSRC (5.9 GHz)
– Allocated in 2004
VII -> IntelliDrive -> Connected Vehicles
Regulatory decision from NHTSA recently announced. V2V will eventually be required in new cars.
• Goals
– Safety
• Forward collision, intersection movement assist, lane change, blind spot, do not pass, control loss warning, emergency brake light warning
– Mobility
– Sustainability
• AERIS
A G
P M

2010
A G
P M

2012-present
• Licensing • Cybersecurity
• Testing
• Regulations
• Currently recommends states only allow testing
NHTSA Levels of Automation
Level
0 – No Automation
1 – Function-specific Automation
Example
Warning only --
Transition Time to Manual
(Heuristic)
ADAS < 1 second
2 – Combined Function Automation Super cruise < 1 minute
3 – Limited Self-Driving Automation Google car < 10 minutes
4 – Full Self-Driving Automation PRT --
A G
P M

Autonomous Car Sharing Light Cars: A Virtuous Cycle
Reduce mass
Smaller fuel supply
Downsize engine
Drivetrain brakes tires MIT’s Stackable City Car

Audi
BMW
Chrysler
Ford
GM
Honda
Kia
Jaguar
Lexus
Mercedes
Nissan
Saab
Toyota
Volkswagen
Volvo
Sensor
Laser
Radar
Radar
Radar
Laser
Radar
Radar
Laser
Radar
ACC
Year
2006
2009
2004
1999
2001
2001
2002
1998
2002
Pre-Crash
Sensor
Radar/Video
Year
2011
Radar
Radar
Radar
Radar
Radar/Video
Radar/Video
2009
2003
2002
2003
2011
2007
Sensor
Camera
Camera
LDWS
Year
2007
2007
Camera
Camera
Camera
Camera
Camera
Camera
Camera
2010
2008
2003
2010
2009
2001
2002
A 2011 review of commercial ADAS systems compares manufacturers, model year, and sensor type for three types of systems (Shaout, Colella, and Awad 2011)

Abb.
System
ESC Electronic Stability Control
FCW Forward Collision Warning
ACC Adaptive Cruise Control
LDW Lane Departure Warning
LKA Lane Keeping Assist
LCA Lane Change Assist
RCTA Rear Cross Traffic Alert
BSD Blind Spot Detection
EBA Emergency Brake Assist
AEBS Advanced Emergency Braking System
ESA Emergency Steer Assist
Abb.
System
DD Drowsiness Detection
AL Adaptive Lighting
PM Pedal Misapplication
TSR Traffic Sign Recognition
TJA Traffic Jam Assistant
CZA Construction Zone Assist
PA Parking Assistant
PP Parking Pilot
HC Highway Chauffeur
HP Highway Pilot

• Fully autonomous
• No operator, no controls
• Low speed
• May use a guideway
• Morgantown PRT entered operation in
1975 in West Virginia

• Morgantown, WV
• Masdar City (on hold)
• London Heathrow
Airport
• City Mobil 2
• Suncheon, South Korea
• Punjab, India
• Early criticisms of PRTs on guideways concern the scalability of the system
• But new concepts are leaving guideways behind, alleviating some of these concerns

High grade LIDAR Inconspicuous LIDAR GPS / IMU
RADAR
Cameras
DSRC
Digital Maps

• The world is messy with uneven edges, bad lighting, poorly marked roads, and unpredictable people
• Applications of probabilistic reasoning
– Histogram filters (lane line tracking)
– Particle filters, Kalman filters (object tracking)
– Bayesian Networks (decision making)
– Hidden Markov Models (state estimation)

• Udacity online courses

• Digital maps negate the need to dynamically map the environment
• Simultaneous Localization & Mapping (SLAM) used to create environments in unmapped areas
• Many modern path planning algorithms are based on A* algorithm
• Must find the proper correspondence between the digital map and other sensor inputs

Bob Donaldson / Post-Gazette

Path Planning
Decision Making
Digital Maps
All speeds
Parking Lots
Many more tests
Histogram Filters
Particle Filters
Data Fusion
More data (images & video)
More test cases
Logic
Sensor Failures
Kalman Filters
False Positives

Example:
Transfer of Control to a Platoon
Level
0 – No Automation
Example Transition Time to Manual
Warning only --
1 – Function-specific Automation ADAS < 1 second
2 – Combined Function Automation Super cruise < 1 minute
3 – Limited Self-Driving Automation Google car
4 – Full Self-Driving Automation PRT
< 10 minutes
--

• “Automated vehicles are probably legal in the
United States” – Bryant Walker Smith
• 1949 Geneva Convention on Road Traffic requires that the driver of a vehicle shall be at all times able to control it
• Who is liable: the driver or the manufacturer?
• California, Nevada, and Florida have paved the way with state laws for automated vehicles

Entry point
Telematics
MP3 malware
Infotainment apps
Bluetooth
OBD-II
Door Locks
Tire Pressure
Monitoring System
Key Fob
Weakness
The benefit of such systems is that the car can be remotely disabled if stolen, or unlocked if the keys are inside. The weakness is that a hacker could potentially do the same.
Just like software apps, MP3 files can also carry malware, especially if downloaded from unauthorized sites. These files can introduce the malware into a vehicles network if not walled off from safety-critical systems.
Car apps are like smartphone apps…they can carry viruses and malware. If the apps are not carefully screened, or if the car’s infotainment software is not securely walled off from other systems, then an attack can start with a simple app update.
The system that connects your smartphone to your car can be used as another entry point into the in-vehicle network.
This port provides direct access to the CAN bus, and potentially every system of the car. If the CAN bus traffic is not encrypted, it is an obvious entry point to control a vehicle.
Locks are interlinked with other vehicle data, such as speed and acceleration. If the network allows two-way communication, then a hacker could control the vehicle through the power locks.
Wireless TPMS systems could be hacked from adjacent vehicles, identify and track a vehicle through its unique sensor ID, and corrupt the sensor readings.
It’s possible to extend the range of the key fob by an additional 30’ so that it could unlock a car door before the owner is close enough to prevent an unwanted entry.

Network
LIN
CAN
Weakness
Vulnerable at a single point of attack. Can put LIN slaves to sleep or make network inoperable
Can jam the network with bogus high priority messages or disconnect controllers with bogus error messages
Can send bogus error messages and sleep commands to disconnect or deactivate FlexRay
MOST controllers
Vulnerable to jamming attacks
Bluetooth Wireless networks are generally much more vulnerable to attack than wired networks. Messages can be intercepted and modified, even introducing worms and viruses

• Electronic Data Recorders (Black Box)
• Identified network traffic
• De-identified data
– The myth of anonymity
• “Google’s self-driving car gathers almost 1 Gb per second” – Bill Gross, Idealab

• Proactive not reactive
• Privacy by default
• Privacy embedded into the design
• Full functionality (positive sum, not zero sum)
• End-to-end security (full lifecycle protection)
• Visibility and transparency
• Respect for user privacy

• Time to Collision (TTC)
– range / range rate
• Autonomous Intersection
Management
– U Texas at Austin
– Reservation system
Autonomous Intersection (Top down)
Autonomous Intersection (Driver's View)
Van der Horst, 1991

• Levels are not a roadmap
• Levels are not design guidelines
• Levels discourage potentially helpful ideas like adaptive automation strategies
The evolution of vehicle automation and associated challenges

5 – 30 years until autonomous vehicles hit the road