Jakob Eriksson, Lewis Girod, Bret Hull, Ryan
Newton, Samuel Madden, Hari Balakrishnan
MIT Computer Science and Artificial Intelligence
Laboratory

Outline
 Introduction
 Architecture
 Data Acquisition
 Algorithm
 Performance
 Related Work
 Discussion

2







Hazardous to drivers and increasing repair costs due to vehicle damage
Determine “which” roads need to be fixed
Static sensors will not do well – requires mobility!
P 2 is first of its kind
Challenge : differentiate potholes from other road anomalies
( railroad crossings, expansion joints)
Challenge : coping with variations in detecting the same pothole.
(speed, sensor orientation)
P 2 successfully detects most potholes
(>90% accuracy on test data)


P 2 Architecture
Vehicles have GPS and 3-axis accelerometer

<time,location,speed,heading,3-axis acceleration>



Opportunistic WiFi/Cellular connections with dPipe to cope network outages
Taxi Testbed


7 Toyota Priuses 1
Soekris 4801 2 Embedded Linux



Wifi Card
Sprint EVDO Rev A 3 Network card
GPS
Some numerical facts




9730 total kms
2492 distinct kms
7 cabs
174 km with >10 repeated passes
1
2
3
1.
2.
3.
http://www.carbuyersnotebook.com/archives/Toyota_Pruis_2006.jpg
http://www.pkgbox.org/Soekris-4801.jpg
http://gizmodo.com/gadgets/peripherals/two-new-sprint-evdo-rev-a-cards-pantech-px500-and-sierra-wireless-aircard-595-200423.php

P 2 Architecture
GPS
Cab 1
Location
Interpolator
3 Axis
Accelero meter
Pothole
Detector
Pothole
Record
Clustering
Central Server
GPS
3 Axis
Accelero meter
Cab 2
Location
Interpolator
Pothole
Detector

P 2 Architecture
Distance Traveled vs. Total Hours
Across All Taxis
Lower line represents unique roads
Segments of roads that were repeatedly covered
258,021 unique road segments

DATA ACQUISITION
 Accelerometer placement

Dashboard


Windshield
Embedded Computer
 GPS Accuracy

Standard deviation 3.3m

DATA ACQUISITION
 Hand Labeled Data







Smooth Road
Crosswalks/Expansion
Joints
Railroad crossing
Potholes
Manholes
Hard Stop
Turn

DATA ACQUISITION
 Loosely Labeled
Training Data



We know only types of anomalies and their rough frequencies
Exact numbers and locations are unknown
Extends available training set

ALGORITHM
 Features of accelerometer data
 High energy events are potholes?

Not really!

Rail road crossings, expansion joints, door slamming are high energy events
 Accelerometer data is processed by embedded computer

256-sample windows

Pass through 5 different filters

ALGORITHM - Filtering
OUT
Pothole
Detections speed vs.
z ratio xz-ratio
IN
Windows of all event classes
Speed
 Input

Raw accelerometer data

256-sample windows
High-pass z-peak

ALGORITHM - Filtering
OUT
Pothole
Detections speed vs.
z ratio xz-ratio
IN
Windows of all event classes
Speed High-pass
 Speed


Car is not moving or moving slowly
Rejects door slam and curb ramp events z-peak

ALGORITHM - Filtering
OUT
Pothole
Detections speed vs.
z ratio xz-ratio
IN
Windows of all event classes
Speed High-pass
 High-Pass


Removes low-freq components in x and z axes
Filters out events like turning, veering, braking.
z-peak

ALGORITHM - Filtering
OUT
Pothole
Detections speed vs.
z ratio xz-ratio
IN
Windows of all event classes
Speed High-pass z-peak
 z-peak


Prime characteristic for significant anomalies
Rejects all windows with absolute z-acceleration < t z

ALGORITHM - Filtering
OUT
Pothole
Detections speed vs.
z ratio xz-ratio

IN
Windows of all event classes xz- ratio


Speed High-pass z-peak
Assumes potholes impact only side of the vehicle
Identifies anomalies that span width of the road (rail crossings, speed bumps)

Rejects all windows with x peak within Δw (=32) samples from z peak
< t x
X z peak
Or, ( X peak
/ z peak
)< t x

ALGORITHM - Filtering
OUT
Pothole
Detections speed vs.
z ratio xz-ratio
IN
Windows of all event classes
Speed High-pass z-peak
 speed vs. z ratio


At high speeds, small anomalies cause high peak accelerations
Rejects windows where Z peak
< t s or, (Z peak
/speed ) < t s
X speed

ALGORITHM – Sample Traces

ALGORITHM - Training
 Tuning parameters t ={t z
,t x
,t s
} are computed using exhaustive search over a set of values
 For each set t, we compute detector score s(t) = corr
– incorr 2
 Corr is no. of pothole detections when sample was labeled as “pothole”
 Maximize s(t)
 Include loosely labeled data s(t) = corr
– incorr 2 labeled
– max(0,incorr loose
– count r
)

ALGORITHM - Clustering
 Improve accuracy


Cluster of at least k events must happen in the same location with small margin of error(Δd)
Clustering algorithm

Place each detection in Δd X Δd grid.

Compute pairwise distances in same or neighboring grid cells



Iteratively merge pairs of distances in order of distance
Max intra cluster distance < Δt
Reported location is the centroid of the locations within it

 Well-known anomalies like bridges, railroad crossings, speed bumps etc can be located from GIS sources and blacklisted
 GPS errors
 Pothole avoidance
 Biased detection will focus on critical anomalies

PERFORMANCE EVALUATION




Goals

Minimize false negative rate for smooth roads
Never a flag a smooth road as anomaly
Missing a few potholes is acceptable
1.
2.
3.
4.
Evaluation
Classification accuracy on hand-labeled data
Performance improvement using loosely labeled data
Performance on loosely labeled roads
Spot-checks

PERFORMANCE EVALUATION
 Performance on Labeled Data
 Randomly divided into training set and test set
Class
Pothole
Manhole
Expansion joints
Railroad Crossing
Hand Labeled
88.9%
0.3%
2.7%
8.1% w/ Loosely Labeled
92.4%
0.0%
0.3%
7.3%


False positive rate is 7.6%
Not accurate

PERFORMANCE EVALUATION
 Estimating the false-positive rate
 Ran the detector on loosely labeled roads
Road # potholes # windows # detections rate
Storrow Dr.
few
Memorial Dr.
few
Hwy I-93
Binney St.
few some
Beacham St.
many
1865
1781
2877
6887
1643
3
2
5
25
231
0.16%
0.12%
0.17%
0.63%
14%

Helps set upper bound on false positive rate (at most 0.15%) on good roads.

PERFORMANCE EVALUATION
 Impact of features and thresholds
1. Only Z peak 2. w. xz-ratio filter t x
=1.5
3. w. speed vs. z ratio t x
=2.5
t s
=5

PERFORMANCE EVALUATION
 Performance under uncontrolled conditions

Slamming doors

Fiddling with the sensor equipment

Driving behaviors

Deliberately avoiding potholes
 Use clustering
 k=4

PERFORMANCE EVALUATION
 Spot Checks
Typical pothole Manhole Expansion joint

RELATED WORK

Surveys


Falling weight deflectometer
Machine vision – cameras, robots


Accelerometer
Microsoft Trafficsense – smartphones

DISCUSSION
 This is what I think




Innovative
Ground truth establishment is tedious, expensive in dense road networks
Will it work in hilly areas ,slopes?
Future work?


Driver feedback – Interactive embedded computers
Smartphones – Cheaper solution, greater coverage
 Comments/Questions ???

REFERENCES
 The Pothole Patrol: Using a Mobile Sensor Network for
Road Surface Monitoring Jakob Eriksson, Lewis Girod, Bret Hull, Ryan Newton,
Samuel Madden, Hari Balakrishnan MIT Computer Science and Artificial Intelligence
Laboratory
 U. Lee, E. Magistretti, B. Zhou, M. Gerla, P. Bellavista, and A. Corradi. MobEyes:
Smart Mobs for Urban Monitoring with a Vehicular Sensor Network. IEEE Wireless
Communications, 2006.
 TrafficSense: Rich Monitoring of Road and Traffic Conditions using Mobile
Smartphones Prashanth Mohan, Venkata N. Padmanabhan, and Ramachandran
Ramjee {prmohan,padmanab,ramjee}@microsoft.com Microsoft Research India,
Bangalore
 http://research.microsoft.com/apps/pubs/default.aspx?id=70573