New England UTC Year 23 - Research Project Description
UTC Project Number: MITR23-5
Project Title: Individual Differences in Peripheral Physiology and Implications for the
Real-Time Assessment of Driver State (Phase 2)
Principal Investigator: Bryan Reimer, (617)452-2177, reimer@mit.edu
Performing organization: MIT
Funding Agency: USDOT/RITA
Grant number: DTRT07-G-0001
Source Organization: New England University Transportation Center
Status: Ongoing
Total Funding: $55,000 Federal + $55,000 non-federal match = $110,000
Start date: 9/1/10
End date: 8/31/12
Abstract:
Driving in the state of distraction due to interactions with visual manipulative devices as
well as more complex cognitive demands is increasing at alarming rates. Recent
observations suggest that the current legislative effort aimed at text messaging is not
having a strong enough impact. Methods for the real time detection of a driver’s state
prior to overt changes in driving performance may play an important role in future
distraction and workload mitigation systems. In this work we propose to focus on the
development of algorithms based in part upon the use of physiological measures and
visual attention to classify periods were a driver is operating under high levels of
cognitive workload. This work should provide advances in recursive and learning
algorithms that can quantify driver state as well as more advanced machine learning
approaches. We aim to devote some effort into studying the influence of window overlap
and window width on the detection of short and long duration demands. In summary,
this project will develop new insights into the use of physiological indices and visual
attention as components of an algorithm to detect driver cognitive distraction. Results
are expected to provide significant insight regarding the feasibility of implementing realtime driver state detection systems.
1
Objective: In-vehicle communication, navigation and comfort systems as well as
nomadic devices are impacting driver attention and safety at alarming rates. To curtail
text messaging, regulations barring text entry and forms of electronic media interaction
have been adopted in over 20 states. However, according to the Highway Loss Data
Institute (2010) there has been limited impact on crash rates. The study authors
postulate that this finding is largely due to drivers ignoring the legislation. It is becoming
clearer that hands-free phone communication and new speech-driven automotive
interfaces may also place high demands on the driver. NHTSA voluntary guidelines for
the design of systems to reduce distraction are not due until 2014. As the development
of new regulations and voluntary guidelines are slow to develop, the consumer
electronics industry as well as independent application developers are developing new
technology at ever faster speeds. The market for devices and applications for social
connectivity will likely continue to grow at unprecedented rates, further driving new
innovation and demand. Consumers pressured by the continued desire for social and
emotional connectivity will continue to bring nomadic technologies developed for the
home or office with them as they drive or will embrace vehicle manufactures
technologies to enhance connectivity in new and unexpected ways. These trends
clearly suggest that connectivity and infotainment devices will continue to be a major
disruptive technology impacting the safety of our transportation system. Coupled with
this issue, shifting demographics will continue to reduce the overall capacity of the
driving population.
It is clear by these trends and observations that traditional models of regulation
may not be adequate to guard safety. Efforts to develop systems that continually
monitor the driver for subtle changes in workload and advise or automate corrective
actions need to be accelerated. Shifting demographics will further enhance this need as
older drivers, often under the influence of multiple medications, attempt to cope with
changes in capacity for driving as well as the additional demands of secondary
interactions such as cellular phone calls that they have become familiar with engaging
in while they drive. Research has shown that physiological and visual attention
measures show evidence of increased workload prior to the appearance of overt
changes in driver performance. Heart rate and skin conductance have been shown to
increase with escalating cognitive demand while gaze dispersion appears to decrease
(Mehler, Reimer and Coughlin, 2010; Reimer, Mehler, Wang and Coughlin, 2010;
Reimer, 2009; Veltman and Gaillard, 1998; Harbluk, Noy, Trbovich, and Eizenman,
2007; Sodhi, Reimer and Llamazares, 2002; Victor, Harbluk, and Engström, 2005;
Backs and Seljos, 1994; Sazabo, Peronnet, Gauvin, & Furedy, 1994;). The detection of
changes in workload prior to observing changes in driving performance is key to the
successful implementation of cues and alerts that help refocus the driver without further
pushing them towards potentially unsafe driving actions (Coughlin, Reimer and Mehler,
2009).
In phase one of this research, we demonstrated that physiological indices and
visual attention measures are sensitive to gradated changes in driver workload. We also
showed that age per se did not impact the reactivity profile in a manner that would
preclude the use of physiology or visual attention as mode of detection. Consistent with
earlier observations, physiological measures and visual attention were more sensitive to
initial changes in demand then driving performance measures.
2
Preliminary work has considered the use of these measures for detecting high
levels of workload at the individual level. The results show that even when using this
basic, non-optimized method for detecting periods of increased workload, relatively high
rates of detection can be obtained. Successful detection of the highest workload
condition was in the order of 90%. In the proposed project, we will focus further on the
development of algorithms that assist in the real-time monitoring of distraction with an
emphasis on relatively simple recursive or learning algorithms as well as more complex
algorithms from machine learning. Emphasis will be placed upon the development of
appropriate measures for detection of driver state across various portions of the YerkesDodson Curve. For example, the optimal parameters for identifying periods where the
driver is functioning in a state of active distraction may differ from the parameters that
best categorize fatigue.
We propose to use physiology, visual attention and driving behavior as attributes
of a model that categorizes driver state during periods of high demand. As part of this
project, we aim to use established simulation and field data sets to validate and
compare the receiver operating characteristic (ROC) curves to identify more optimal
models and attribute sets. Specific emphasis will be placed on assessing the impact of
different window overlaps and window widths on detection. By selecting appropriate
windowing characteristics, models can be better adapted to detect short and long
duration demands. This effort should help categorize what reasonable bounds will be for
the length of a high demand period before it is detectable. Although our efforts will focus
on cognitive demands, it is expected that the concepts developed here will help
augment current efforts being placed on the detection of visual demand using eye and
head tracking approaches. In summary, efforts will focus on assessing questions such
as:
• Can physiological measures or visual attention indices be used to
effectively monitor the level of a driver’s cognitive workload and provide an
indication of a driver’s spare capacity?
• What type of recursive or learning algorithms can be developed to quantify
driver state?
• How do different measurement parameters map to detection across
multiple regions of the state curve?
• What type of machine learning algorithms could be applied to further
advance the detection capabilities?
• How does varying window overlap and width impact detection of short and
long duration demands?
Task Descriptions: As part of this project, we will focus on continuing the analysis of a
field driving study that included just over 150 participants. The participants were in
three age groups, 20’s, 40’s and 60’s, and roughly balanced by gender. Subjects were
asked to drive an instrumented vehicle from MIT into southern New Hampshire and
back. Data was recorded during their drive from MIT to the highway and an
approximately hour and a half drive on Interstate 93. During portions of the drive
participants were asked to engage in three different difficulty levels of an auditory nback task (Mehler et al., 2009). This task provided a structured level of workload with
3
little contextual association to use as a reference to changes in state across the two
other tasks and different driving conditions.
Data will be assessed to characterize individual patterns in driver behavior,
physiological arousal and visual attention allocation to the roadway. Efforts will then be
invested in developing more advanced algorithms for detecting state changes
associated with a high level of cognitive workload. In addition to simple modeling
approaches, signal classification methods typically seen in the machine learning and
data mining literature will be assessed. Results will highlight appropriate methods for
characterizing driver cognitive demand.
Technology Transfer Activities: This project will provide translational research from a
laboratory setting to associated academic publication and public outreach during New
England University Transportation Center Events and selected Department of
Transportation and other sponsored events. Presentation of results at the
Transportation Research Board’s Annual meeting and publication of results in one or
more peer reviewed journals is expected. This research will provide training
opportunities for graduate and undergraduate students involved with the project.
Professionals from the automotive industry, other academic researchers and students
who attend a human factors workshop at the Transportation Research Board’s Annual
meeting will gain insight on the results from this work and methodologies used in the
capture of physiological measures in an automotive environment. Results of this effort
have been shared with the human factors teams at NHTSA, FHWA and the VOLPE
center. Finally, a synthesis of results from this work is being passed to the general
public through interviews and other presentations of the data.
Potential Benefits of the Project: This research aims to advance the use of
physiology and visual attention as measures for classifying driver state. The outcome of
this effort should help industry and government respond to the growing need for realtime distraction management. The timing of this project is particularly relevant as
information gained from this effort can inform current NHTSA research related to driver
distraction.
4