Rutgers Intelligent Transportation Systems (RITS) Laboratory
Department of Civil & Environmental Engineering
Modeling Traveler Behavior via Day-to-Day Learning Dynamics: Impacts of Habitual Behavior
Ozlem Yanmaz-Tuzel, M.Sc. And Kaan Ozbay, Ph.D.
Paper No: 10-2607
ABSTRACT
LITERATURE REVIEW
This paper focuses on the development of learning-based behavioral
mechanisms for updating route and departure time choices when a major and
brand new facility is added to the existing transportation system by creating new
route choices that did not exist previously. To model this complex user behavior
based on empirical observations; this study applies the Bayesian-SLA framework
recently developed by the authors. In this approach, Bayesian-SLA framework
systematically accounts for commuters’ belief, perceptions and habitual tendencies
about the transportation system, and represents these dynamics as random
variables. The developed learning model is calibrated and validated using real
traffic and travel time data from New Jersey Turnpike (NJTPK) toll road to
investigate the impacts of Interchange 15X installation on the day-to-day departure
time and route choice behavior of NJTPK travelers. The estimation confirm strong
effect of habitual behavior on traveler choice, consistent with the preliminary traffic
volume analysis findings. The proposed Bayesian-SLA model can successfully
capture the significant learning dynamics, demonstrating the possibility of
developing learning models as a viable approach to represent travel behavior.
Rutgers, the State University of New Jersey
METHODOLOGY
• In the proposed framework, an agent-based learning system via Bayesian-SLA is
designed which can learn the best possible actions and model travelers’ day-to-day
travel choices in a non-stationary stochastic environment.
• Each traveler maintains a choice probability profile for the available alternatives,
and updates his/her probability profile based on previous travel choices, exhibiting
a tendency to search for satisfying choices rather than the best behavior; inertia
and bounded rationality.
•The estimated learning parameters reflect travelers’ perception about the system
and their response to the experienced traffic conditions.
INTRODUCTION
• Modeling and understanding the relationship between individuals’ travel
perception and learning process and the day-to-day traffic flows remain an
important challenge for transportation researchers.
• This paper focuses on the development of learning-based behavioral
mechanisms for updating learned route and departure time choices in the presence
of new route inclusions to the transportation system considering the impacts of
habitual behavior on travelers’ choice mechanisms.
• This study applies the Bayesian-SLA framework. In this approach, Bayesian-SLA
framework systematically accounts for commuters’ belief, perceptions and habitual
tendencies about the transportation system, and represents these dynamics as
random variables.
EMPIRICAL SETTING
• The developed Bayesian-SLA framework is implemented to investigate the
impacts of the addition of 15X Interchange on December 2005 on the day-to-day
departure time and destination choice behavior of NJTPK travelers.
• After nearly three years of construction, NJ Turnpike Authority (NJTA) opened the
$250 million Interchange 15X on the Eastern Spur (just south of Interchange 16E)
Posterior
Distribution
of
the
Learning
Parameters
on December 1, 2005. The new interchange serves the new Secaucus Junction rail
• In this paper, learning parameters (a,b) are estimated via Bayesian Inference
transfer station.
approach.
•Given the likelihood of the observations and the prior information regarding
parameters (a, b), joint posterior distribution of the learning parameters are
estimated.
• The developed learning model is calibrated and validated using real traffic and
travel time data from New Jersey Turnpike (NJTPK) toll road to investigate the
impacts of Interchange 15X construction on the day-to-day departure time and
route choice behavior of NJTPK travelers.
•Unlike maximum likelihood analysis, the aim of a Bayesian analysis is not to
provide so-called point estimates of the model parameters; the result of the
analysis is the posterior probability distribution itself. With this approach, it is
possible to introduce user heterogeneity into the estimation process, and to
investigate distribution of the learning parameters among different users.
•The proposed likelihood function of the observations is estimated via following
equation:
• The NJTA contributed an additional $84 million to
adjacent Allied Junction, which will have 3.5 million
commercial and residential development, as well as
spaces when the development is completed. Upon full
15X is expected to handle 40,000 vehicles per day.
develop the $450 million
square feet of combined
up to 2,600 new parking
development, Interchange
• The proposed Bayesian-SLA model is applied to the road sections from
Interchanges 11 (Garden State Parkway) and 14 (Jersey City – Holland Tunnel) to
Interchanges 15X (Secaucus Junction) and 16E (Lincoln Tunnel). Before opening
of Interchange 15X, the only alternative traveling to Lincoln Tunnel area was to
travel through Interchange 16E. However, since December 2005, Interchange 15X
became a viable alternative for these travelers.
Rutgers Intelligent Transportation Systems (RITS) Laboratory
Department of Civil & Environmental Engineering
Modeling Traveler Behavior via Day-to-Day Learning Dynamics: Impacts of Habitual Behavior
Paper No: 10-2607
•The prior distribution of the learning parameters (a, b) can be represented by
p(a,b). In this paper, Dirichlet (multivariate generalization of the beta distribution),
multivariate Normal and Uniform distributions are tested as joint prior distributions
of p(a,b):
Ozlem Yanmaz-Tuzel, M.Sc. And Kaan Ozbay, Ph.D.
Rutgers, the State University of New Jersey
ESTIMATION RESULTS
• The estimation process resulted in Beta distribution for the posterior distribution of
each parameter. Since beta distribution always lies within [0, 1], the constraints on
the learning parameters will be satisfied at all times.
•Mean values for the parameters (a, b) are (0.029, 0.0029), and standard
deviations are (0.011, 0.00093), respectively.
•The posterior distribution of the learning parameters given the observations,
p(a,b|D), can be calculated using Bayes’ theorem:
• Posterior distribution of the learning parameters is a very complex
multidimensional function which requires integrating. Thus, to obtain the mean and
variance of the parameters (a, b) Metropolis-Hastings (M-H) algorithm is used. The
M-H algorithm is a rejection sampling algorithm used to generate a sequence of
samples from a probability distribution that is difficult to sample from directly.
• To ensure MCMC convergence, Heidelberger and Welch first test diagnostic was
employed. This diagnostic compares the observed sequence of MCMC samples to a
hypothetical stationary distribution, using the Cramer-von-Mises statistic. The test
iteratively discards the first 10% of the chain until the null hypothesis is not rejected
(i.e. the chain is stationary), or until 50% of the original chain remains
CONCLUSIONS
• This research focuses on modeling learning based behavioral mechanisms for
updating route and departure time choices in light of new facility additions to the
existing transportation system. The proposed model extends the existing SLA
theory by using it in a Bayesian framework and bounded rationality (BR), while
considering the impacts of habitual behavior:
Calibration and Validation of the Learning Parameters
• Calibration and validation are important processes in the development and
application of day-to-day DTA models. These processes are developed to ensure
that the models accurately replicate the observed traffic condition and driver
behavior.
1. Day-to-day learning behavior is modeled based on Bayesian-SLA theory,
where each user updates his/her choice based on the rewards/punishments
received due to selected actions in previous days. A linear reward-penalty
reinforcement scheme is considered to represent day-to-day behavior of
NJTPK users as a response to construction of a new Interchange.
2. In order to account for travelers’ resistance to switch routes, concept of
habitual behavior (inertia) is included in the proposed model, such that the
travelers switch to the new route only if it has significantly less cost.
3. Finally, learning parameters were modeled as probability distributions rather
than deterministic values, and Bayesian posterior distributions are estimated.
•Model calibration is a process whereby the values of model parameters are
adjusted so as to match the simulated model outputs with observations from the
study site. It is usually formulated as an optimization problem to determine the best
set of model parameter values in order to minimize the discrepancies between the
observed and simulated values.
•The calibration process is then to modify the values of the model parameters θ, so
to find the best set of values which minimizes F. The proposed objective function F
minimizes the difference between observed and simulated volumes:
• After determining the optimal set of parameters from the calibration process,
a validation process is performed in order to determine whether the simulation
model replicates the real system. Mean standard errors (MSE) are calculated
for each day the validation process:
• Mean values of the reward and penalty parameters estimated for NJTPK users
are different than the ones in other disciplines.
• The empirical results obtained from the real transportation network confirm the
strong effect of habitual behavior on traveler choice.
• The low learning parameters indicate that even though the travel time from
Interchanges 11 and 14 to Interchange 15X is lower compared with the travel time
to Interchange 16E, high percentage of travelers put higher utility to Interchange
16E continue to use this choice.
•The proposed Bayesian-SLA model can successfully capture the significant
learning dynamics, demonstrating the possibility of developing a psychological
framework (i.e., learning models) as a viable approach to represent dynamic travel
behavior.
• These results confirm the strong effect of habitual behavior on traveler choice,
consistent with the preliminary analysis findings.
• A possible extension of the proposed methodology is to investigate how
individuals’ tolerance level, and learning parameters change over time as the
users gain more experience with the transportation system.