Development of a Framework to Measure and Model the
Performance of BRT Systems
Project LS1
David A. Hensher
Corinne Mulley
John Stanley
Institute of Transport and Logistics Studies
The University of Sydney Business School
The University of Sydney
December 2010
Current status of LS1 objectives (2010-2012)
Obj 1: Review and evaluate methodological approaches within which to quantify
the salient factors that define the dimensions of performance of BRT systems
Completed
Obj 2: Identify and review existing data sources to identify gaps in data needs, and
a data collection strategy to obtain all necessary data
Completed
Obj 3: to set out the selected method in sufficient detail so as to ensure that it is
capable of estimation so as to deliver an agreed set of performance metrics
Completed
Obj 4: to undertake preliminary model estimation of data that is readily available in
the first 12 months, essentially from secondary sources,
Not Commenced
Obj 5: to use the evidence to define criteria for selecting BRT systems that should
be included by invitation as members of a ‘BRT cities club’.
Not Commenced
Key Elements completed in the 2010 Work
Program
› Identifed the influencing dimensions within a framework broadly distinguising
internal and external factors.
- The external factors emphasised the institutional, cultural, geographic influences
including evidence on modal biases present in the population as a whole and key
stakeholders.
› Identified key externalities (noise, congestion, safety, emissions) and how they
should be sourced and measured and built into the performance assessment
framework.
› Reviewed the broad literature on performance measurement as context in which to
develop an analytical framework that will serve to define data needs and the
hierarachy of performace indicators.
- A hierachical structure is crucial so that we can adopt a bottom up approach in
data collection that provides consistency and accuracy, but will also deliver
higher level performance indicators as appropriate that can be tracked back to
very specific disaggregated indicators, which in combination
facilitatesunderstanding of sources of influence on the higher order indicators.
› Developed a template for data needs that are candidates for the the SMART data
base, what data currently exists (with some assessment of its reliability), what data
is missing.
Major Activity I
› Title: The Dimensionality of Performance Frameworks and Performance Measurement
for Bus Rapid Transit Systems
› Development of a framework within which performance metrics can be defined and
introduced in the context of meeting strategic, tactical and operational objectives in the public
transport sector.
› We discuss the important matters of definition of performance, data requirements, standards,
the hierarchy of integrated partial and global measures of performance and frameworks to
compare enterprises, and to explain why there are differences, and what actions might
contribute to closing the gap between relatively poor and better overall performance.
› The relationship between inputs, outputs and outcomes is central to the performance rubric,
as well as an understanding of the processes that underlie the mappings between these
three dimensions.
› An important aim of the research is to ensure that the data collection activity planned for a
global study of the performance of Bus Rapid Transit (BRT) systems is guided by an
integrated and comprehensive framework for performance management, measurement,
feedback, and improvement.
› Ref: Hensher, D.A., Mulley, C. and Stanley, J.K. The dimensionality of performance
frameworks, paper prepared for Thredbo 12, Durban, South Africa September 2011
Major Activity II
› Title: Development of a comprehensive survey instrument to collect primary and
secondary data from a sample of 100 BRT corridors
› Reviewed literature to establish initial suite of candidate data items.
› With Embarq and PUC, reviewed these candidate data items and refine based on key
categories of data and extent to which they can be obtained.
› Classified all data items as shallow and deep with shallow as a minimum being included in the
LS1 performance metrics but recognising the essential need for a number of deep indicators
such as those related to costs, revenue, patronage and service kilometres.
› Prepared the data base as a survey instrument (through OS activities on Embarq) ready for
pilotting in early 2011.
› Refs:
VREF 1.1_Shallow pilot questionnaire_29 Nov.xlsx
Survey instrument 20 Nov 2010.xls
Hensher, D.A. and Mulley, C. Minimum ITLS BRT Data Requirements October 22
2010
Major Activity III
› Title: The relationship between bus contract costs, user perceived service quality and
performance assessment
› In contract tendering or negotiation, there is a growing interest in identifying and integrating user
perceptions of service quality into the determination of contract costs; be they total cost or cost per
service vehicle kilometre.
› Despite progress in measuring service quality, there is an absence of any structured mechanism for
determining the relationship between costs and user perceptions of service quality after accounting
for other influences on costs, and in determining what elements of service quality are under the
control of the operator, and those that are influenced by factors outside the operator’s control.
Furthermore, when the relationship between costs and service quality elements under the control of
the operator is identified, a mechanism is needed to quantify the extent of cost inefficiency
associated with service quality performance that is below an agreed set of performance standards.
› This activity developed a framework within which a cost-service quality relationship is quantified,
and then implemented to identify benchmark targets for cost efficiency improvements required to
achieve a pre-defined service quality performance target.
› We use data from metropolitan and non-metropolitan bus operators in New South Wales to
demonstrate the way the method can be used in contract negotiation and ex-post monitoring of
performance leading up to contract renegotiation or tendering.
› New data collected and compiled in the CoE, (in the observatory and LS1) and the new SQI
surveys (also to be developed under LS1) will allow this methodology to be applied to BRT
corridors
› Ref: Hensher, D.A. The relationship between bus contract costs, user perceived service quality and
performance assessment, prepared for Thredbo 12, Durban, South Africa, September 2011.
2011 Work Program with funding already agreed
› We see the LS1 activity begun in 2010 continuing throughout 2011 plus other elements of the
objective set using the funding of SEK 489,913 ($US71,513 as of 16 Dec 2010) already
been agreed.
› Key activities in 2011 (as set out in LS1 Proposal 2010-2012):
- Appointment of a post-doc position using the funds as part contribution
- Continue working with Embarq and PUC in the piloting and refinement of a questionnaire to
collect the baseline data for studying the performance of 100 plus BRT systems (at a
corridor level)
- Undertake pilot data collection on a BRT corridor in Sydney, NSW
- Undertake pilot on Brisbane Busway System in February 2011
- Provide feedback on the pilot and contribute to refining the survey form
- Advise Embarq and PUC as data collection progresses, on issues that arise in relation to
data availability and quality
- Commence preliminary modelling using data as it comes in, on models to establish a set of
key performance indicators as a means of benchmarking all systems, and developing
procedures to meaningfully compare operations given differences in operating environment
and what is and is not under the control of the operator, the regulator and the market.
2011 Work Program requiring additional funding
› Design a Service Quality Index (SQI) in conjunction with PUC as a way of capturing the
dimensions of service quality, and identify BRT systems that are prospective candidates for
the SQI survey of passengers.
› The intellectual contribution of this task is an objective of LS1 but the survey of cities required
to paramatise the model is not funded as part of LS1, although the Lisbon meeting of the
CoE identified this as an important contribution.
› We suggest 10 systems would initially be appropriate in order to test the portability of
preference weights across systems. LS1 is actively seeking cities interested in participating
in contributing to this important element of research.
8
Summary
2010
1. Prepared the project proposal LS-1
2. Prepared a paper on performance frameworks to inform data collection and
methodolgy
3. Designed data needs for the econometric models of comparative performance
(we are working on it as I email you) - we want this out there soon so Embarq
knows what data we need and this can be compared to what they have already
suggested.
4. Prepared a paper on SQI methodology and its interaction with costs of service
delivery and performance standards.
Papers prepared as CoE contributions under LS1:
Hensher, D.A. and Mulley, C. The Dimensionality of Performance Frameworks, paper
prepared for Thredbo 12 and Working Paper LS1-1 for VREF CoE.
Hensher, D.A. The relationship between bus contract costs, user perceived service quality
and performance assessment, prepared for Thredbo 12, Durban, South Africa, September
2011.
2011
1. Appoint postdoc position in BRT
2. Continue LS1 program
3. Identify target cities to join SQI development