MNDOT
Traffic Monitoring and
Analysis Business
Processes
Contents subject to change
Office of Transportation Data and Analysis, Traffic Forecast and Analysis Section
11/28/2011
PURPOSE
This document was prepared as part of a User Requirements assessment (see Project 1237) that is
intended to support decisions to replace existing data systems and business methods with updated and
better coordinated software, application and business processes. It is also the basis for developing a
‘living document’ that will evolve as changes are made to the traffic monitoring program, even as
alternatives to replace systems and processes are considered. Selected staff of the Traffic Forecasting
and Analysis Section (part of the Office of Transportation Data and Analysis) has been involved in the
authoring and review of the contents. The document is organized into four basic sections:
1.0 Introduction and Context – This section contains a brief historical summary of developments over
the past thirty years that created the systems and processes that Minnesota Department of
Transportation (MNDOT) now uses.
2.0 General Traffic Monitoring Program Precepts – This section presents ‘Business Rules’ that continue
to guide decisions about change. The section also contains a Business Terminology glossary
(incomplete).
3.0 Business Process Diagrams and References to Supporting Documentation – This section contains
business process flow diagrams that illustrated sequential linkages between business partners,
processes, internal and external data systems, data sets, and critical decision points. Each diagram is or
will be supported by electronic file documents that refer directly to sections of the diagram. Copies of
these documents in their current state (dated 2/28/2011) are currently stored on the TDA Office
network drive N: under the folder: \TrafMon\UserNeeds with folders named according to what type of
data is documented (volume, classification or WIM, then folders for short duration, continuous or
annual process). The document names contain the diagram letters and numbers that correspond to the
first, ‘overview’ diagram. This section will eventually contain a schedule that documents when certain
process are performed for any given count cycle. However, such a schedule must be read with an
understanding that two to three count cycles overlap each other creating an approximate 30 month
work cycle timeframe.
4.0 Detailed Data System Documentation for Mature Data Systems – This section will be augmented
with database structure, object descriptions, variable definitions, and listings/purpose descriptions for
scripts as time allows and as the need arises.
1.0 INTRODUCTION AND CONTEXT
The logically related series of systems and processes that support MNDOT’s Traffic Monitoring Program
perform a vital role in traffic data collection, quality control, analysis and reporting for the department.
Over time, personal computer and server technology has been used to replace many of the paper, pen
and pencil methods, and at the time, innovative mainframe systems that were used in the past. Most of
the replacement or new systems were designed and programmed by MNDOT analysts using software
tools, programming skills and electronic ‘infrastructures’ that were available. Each system has one or
more of the following components:
1. Traffic sample scheduling and cycle administration
2. Location and attribute management (add, modify, retire, delete) using logically embedded or
manually implemented business rules
3. Data load utilities supporting various sources of external data
4.
5.
6.
7.
Raw data or aggregate data quality assurance and quality control algorithms and analyst tools
Graphical and non-graphical data representation
Reporting tools to assist the analyst and to provide information for internal and external users
Outside system and subsystem interface utilities based upon key field equivalencies (typically using
file exports) to facilitate data exchange, interoperability, and production of final map, report and GIS
products.
During the 1970s and 1980s systems to support continuous volume data and short duration vehicle
classification data relied upon mainframe programs that were either programmed separately in COBOL
or were closely tied to the department’s Transportation Information System, still being used today.
Considerable manual data entry was required for these systems. Processing of short duration volume
counts and mapping of count locations, Annual Average Daily Traffic (AADT) and Heavy Commercial
Average Daily Traffic (HCADT) was done by hand.
During the 1990’s greater use of personal computer and server based computer aided drafting (CAD)
software and files, database management systems and statistical analysis software led to less reliance
upon the mainframe and manual processing but increased the variety of skills and coordination required
within MNDOT to implement such change. These years saw a) the implementation of SAS programs to
manage the continuous volume data and create temporal adjustment factors, b) the use of automated
mapping using a server based database and coordinated CAD files, c) development and implementation
of an expert system to screen and impute continuous volume data, d) the use of detector data from the
Regional Traffic Management Center (RTMC), e) the use of relational database systems to manage short
duration volume and vehicle classification data, and f) a rapid expansion of the weigh-in-motion
program requiring the use of vendor provided analysis software, auto-calibration subroutines and
development of an ORACLE database to facilitate post data collection ‘data salvaging’.
The first decade of the century saw a) the development of new heavy commercial average daily traffic
(HCADT) estimation algorithms and data base and spreadsheet applications to support such estimation,
b) an increased automation of data flows for RTMC detector data and partner agency and district short
duration counts, c) the use of edit flags in continuous volume data to filter out vehicle classification data
prior to federal submission, d) an increase in the use of geographic information systems (GIS) given the
initiative to create a statewide base map, and e) the addition of software supplied by Dr. Kwon at the
University of Minnesota, Duluth (UMD) to process and display WIM data and an increasing amount of
continuous vehicle classification data. WIM data auto-calibration and data ‘salvaging’ was not being
done during this time. Even now, enhancements to existing systems are being implemented and other
useful spreadsheet and database applications are being created by MNDOT staff to support new
processing needs and analysis requests. Additionally, efforts are being made to better coordinate field
locations across the various programs so duplication of effort and travel can be avoided.
Omitted from this history are the databases and spreadsheets developed to assist in forecasting traffic
due to the necessity of focusing on the Traffic Monitoring Program.
All systems developed since 1995, except the WIM Oracle database, exist today. Traffic Monitoring
Program business rules have also changed over the years as computerization has increased the quantity
and quality of data, improved the administration of programs, led to more transparency, and provided a
springboard for additional data sharing and dissemination. It is expected that as this user needs analysis
proceeds, business rules and replacement data systems will evolve together. To that end the next
section outlines many of the current, general business rules that underlay the traffic monitoring
program so evolution may begin again.
2.0 GENERAL TRAFFIC MONITORING PROGRAM PRECEPTS
MNDOT’s Office of Transportation Data and Analysis (TDA) provides estimates of AADT on all public
roadways in the state each year following recommended practices. Additionally, estimates of vehicle
classification are made for all state highways and continuous vehicle/axle weights are collected. These
data support federal reporting, roadway design, safety analysis and other planning and budgeting
requirements. Operational data to support traffic signals or ramp meters are not directly managed by
TDA but provide a source of volume data once screened and aggregated.
Systematically addressing client needs in the context of limited resources has led to a structured method
for providing the ‘right’ amount of data at the ‘right’ times. For example, hourly count data is not
routinely collected or stored for short duration volume counts since there is very little call for it from
stakeholders. That does not mean, however, that such data collection wouldn’t be helpful for other
reasons.
Throughout the traffic monitoring program the concepts of count location (points), traffic segments
(two-way roadway segments) and sampling cycles provide the conceptual underpinnings for many of the
processes and systems contained in this document. It is assumed that there is variability in everything
measured and that variability can mask trends when considering longer time frames. Most processes
illustrated in this document attempt to accommodate or mitigate the effects of that variability. The
statements that follow represent current practice and policies that guide decisions about change within
the traffic monitoring program.
1. All roadways need AADT estimates every year (and Trunk Highways need HCADT every year)
and should represent ‘typical conditions’ as best as possible when estimated during a count
year.
2. AADT estimates consist of five different types. Each type has different statistical properties
and appropriate uses. Counts from each count program should supplement the count
record on a segment for other programs where possible and appropriate.
a) Default values based upon sampling and applied to ‘uncounted’ route systems,
b) Annually adjusted prior year estimates (for growth),
c) Calculated based upon adjacent or contributing segment AADTs,
d) Short Duration counts taken on 2, 4, 6 or 12 year cycles using a mix of counting
equipment during the temperate time of year (April – October). Recounts are taken
where necessary. Counts are immediately adjusted for day-of-week and month using
factors from prior years (by averaging up to three years’ factors), and axle correction
factors when applicable (TH only). When determining AADT, adjusted counts are
tempered with past estimates and adjacent counts. The official count cycle length may
change for count locations, but under limited and controlled circumstances.
e) Permanently installed sensors at continuous counting locations (RTMC, continuous
volume and classification at ATRs and WIMs).
3. AADT estimates apply to roadway segments on the ‘counted’ system using traffic segment
delineation criteria:
a) Sliding-scale longitudinal AADT change along a segment based on tolerances for ‘along
roadway’ variation in traffic volumes. Special counts taken simultaneously may provide
data supporting alternate segment definitions.
b) TH intersections with TH roadways create traffic segment breaks.
c) HPMS sample segments spur the need for a counted segment if not regularly sampled
d) Ramps require counts.
4. All traffic segments have ‘direct’ or ‘indirect’ traffic count locations determined by traffic
monitoring program requirements (continuous vs. short duration weekday sampling,
volume vs. classification, WIM needs) and field considerations (accessibility, safety, available
equipment, anchorage points, partner agency needs). When possible, the entire historical
record of when and where counts were taken, as stored in databases and geographic
representations, should be preserved.
5. HCADT and other vehicle classification estimates vary by AADT segment for the TH system
and are based upon :
a) Short-duration vehicle classification counts adjusted for day-of-week, month and AADT
estimate when counted. These consist of approximately 1200 ‘parent’ sites distributed
across the entire TH system and sampled on a six year cycle. Recounts are taken where
necessary.
b) Continuous vehicle classification data (additional ‘parent sites’ and source of data to
calculate vehicle classification daily and seasonal factors).
c) Relationship between short duration vehicle classification counts (parent sites) and
related contiguous segments (children segments). A child segment HCADT estimate is
usually based upon the actual parent site HCADT estimate, and an addition or
subtraction of an HCADT increment based on the difference between the parent and
child AADTs and one of two proportions of HCADT present in that difference (depending
upon an urban or rural distinction).
d) Annually adjusted prior year estimates (for growth).
6. We maintain that traffic data of all varieties are accurate unless recounts, specific
calibration data, serious departures from historical data, or field checks indicate that
additional modification or qualification of the data is necessary.
7. Comments from field staff and outside agencies can affect the ultimate determination of a
traffic estimate. Such comments are routinely solicited from County and Municipal State
Aid partner agencies and evaluated in the context of past counts, local land use and
roadway network changes.
8. Functional class is not used as a predictor for axle correction or temporal adjustment factor
application due to the wide variety of traffic patterns within any functional classification.
9. The traffic volume program attempts to collect data at 100% of the count locations for all
segments in a jurisdiction in the year(s) within a count cycle for roadways of the following
types: Trunk Highways, County State Aid, Municipal State Aid (and County Roads in the
Minneapolis/St. Paul metropolitan area).
10. Weigh-in-Motion data are collected by lane and used to establish ESAL factors for use in
pavement design related forecasts, and to alert enforcement officials. Calibration is
performed twice a year and average front axle weights and GVW distributions of Type 9
trucks are used to track calibration drift in the data.
11. Traffic data collection equipment and locations are treated as infrastructure when
incorporated into the department’s GIS. The management of such locations is independent
of route or other attributes since a given data collection point may exist on the same
roadway over time with varying route designations and characteristics.
12. All traffic data collection, processing and adjustment procedures require a full audit trail.
Methods are usually documented and transparent to data users on-line and upon demand.
13. Data about the performance of data collection devices is captured and analyzed since such
data permit a closer evaluation of the accuracy and reliability of the measurements and
subsequent estimates. Data review by an analyst or ‘intelligent’ data processing system will
initiate a series of field verification activities or bench tests to check for bias and other data
related performance issues.
14. Continuous volume data are edited or ‘imputed’ when bias is introduced by known
equipment malfunctions and when data are missing. Original data are always retained.
Such flagged edits are based upon hourly, directional, day-of-week and month-of-year
averages and ranges calculated for that site from the past year. The analyst may modify
estimates to accommodate week within month variability, events, holidays, and other
special considerations.
15. When necessary, traffic data are qualified using codes, comments and other means to guide
decisions about subsequent data uses. For example, data flags on imputed volume data
prevent the reporting of vehicle classification data from the same site and time period to
the federal government, yet the imputed volume data is usually included in the computation
of adjustment factors used to adjust short duration volume counts.
16. For all traffic monitoring program related data there should be only one source for official
annual estimates that are used for federal reporting and other internal/external uses.
Extreme efforts are continually being made to synchronize traffic data systems with the
most appropriate roadway representations. It follows that historical traffic estimates need
to be illustrated with historically accurate roadway representations.
17. Annual estimates of AADT, HCADT, other vehicle classification breakdowns, and K and D
factors are usually produced in a coordinated fashion as soon after February as possible
following the data collection year. Depending upon available staffing and workload this
data release has occurred between April and the beginning of June.
Glossary of Terminology – not complete
AADT - Annual Average Daily Traffic
AASHTO - American Association of State Highway and Transportation Officials
ATR - Automatic traffic recorder – Continuous Volume and/or Classification
DD - Directional distribution of traffic
DHV - Design hour volume
DOW - Day of week
ESALs - Equivalent single axle loads
FHWA -Federal Highway Administration
GIS - Geographic Information System GVW Gross vehicle weight
HCADT or HCAADT - Annual Average Daily Truck Traffic
HPMS - Highway Performance Monitoring System
K factor - (design-hour traffic as a percent of AADT – Federal reporting uses 30th highest hour)
MADT - Monthly Average Daily Traffic
MEPDG - Mechanistic-Empirical Pavement Design Guide
QA/QC - Quality assurance / quality control
SAS - Statistical Analysis Software (produced by the SAS Institute)
TDA - Transportation Data and Analysis office in MNDOT
TIS - Transportation Information System
TMG - Traffic Monitoring Guide produced by the FHWA
VMT - Vehicle-miles of travel
WIM - Weigh-in-motion
3.0 BUSINESS PROCESS DIAGRAMS AND DOCUMENTATION
“Existing Conditions” Traffic Monitoring Business Process Diagrams
The objective of this activity was to obtain and summarize a complete set of business process flow
diagrams describing the current Traffic Monitoring program from the MNDOT central office point of
view. Temporal aspects of the process flows are addressed in the diagrams as well. It should be noted
that some processes are recursive and involve feedback loops that require a more complete
understanding of time-related dependencies.
Each process within a group of related processes are illustrated with a unique color that represents the
predominant role the process plays according to the terms below. The MNDOT traffic monitoring
program has been diagrammed and described using the following terms:
1. Collection – (PINK) Prefaced with the letter ‘A’ - Polling, gathering and administering the collection of
continuous, short-duration and weigh-in-motion (WIM) data.
2. Processing, Analysis and Storage – (BLUE) Prefaced with the letter ‘B’ for daily/weekly or (GREEN) ‘C’
for monthly - These processes include processing data from counters, entering and evaluating data,
program status review, creating draft estimates of AADT and monthly reporting.
3. Year-end or Annual Processing – (YELLOW) Prefaced with the letter ‘D’ - Developing adjustment
factors - monthly, daily, hourly, design-hour volume( DHV), directional distribution, axle correction, and .
estimation of Annual Average Daily Traffic (AADT), Heavy Commercial Annual Average Traffic (HCADT),
Vehicle Weight characteristics and annual Vehicle Miles Traveled (VMT).
4. Reporting – How customer needs are met through electronic data transfers, hard copy or web tools
and reports.
5. Determine Traffic Monitoring Program Traffic Segment and Count Locations – Prefaced with the
letter ‘L’ – Continually assessing the need to determine where traffic segment breaks need to be
created, merged, or deleted, and where count locations need to added, modified, recoded or retired.
Count location attributes are maintained in multiple data systems and GIS layers creating a
synchronization problem between systems.
To be more meaningful for the Traffic Monitoring Program staff, selected names of files or process
execution files were included on process shapes. More complete representations of process mechanics
exist in the supporting documents found on the network drive N: as noted in the introductory ‘Purpose’
section but are incomplete at this time.
Shapes used in the diagrams
Drums represent mature data systems or database applications.
Rectangles represent an analyst initiated processes.
Diamonds represent Key Decision points.
Parallelograms represent interim data files or sources of ‘outside’ information.
‘Wave Bottomed’ Rectangles represent printed or electronic reports.
‘Slant Roofed’ Rectangles represent manual data entry processes.
Please see the Process and Data Flow Diagrams later in this document.
Double click on the Adobe Document reference – requires Adobe Reader.
Thoughts About the Future
Anticipating the future, the traffic monitoring staff has expressed interest in the additional areas
outlined below. These are not prioritized and do not necessarily include additional needs and desires of
external users. Project 1237 will help to identify how outside users and partners view the program and
how those views may affect future traffic monitoring program activities.
1. Unifying and better coordinating count location designation and attribute coding so all counting
programs reference a centralized or shared location data structure (preferably using a GIS
interface).
2. Incorporate more data screening algorithms and post polling calibration adjustments to vehicle
weight data from WIM installations.
3. Maintain or increase control over data system and application development to increase
flexibility when responding to changing customer requirements, program scheduling and
methodological improvements.
4. Increase the ability of the volume count program to accept hourly data.
5. Increase the use of the internet to collect data from partner agencies or outside data services.
6. Increase the use of GIS in data screening processes, analysis, reporting and mapping.
7. Increase the number of sources and types of data that can be accessed using the future system.
8. Improve the ability to create reports and disseminate data to outside interests from all aspects
of the traffic monitoring program.
Adobe (.pdf) Document containing Process and Data Flow diagrams,
pages 1-15. Double-click on diagram below to see all diagrams.
(Page 17 is simply a diagram assembled together from pages 1-15.)
4.0 ‘Mature’ Data Systems – (detailed data system documentation available if needed)
COUNTS – Relational Database Management System using client/server
R:BASE tables, objects, application and scripts
VCCOUNTS – Database using ACCESS, application, VB scripts, linked
EXCEL spreadsheets, macros
RTMC Data Warehouse – Custom Program (Kwon) using Microsoft
Visual Studio and .Net Framework 3.5 to work with archived sensor
data files
SAS Continuous Data System – SAS datasets and program scripts
Expert System for Continuous Volume Data Screening and Editing –
Nexpert Object for Windows (1995), and a structured file system
Parent/Child Spreadsheet/Database System – Applies segment specific
classification counts to related up/downstream traffic segments
Bull Piezo – Custom Program (Kwon ‘09) for continuous vehicle class
graphing, charting, reports, analysis for seasonal adjustment factors
(Kwon)
Bull Guide – Custom Program (Kwon ‘09) for visualizing and evaluating
WIM data for Load Spectra and for creating input files for MEPDG
software
Bull Reports – Custom Program (Kwon ‘09) for graphing and reporting
class and weight data, and evaluating WIM data for Load Spectra
Peak Converter – Custom Program (Kwon ‘09) for reading vendor
binary format files, converting them to spreadsheet friendly .csv
format, and providing a ‘standard’ directory structure for data storage
Spatial Data Warehouse – ESRI, ORACLE and managed enterprise
(MNDOT) resource
Kwon – Professor Taek Kwon, Transportation Research Laboratory, University of Minnesota, Duluth