Performance Metrics for
Oceanic Air Traffic Management
Moving Metrics Conference
Pacific Grove, California
January 29, 2004
Oceanic Metrics Team
Agenda
4
Metrics Team
•
•
•
•
4
4
4
4
4
4
4
4
4
Michele Merkle, FAA AUA-600
Lynne Hamrick, MITRE/CAASD
Yueh-Shiou Wu, MITRE/CAASD
Tamara Karakis, CSSI
Introduction: Purpose & Background of Oceanic Air Traffic
Control (ATC)
Background of Oceanic Metrics
Air Carrier Meetings
Oceanic Metrics Overview
Metrics Based on Priorities
Sample Dashboard Charts
Current Challenges Related to Oceanic Metrics
Baseline Performance Results
Summary
2
Introduction
4
Purpose of Briefing
•
•
•
4
Provide an overview of oceanic performance metrics
Describe challenges related to measuring oceanic Air Traffic Control
(ATC) service qualities
Discuss initial results and trends
Background of Oceanic Air Traffic Control (ATC)
•
•
Non-radar procedural separation
Communications via
¾
¾
•
•
Controller-Pilot Data Link (CPDLC) for Future Air Navigation System 1/A
(FANS 1/A)-equipped aircraft
High Frequency (HF) Radio Operator for non-equipped aircraft
Oakland Oceanic Center (ZOA) controls 21.3 million square miles
New York Oceanic Center (ZNY) controls 3.3 million square miles
3
Oakland Oceanic Airspace
4
New York Oceanic Airspace
Gander Oceanic
Shanwick Oceanic
Moncton
Boston
New York
Non-WATRS
Washington
Bermuda
Santa Maria Oceanic
WATRS
New York Oceanic
Miami
San Juan
5
Background of Oceanic Metrics
4
1993 Government Performance and Results Act (GPRA)
•
4
Required federal agencies to measure their performance and
effectiveness
FAA moves towards Performance Based Organization (PBO) and
Air Traffic Organization (ATO) formed
•
•
Goal to develop a more efficient and businesslike air traffic system
AUA goal to continue improving oceanic service, while measuring the
effect of new automation/procedures on the service provided
6
Air Carrier Meetings
4
Onsite Air Carrier Meetings in 1999 and 2003
•
•
Dialogue with air carriers coordinated via Air Transport Association
(ATA) meeting
Onsite air carrier visits in 1999 and 2003
¾
•
Air Carrier Personnel
¾
4
Air Canada Corporation, American Airlines, Continental Airlines, Delta
Airlines, Federal Express Corporation, Northwest Airlines, United Airlines,
United Parcel Service Corporation, US Airways Corporation
Operational Analysts, Dispatchers, Meteorologists, Pilots, Instructors, ATC,
Operational Managers
Summary of Air Carriers Visits
•
Received an overview of air carrier operations
¾
•
•
•
operating environment, route structure, fleet mix
Consolidated and compared lists of priorities
Discussed data sources
Established baseline metrics
7
Oceanic Metrics Overview
4
Purpose of the “Dashboard”
•
•
•
•
•
4
Established data exchange process with Centers
•
•
4
Provides visual summary of performance of pertinent metrics for facility
and airspace regions within the Flight Information Region (FIR)
Tracks customer demand and level of service provided by the FAA
oceanic ATC
Establishes baseline to determine affect of automation and/or procedure
changes (e.g., Advanced Technology and Oceanic Procedures or ATOP)
Identify anomalies and areas that need more tracking
Provide monthly charts for monitoring trends in oceanic service qualities
Primary data source: Oceanic Display and Planning System (ODAPS)
Other data sources: Oceanic Data Link (ODL), Track Advisory (TA)
Generated programs to process and analyze data
•
•
•
Oceanic Data Repository (ODR)
Oceanic Analysis Tool Set (OATS)
Oceanic Metrics Generator (OMG)
8
Metrics Based on Priorities
4
Air Carrier Priorities and Oceanic Metrics
Air Carrier Priorities
Metrics
Data Availability and Sources
Operating Environment
4Flight
count
4Avionics equipment
4Fleet mix
Available
4Flight Plans from ODAPS
Communication
4Avionics
equipment
4Altitude requests
4Response times
Available
4 Flight Plans from ODAPS
4 HF messages from ODAPS
4 CPDLC messages from ODL
Safety
4Operational
errors
4Altitude requests due to WX
4Deviation requests
Available
4 AAT-200
4 HF messages from ODAPS
4 CPDLC messages from ODL
Efficiency - Flexibility
4Requests Granted
4Altitude
Available
4 Plans from ODAPS
4 HF messages from ODAPS
4 CPDLC messages from ODL
requests granted
4Response time
9
Metrics Based on Priorities (Continued)
4
Air Carrier Priorities and Oceanic Metrics
Air Carrier Priorities
Metrics
Data Needed
User Satisfaction Predictability
4Optimal vs. Actual
4Planned vs. Actual
4First
Available
4 Flight Plans from ODAPS
4 Position reports from ODAPS
4 Track Advisory reports
preference granted
4Entry altitude flown as filed
User Satisfaction Predictability
4Delay / On-time Performance
In Development with Aviation
System Performance Metrics
(ASPM) and Enhanced Traffic
Management System (ETMS)
4 Planned departure and arrival
times
4 Actual departure and arrival
times
User Satisfaction
4Fuel Consumption
In Development
4 Preferred route and altitude in
ICAO Flight Plan (FPL) vs.
Actual route and altitude flown
10
Sample Dashboard Charts
ZNY Oceanic Flights
50
100%
45
600
500
398 381
479 468
450 441
429 432
504
445
395 401
378 384
80%
40
35
80%
60%
30
60%
25
300
40%
20
40%
200
20%
15
10
20%
0%
0
100
5
Response Time in Minutes
Oct-03
4
30%
20%
0
10%
0%
Percentage
90%
20
80%
70%
16
60%
50%
12
40%
8
30%
20%
4
10%
0
0%
% Modified
Oct-03
Sep-03
Aug-03
% Cleared
Jul-03
Jun-03
May-03
Unable
Apr-03
Mar-03
Feb-03
Modified
Jan-03
Dec-02
Cleared
Nov-02
Oct-02
% Unable
100%
24
Sep-02
% Modified
Oct-03
Sep-03
Aug-03
Jul-03
Jun-03
May-03
Apr-03
Mar-03
Feb-03
Jan-03
Dec-02
Nov-02
% Cleared
Sep-03
8
50%
40%
Unable
Aug-03
12
70%
60%
Modified
Jul-03
16
90%
80%
Cleared
Jun-03
100%
Oct-02
In Use
1st Option
ZOA Altitude Requests due to Weather - HF Flights
20
Sep-02
ADS
Oct-03
Sep-03
Aug-03
Jul-03
Jun-03
May-03
Apr-03
Mar-03
Feb-03
Jan-03
Dec-02
Nov-02
Oct-02
Sep-02
Reduced Vert. Sep. Min.
May-03
Total
Entry Alt Higher
Apr-03
0%
Mar-03
20%
Feb-03
40%
Jan-03
60%
Dec-02
80%
Nov-02
Oct-02
Sep-02
Oct-03
Sep-03
Aug-03
Jul-03
Jun-03
May-03
Apr-03
Mar-03
Feb-03
Entry Alt Same
0%
ZOA Altitude Requests - HF Flights
ZOA Avionics Equipment
Required Nav. Perf.
Jan-03
Entry Alt Lower
100%
Datalink
Dec-02
Nov-02
Oct-03
Sep-03
Aug-03
Jul-03
Jun-03
May-03
Apr-03
Mar-03
Feb-03
Jan-03
Dec-02
Nov-02
Oct-02
Sep-02
Oct-02
Sep-02
0
Response Time in Minutes
Avg Number of Flights per Day
700
400
ZOA First Preference Granted
ZNY Entry Altitude Flown as Filed
100%
% Unable
11
Current Challenges Related to Oceanic Metrics
4
4
4
4
US oceanic performance metrics are affected by actions taken by
non-US oceanic ATC
Limited end-to-end data available
Variations in operations and priorities across different geographic
and domain sub-regions
Processing HF messages
12
Baseline Performance Results
4
Trends observed
•
•
•
•
•
4
Plausible reasons for the above trends
•
•
•
•
4
Response time for HF flights are longer than that for FANS 1/A flights
Percent of positive or negative response to request are basically the same regardless of the
aircraft communication capabilities (i.e., FANS 1/A or HF)
Daily traffic varies more than 30% (e.g., May and August); but variation of performance level
is small
Most flights (80%) received preferred entry altitude for New York airspace or first preference
for the Pacific Organized Track System (PACOTS)
Average response time to altitude change requests has decreased from 10-50 minutes in 1998
to 5-15 minutes for HF flights and 3-6 minutes for FANS 1/A flights in 2003
Introduction of data link not only allow FANS 1/A flights to communicate with ATC faster, it
also reduced congestion on the channel allowing HF flights to get better services
Oceanic Data Link enhanced controller productivity for all flights, not just FANS 1/A flights
Whether a positive response can be granted is dependent on traffic situation, not on
communication means
Implementation of Reduced Vertical Separation Mimima (RVSM) allowed more flights to fly
their preferred altitude profile
Sample slides follow
13
Baseline Performance Results (Continued)
ZOA Oceanic Flight Count
560
540
520
1998
1999
2000
2001
2002
2003
500
480
460
440
420
400
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
14
Baseline Performance Results (Continued)
ZOA - HF Flig hts : Altitude Chang e Re que s ts
Ave ra g e Re qs / Da y
215
218
206
198
220
225
239
222
225
221
228
60
50
40
30
20
10
0
100%
80%
60%
40%
Pe rce nt
Ave ra g e
Re sp o n se Time
(Minute s )
227
20%
0%
Ja n
Fe b
Ma r
Apr
Ma y
Jun
Jul
Aug
Sep
Oc t
No v
De c
Ye ar of 1998
Cle a red Time
% Clea re d
Modifie d Time
% Modified
Unable Time
% Una ble
15
Baseline Performance Results (Concluded)
ZOA - HF Flights: Altitude Requests
Average Reqs / Day
285
258
241
201
188
182
196
216
196
201
245
100%
80%
15
60%
10
40%
5
Percent
Average Response
Time (Minutes)
20
20%
0
0%
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Year of 2003
Cleared Time
% Cleared
Modified Time
% Modified
Unable Time
% Unable
ZOA - FANS 1/A Flights: Altitude Requests
Average Reqs / Day
130
159
154
139
119
135
166
190
186
175
178
100%
80%
15
60%
10
40%
5
Percent
Average Response Time
(Minutes)
20
20%
0
0%
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Year of 2003
Cleared Time
% Cleared
Modified Time
% Modified
Unable Time
% Unable
16
Summary
4
Oceanic Performance Metrics
•
Assesses the operating environment and quality of oceanic service
provided to the airspace users
¾
¾
•
Provides a foundation for making sound business decisions
¾
¾
¾
4
Summary of performance
Tracks customer demand and level of service provided
Baseline comparison
Anomalies
Trends
Oceanic Metrics are evolving and expanding to meet the challenges
of measuring a complex system and the performance of ATC service
in a meaningful way
•
•
•
Different data sources (e.g., ATOP is replacing ODAPS and ODL)
Additional facilities (e.g., Anchorage)
Changing priorities
17