System Performance and
Convective Weather
Kenneth Lamon, Ph.D.
15 March 2006
©2006. All rights reserved. This is the copyright work of The MITRE Corporation and was produced for the U.S.
Government under Contract Number DTFA01-01-C-00001 and is subject to Federal Aviation Administration
Acquisition Management System Clause 3.5-13, Rights in Data-General, Alt. III and Alt. IV (Oct., 1996). No other
use other than that granted to the U.S. Government, or to those acting on behalf of the U.S. Government, under that
Clause is authorized without the express written permission of The MITRE Corporation. For further information,
please contact The MITRE Corporation, Contracts Office, 7515 Colshire Drive, McLean, VA 22102, (703) 983-6000.
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
June, July and August are the Peak of the
Convective Weather Season
Percent Convective Coverage (Weekly Average, Level 3)
1.2%
2002
2004
1.0%
2003
0.8%
0.6%
0.4%
2001
0.2%
0.0%
Jan
2
Feb
Mar
Apr
May
Jun
Jul
Source: NOAA, FSL; measures the average area of thunderstorms over the
US at 19Z, 21Z, and 23Z , weekly average
Aug
Sep
Oct
Nov
Dec
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Delays
Scheduled Arrivals
Delays
= 500versus
+ a*Weather*(Schedule
- 16,500)
Arrival and Departure Delays (>1 Hour)
6,000
5,000
4,000
2000
2003
2004
3,000
2,000
1,000
16,500
17,500
18,500
19,500
20,500
21,500
22,500
Scheduled Arrivals
3
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Minutes of Delay versus Scheduled
Minutes = 100,000 + b*Weather*(Schedule
- 16,500)
Arrivals
600,000
Minutes of Arrival Delay
500,000
400,000
2000
2003
2004
300,000
200,000
100,000
16,500
17,500
18,500
19,500
20,500
21,500
22,500
Scheduled Arrivals
4
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
An Airport-specific Convective Weather
Metric
• Take an airport’s maximum storm intensity each
hour
• Multiply by the number of scheduled arrivals that
hour
• Sum over all hours of the day
• Sum over 45 airports
• (Divide by total number of scheduled arrivals)
5
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Show Movies Here
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
ATL Delays Versus Weather Score
1,000
900
800
2
R = 0.51
One-Hour Delays at ATL
700
600
500
400
300
200
100
0
200
400
600
800
1000
1200
1400
1600
1800
Weather score
7
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
8
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Monthly Convective Weather Score by
Major Airport
12000
10000
8000
YYYYMM
200406
6000
200407
200408
4000
2000
9
PBI
PDX
PHL
PHX
PIT
RDU
SEA
SLC
STL
TEB
TPA
MSY
ORD
FLL
HOU
IAD
IAH
IND
JFK
LAS
LGA
MCI
MCO
MDW
MEM
MIA
MSP
CVG
DCA
DEN
DFW
DTW
EWR
CLE
CLT
ABQ
ATL
BNA
BOS
BWI
0
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Number of Hours Having
Thunderstorm Intensity>3
June 1 - August 31, 2005
June 1 - August 31, 2004
14
18
24
32
15
32
19 37
21
30
28
29
36
30
30
25
31
37
28
16 34
15 25
32
44
19 35
36
37
22
28
26
26
31
34
4131
29
19
28
38
35
28
34
46
59
41
24
61
42
70
97
52
58
91
58
64
75
66
104
72
77
97
107
BROWN indicates locations where the frequency of thunderstorms increased from 2004 to 2005
10
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Delay and Convective Weather Score:
June  August 2004
16,000
ATL
ORD
14,000
Delays (>1 Hour)
12,000
10,000
DFW
PHL
EWR
8,000
IAD
LGA
6,000
4,000
2,000
BOS
IAH
JFK
DEN
CLT
DTW
MSP
PHX
LAS
MDW PIT
DCABWI
SEA
CLE MEM
RDU
TEB SLCIND STL
BNA
PDX MCI HOU
MSY
ABQ PBI
CVG
MIA
MCO
FLL
TPA
-
11
5,000
10,000
Weather Score
15,000
20,000
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
R-Squared Improves (a little) if Airports
are Grouped
2
2
1
1
22
2
2
1
2
2
1
1
1
1
1
12
12
33
13
1
2
33
1
3
1
113
3
1
22
11
1
G1
ABQ
ATL
BNA
BOS
CLT
DCA
DFW
EWR
FLL
IAD
IAH
JFK
LAS
LGA
MEM
MSP
ORD
PHL
PHX
SLC
STL
TEB
G2
DEN
HOU
LAX
MCI
MCO
MIA
MSY
OAK
PBI
PDX
SAN
SEA
SFO
SJC
TPA
G3
BWI
CLE
CVG
DTW
IND
MDW
PIT
RDU
21
2
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Delays Versus Weather Score:
45 Airports
6,000
5,000
2
2004
2
2003
R = 0.62
R = 0.67
Delays (>1 Hour)
4,000
3,000
2,000
1,000
-
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
Weather*Schedule
13
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Regression Results 2003 and 2004
Delays = 507 + 1.8*Weather*(Schedule - 15,429)
+ 298*Year2004
R-squared = 0.65
Minutes = 106,000 + 174*Weather*(Schedule - 16,870)
+ 24,454*Year2004
R-squared = 0.67
14
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Airborne Holds Versus Weather Score
2003 and 2004
600
500
2
2003
2
2004
Airborne Holds (>15 Minutes)
R = 0.59
R = 0.53
400
300
200
100
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
Weather*Schedule
45 Airports
15
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Diversions Versus Weather Score
2003 and 2004
250
Diversions
200
150
2
R = 0.48
2
R = 0.50
2004
2003
100
50
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
Weather*Schedule
45 Airports
16
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Cancellations Versus Weather Score
2003 and 2004
800
2
R = 0.63
2003
700
Cancelled Departures
2004
600
2
R = 0.52
500
400
300
200
100
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
Weather*Schedule
45 Airports
17
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Delays Versus Weather Score
2003, 2004, and 2005
8000
2005
One-Hour Schedule Delays
7000
6000
2004
2003
5000
2003
2004
2005
4000
3000
2000
1000
0
0
2000
4000
6000
8000
10000
Weather*Schedule
45 Airports
18
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Airborne Holding Up in Summer 2005
600
Year
500
2
R = 0.46
2005
Airborne holds (>15 min)
2004
400
2
R = 0.51
300
There were 28 additional holds per
day in summer 2005 compared to 2004
200
100
0
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
Weather*Schedule
19
45 Airports
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Airborne Holding Up in Summer 2005
70
Number of Daily Airborne Holds (>15 min)
60
50
Holding was up sharply at ATL in summer 2005
Year
2003
2004
2005
40
30
20
10
*June-August
20
TPA
STL
SLC
SFO
SEA
SAN
PIT
PHX
PHL
ORD
MSP
MIA
MEM
MDW
MCO
LGA
LAX
LAS
JFK
IAH
IAD
FLL
EWR
DTW
DFW
DEN
DCA
CVG
CLT
CLE
BWI
BOS
ATL
0
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Schedule Delays Lasting More Than
One Hour Up 16% Compared to 2000
Arrivals and Departures More than One Hour Late Relative to Schedule
300,000
256,904
250,000
221,928
200,000
219,108
171,629
149,168
150,000
120,721
100,000
50,000
2000
21
Source: ASPM, 45 Airports
2001
2002
2003
2004
2005
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Delays Due to GDPs Up in 2004 and 2005
OPSNET Delays by Type, 45 Airports, June-August
90
80
thousands
70
60
GDP
50
GS
40
Departure
30
20
10
0
2000
2001
2002
2003
2004
2005
Source: OPSNET, includes delays attributed to en route centers
22
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Proportion of OPSNET Delays at the
8 Most Delayed Airports Has Grown
OPSNET Delays, 45 Airports, June-August
120
100
Up 23% compared to 2000
thousands
80
ATL, ORD, PHL, EWR, LGA,
IAD, IAH, JFK
37 other airports
60
40
Down 39% compared
to 2000
20
0
2000
2001
2002
2003
2004
2005
Includes delays attributed to ARTCCs. Delays up 7% overall compared to 2000
23
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Operations, on Average, Up 9% at the
8 Most Delayed Hubs, and Down 9% Elsewhere
Operations June-August (OPSNET)
4.0
3.5
Down 9% compared
to 2000
3.0
millions
2.5
ATL, ORD, PHL, EWR, LGA,
IAD, IAH, JFK
37 other airports
2.0
1.5
1.0
Up 9% compared to 2000
0.5
0.0
2000
24
2001
2002
2003
Operations down 5% overall at OPSNET 45 airports compared to 2000
2004
2005
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
ATL Simulated Minutes of Arrival Delay
Using Arrival Capacity from 8/7/05
60,000
Arrival Delay Minutes (Simulated)
50,000
Aug 8 - 12, 2005
Aug 7, 2005
Aug 9 - 13, 2004
40,000
Aug 8, 2004
30,000
Schedule Dates
2005
Aug 7, 2004
Schedule Dates
2004
Aug 11 - 15, 2003
Aug 10, 2003
20,000
Aug 9, 2003
10,000
Schedule Dates
2003
1450
1400
1350
1300
1250
1200
1150
-
Scheduled Arrivals
25
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Summary
• Introduced a convective weather metric defined
as the product of an airport’s maximum hourly
thunderstorm intensity and the number of flights
scheduled to land that hour
• Metric has several advantages
– Can be aggregated and disaggregated
– Small, computationally simple dataset
– Yields many insights about NAS performance
26
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Other Findings
• Factoring out differences in weather and the total
number of scheduled ops, hour-long delays
increased substantially from 2003 to 2004 and
from 2004 to 2005
– Increases caused by greater concentration of traffic at
already delayed airports
• Airborne holding and diversions were unchanged
from 2003 to 2004, however, holding increased
from 2004 to 2005
27
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
And Lastly
• Similar approaches for measuring system
performance based on delays, cancellations,
diversions, holding, etc. ought to yield results
similar to those presented here.
28
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Appendix
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Number of Hours Having Thunderstorm Intensity>3
(June  August 2005 Minus June  August 2004)
Difference
(Hours)
-18
-10
-4
+4
+10
+18
Source: National Convective Weather Detection
30
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
OPSNET Delays Up 7% Compared to 2000
OPSNET Delays by Type, 45 Airports, June-August
160
145,919
140
138,312
136,715
127,111
thousands
120
100
92,543
94,087
GS/GDP
80
Arrival
Departure
60
40
20
0
2000
2001
2002
2003
2004
2005
Source: OPSNET, includes delays attributed to en route centers
31
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Average Number of Seats per Departure
has Decreased
Average number of seats per scheduled departure
down in July 2005 compared to July 2000
160
Average Seats per Departure
140
-10%
-17%
-19%
120
-16%
-15%
-8%
100
Jul-00
-1%
Jul-05
80
60
40
20
0
ATL
EWR
IAD
IAH
LGA
ORD
PHL
Source: OAG
32
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Change in Operations by Major Airport
Summer 2005 vs Summer 2000
SEA
-23%
PDX
-19%
DTW
-5%
MSP
+2%
ORD
+9%
SLC
+23%
LAS
+29%
SFO
-17%
LAX
-15%
SAN
+14%
DEN
+11%
STL
-38%
DCA
-18%
CVG
+6%
MEM
-1%
PHX
-2%
DFW
-16%
Source: OPSNET, Overall change = -2%, 35 Airports
BWI
-3%
IAD
+16%
MDW
-2%
IAH
+19%
33
CLE
-22%
BOS
-15%
EWR
-1%
JFK
+1%
LGA
+8%
PHL
+12%
ATL
+5%
TPA
-4%
MCO
-3%
FLL
+14%
MIA
-26%
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Change in Operations by Major Airport
Summer 2005 vs Summer 2004
SEA
-5%
PDX
-4%
DTW
+1%
MSP
0%
ORD
-1%
SLC
+13%
LAS
+8%
SFO
+1%
LAX
-1%
SAN
+9%
CLE
-1%
STL
+4%
DFW
-10%
IAH
+12%
EWR
0%
DCA
+3%
CVG
-1%
MEM
+9%
PHX
+4%
BWI
0%
IAD
+10%
MDW
-14%
DEN
+2%
BOS
-1%
JFK
+8%
LGA
+1%
PHL
+15%
ATL
-2%
TPA
+12%
MCO
+9%
FLL
+7%
MIA
-4%
Source: OPSNET, Overall change = -2%, 35 Airports
34
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Delays from GDPs: June  August 2005
In summer 2005 ATL accounted for 33% of all delays from GDPs
The average duration of this type of delay was 74 minutes
14,319
16,000
14,000
12,000
492
349
295
279
179
102
94
76
71
67
55
51
48
35
29
17
BWI
LAS
IAH
DCA
MDW
MSP
TEB
FLL
DTW
CVG
CLT
SEA
PHX
HOU
LAX
DFW
856
1,438
2,000
1,754
4,000
2,584
6,000
3,761
5,332
8,000
4,243
7,081
10,000
JFK
SFO
IAD
BOS
PHL
LGA
EWR
ORD
ATL
-
Source: OPSNET
35
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
PHL Simulated Minutes of Arrival Delay
Using Arrival Capacity from 7/13/05
55,000
50,000
Jul 11 - 15, 2005
Arrival Delay Minutes (Simulated)
45,000
40,000
May 30, 2005
Jul 10, 2005
35,000
Jul 12 - 16, 2004
30,000
Schedule Dates
2005
Jul 11, 2004
25,000
Jul 14 - 18, 2003
20,000
Jul 12, 2004
15,000
Schedule Dates
2004
Jul 10, 2004
Schedule Dates
2003
10,000
5,000
730
680
630
580
530
480
Scheduled Arrivals
36
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
PHL: Actual Minutes of Arrival Delay,
June  August
Actual Delay
2005
45,000
Actual Delay
2004
Minutes of Arrival Delay (Actual)
40,000
35,000
30,000
Actual Delay
2003
25,000
2005
2004
2003
20,000
15,000
10,000
5,000
480
580
630
680
Scheduled Arrivals
Source: ASPM, Each point represents one day, one outlier day excluded from 2003 and 2004
37
530
730
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
OPSNET Delays for 22 Most Delayed
Airports: June  August
2000
2001
2004
2005
30
thousands
25
20
15
10
5
0
ORD
ATL
DFW
IAD
MIA
BOS
LAX
PHX
LAS
MSP
DTW
IAH
CVG
PHL
EWR
STL
LGA
SEA
SFO
JFK
MDW
TEB
30
thousands
25
20
15
10
5
0
Ordered by busiest airport, includes delays attributed to en route centers
38
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
ATL: Actual Minutes of Arrival Delay,
June  August
90,000
Actual Delay
2005
Minutes of Arrival Delay (Actual)
80,000
70,000
Actual Delay
2004
60,000
50,000
Actual Delay
2003
2005
2004
2003
40,000
30,000
20,000
10,000
1150
1200
1250
1300
Scheduled Arrivals
1350
1400
1450
Source: ASPM, Each point represents one day, two outlier days excluded from 2003 and 2004
39
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
To Improve Airport Model
• Adding a variable representing amount of systemwide delay raises r-squared to 0.66 for ATL
• For ORD two-variable model has r-squared of 0.71
40
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Three-Letter Airport Abbreviations
ABQ
ATL
BOS
- Nashville International
- Boston/Logan International
BWI
CLE
- Baltimore-Washington International
- Cleveland-Hopkins International
BNA
41
- Albuquerque International Sunport
- Atlanta International
MCO
- Orlando International
MDW
- Chicago Midway
MEM
- Memphis International
MIA
- Miami International
MSP
- Minneapolis St Paul International
CLT
Charlotte/Douglas Internationa
MSY
- New Orleans International
CVG
- Covington/Cincinnati International
OAK
- Metropolitan Oakland International
DCA
- Reagan National
ORD
- Chicago O'Hare International
DEN
- Denver International
PBI
- Palm Beach International
DFW
- Dallas-Ft Worth International
PDX
- Portland International
DTW
- Detroit Metro Wayne Co
PHL
- Philadelphia International
EWR
- Newark International
PHX
- Phoenix Sky Harbor International
FLL
-
PIT
- Pittsburgh International
HOU
- William P. Hobby Airport
RDU
- Raleigh-Durham International
IAD
- Washington Dulles International
SAN
IAH
- Houston/G Bush Intercontinental
SEA
- San Diego International
- Seattle Tacoma International
IND
- Indianapolis International
SFO
- San Francisco International
JFK
- John F Kennedy International
SJC
- San Jose International
LAS
- Las Vegas/Mc Carran International
SLC
- Salt Lake City International
LAX
- Los Angeles International
STL
- Lambert-St Louis International
LGA
- La Guardia
- Kansas City International
TEB
- Teterboro
TPA
- Tampa International
MCI
Fort Lauderdale/Hollywood International
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007
Glossary
ASPM
FSL
GDP
NAS
NOAA
OAG
OPSNET
US
Z
42
Aviation System Performance Metrics
Forecast Systems Laboratory
Ground Delay Program
National Airspace System
National Oceanic & Atmospheric Administration
Official Airline Guide
Operations Network
United States
Zulu Time
© 2006 The MITRE Corporation. All rights reserved.
F065-B06-007