Demand Generation for
System-wide Simulation
Glenn Foster
MITRE CAASD
January 27, 2010
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Background
• MITRE CAASD performs system-wide
analyses to answer many questions
• Demand is a key input into system-wide
analyses
2
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
System-wide Modeling
Question
Which sectors will not be able to
accommodate expected demand?
How many scenario days are required and which days?
How to handle airspace changing over time?
3
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Sample Size and Day Selection
4
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Number of Scenario Days
Background
• Select a sample of days that can be used as input
into system-wide analysis
– Represent a range of selected metrics over the year
of interest
– Results from system-wide analysis can be used to
characterize the performance over the year of
interest
• Analysis first focused on determining number of
days that are needed to achieve coverage of
traffic at the sector level
– Motivated by use of the system-wide analysis for
identifying bottleneck en route sectors
5
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Coverage of Sector Traffic
• Analysis conducted on FY2006 sector traffic data
– Daily sector counts were sorted into quartiles
• Random samples of various sizes (number of days)
were chosen
– Each day in the sample can be mapped to one of four
quartiles for every sector
Sample Day 1
6
Sample Day 2
1
Sample Day 3
+… =
11
22
33
44
33
22
11
22
25
11
11
11
…
…
ZNY85
ZNY85
ZAB15
ZAB15
ZAB16
ZAB16
ZNY85
1
Quartile
Quartile
+
…
1
1
ZAB16
…
ZAB16
ZAB15
+
1 1
2
3
4
Quartile
1
ZAB15
ZNY85
…
ZAB16
1
1 1
2
3
4
ZNY85
1 1
2
3
4
Quartile
Quartile
ZAB15
– Coverage = The sample set contains at least one day
mapped to each of the quartiles for all sectors
33
22
12
11
Coverage
Coverage
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Evaluating Sample Sizes for Coverage
of Sector Traffic
100%
Percentage of runs with sector coverage
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Sample size
7
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Effect of Sample Size on
Estimates of Average Daily Operational Delay
Each box-whiskers plot represents 200 random samples of n days from FY2006 ASPM
Each sample is a different estimate of Average Daily Operational Delay
Box = interquartile range the estimates
Whiskers = Total range of the estimates
Red = Average of the estimates
Most of the improvement in the range of the estimates
occurs as samples sizes increase up to ~ 30
8
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Creating a Sample Set with FY2008 Data
- Extension of Approach to FY2008 Data
and Other Metrics
• Sample of size 36 chosen (same number of days per
month)
• Sector traffic counts only analyzed for FY2006 data
– However, conclusion expected to hold for other years
• Look at coverage for other metrics
– Airport delays from ASPM
– Number of non-VMC hours per day per airport from
ASPM
– Center counts from OPSNET
• Generate 5000 samples of 36 days
– 3 days from each month in FY2008, including the 8 FAA
sample days
• Select sample with the most even coverage across the
quartiles of the available metrics
9
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
CAASD’s FY2008 Selected Scenario
Days
7
14
21
28
October
1 2 3 4 5 6
8 9 10 11 12 13 4 5
15 16 17 18 19 20 11 12
22 23 24 25 26 27 18 19
29 30 31
25 26
6 7
13 14
20 21
27 28
January
1 2 3 4 5
8 9 10 11 12 3 4
15 16 17 18 19 10 11
22 23 24 25 26 17 18
29 30 31
24 25
6 7
13 14
20 21
27 28
April
1 2
8 9
15 16
22 23
29 30
6 7
13 14
20 21
27 28
10
3 4 5
10 11 12 4 5
17 18 19 11 12
24 25 26 18 19
25 26
November
1 2 3
6 7 8 9 10 2
13 14 15 16 17 9
20 21 22 23 24 16
27 28 29 30
23
30
February
1 2
5 6 7 8 9 2
12 13 14 15 16 9
19 20 21 22 23 16
26 27 28 29
23
30
May
1 2 3 1
6 7 8 9 10 8
13 14 15 16 17 15
20 21 22 23 24 22
27 28 29 30 31 29
December
1
3 4 5 6 7 8
10 11 12 13 14 15
17 18 19 20 21 22
24 25 26 27 28 29
31
March
1
3 4 5 6 7 8
10 11 12 13 14 15
17 18 19 20 21 22
24 25 26 27 28 29
31
June
2 3 4 5 6 7
9 10 11 12 13 14
16 17 18 19 20 21
23 24 25 26 27 28
30
CAASD produces a 36-day
sample
CAASD’s sample attempts to
provide coverage over range
of
1. Airport delays
2. Airport weather
3. Center traffic levels
July
August
September
1 2 3 4 5
1 2
1 2 3 4 5 6
8 9 10 11 12 3 4 5 6 7 8 9 7 8 9 10 11 12 13
15 16 17 18 19 10 11 12 13 14 15 16 14 15 16 17 18 19 20
22 23 24 25 26 17 18 19 20 21 22 23 21 22 23 24 25 26 27
29 30 31
24 25 26 27 28 29 30 28 29 30
31
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
CAASD’s FY2009 Selected Scenario
Days
5 6
12 13
19 20
26 27
October
1 2
7 8 9
14 15 16
21 22 23
28 29 30
January
1
4 5 6 7 8
11 12 13 14 15
18 19 20 21 22
25 26 27 28 29
11
5 6
12 13
19 20
26 27
April
1
7 8
14 15
21 22
28 29
5 6
12 13
19 20
26 27
July
1
7 8
14 15
21 22
28 29
November
3 4
10 11 2 3 4 5 6
17 18 9 10 11 12 13
24 25 16 17 18 19 20
31
23 24 25 26 27
30
February
2 3 1 2 3 4 5
9 10 8 9 10 11 12
16 17 15 16 17 18 19
23 24 22 23 24 25 26
30 31
7
14
21
28
December
1
1 2 3 4 5 6
8 7 8 9 10 11 12 13
15 14 15 16 17 18 19 20
22 21 22 23 24 25 26 27
29 28 29 30 31
6 7 1 2
13 14 8 9
20 21 15 16
27 28 22 23
29 30
March
3 4 5 6 7
10 11 12 13 14
17 18 19 20 21
24 25 26 27 28
31
1 2
1
8 9 7 8
15 16 14 15
22 23 21 22
29 30 28 29
June
2 3
9 10
16 17
23 24
30
May
2 3 4
9 10 11 3 4 5 6 7
16 17 18 10 11 12 13 14
23 24 25 17 18 19 20 21
30
24 25 26 27 28
31
August
2 3 4
9 10 11 2 3 4 5 6
16 17 18 9 10 11 12 13
23 24 25 16 17 18 19 20
30 31
23 24 25 26 27
30 31
7
14
21
28
CAASD produces a 36-day
sample
CAASD’s sample attempts to
provide coverage over range
of
1. Airport delays
2. Airport weather
3. Center traffic levels
4 5 6
11 12 13
18 19 20
25 26 27
September
1
1 2 3 4 5
8 6 7 8 9 10 11 12
15 13 14 15 16 17 18 19
22 20 21 22 23 24 25 26
30 27 28 29 30
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Defining Airspace for Scenarios
12
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Choice of Airspace Adaptation
Demand Type 1: Base Year
Approach: Match demand day with adaptation
Base year airspace adaptations (56 day cycle)
Adaptation #1
FY2008
#2
#3
#4
= Selected base year day
FY2009
Demand Type 2: Future Year No Action
Recent airspace adaptation (started Oct. 22, 2009)
FY2013
#5
#6
#7
#8
FY2010
Approach: Recent adaptation and
consistent routes
FY2014
FY2015
= Corresponding future year day
Demand Type 3: Future Year Action
Approach: Recent adaptation and
consistent routes Plus
Recent airspace adaptation (started Oct. 22, 2009) plus
planned airspace changes (Chicago, Houston, Las Vegas, New York)
FY2013
13
FY2014
FY2015
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Future Airspace and Routes
Reroute Example
• April 10, 2008 – ZAU and C90
begin placing southbound
departures onto five new
routes (replacing existing
BACEN
three)
• Example
RBS
BLOKR
BEKKI
– April 8
KORD..RBS..FAM..LIT.J180.SWB.DAS6.KIAH
– April 17
KORD..BACEN..BLOKR..BEKKI..ROCKY..BRADL..SWB.DAS6.KIAH
• All C90 southbound departure flight plans for
scenarios prior to April 10 are placed on new
routes in the future scenarios
14
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Altitude Restrictions Example
• Impact on ZLA sector 15 as a result of updating an
altitude restriction applying to LAX arrivals
Using old
restriction
Modified
restriction
Actual Track data
Trajectories with old altitude restriction
Trajectories with modified altitude restriction
15
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Summary
16
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Summary
• MITRE CAASD generates demand for systemwide analyses
• Analysis was performed which resulted in
defining samples of size 36 days
• Day selection was performed for FY2008 and
FY2009 to provide the “best” coverage over
chosen metrics
• Airspace adaptations are chosen to match
scenarios
– Future scenarios can include planned airspace and
the corresponding route changes
17
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Backup Slides
18
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
FAA and CAASD’s FY2008 Selected
Scenario Days
1
7 8
14 15
21 22
28 29
October
2 3 4 5 6
9 10 11 12 13 4 5
16 17 18 19 20 11 12
23 24 25 26 27 18 19
30 31
25 26
6 7
13 14
20 21
27 28
January
1 2 3 4 5
8 9 10 11 12 3 4
15 16 17 18 19 10 11
22 23 24 25 26 17 18
29 30 31
24 25
6 7
13 14
20 21
27 28
April
1 2
8 9
15 16
22 23
29 30
6 7
13 14
20 21
27 28
December
1
3 4 5 6 7 8
10 11 12 13 14 15
17 18 19 20 21 22
24 25 26 27 28 29
31
March
1
3 4 5 6 7 8
10 11 12 13 14 15
17 18 19 20 21 22
24 25 26 27 28 29
31
June
2 3 4 5 6 7
9 10 11 12 13 14
16 17 18 19 20 21
23 24 25 26 27 28
30
July
August
September
1 2 3 4 5
1 2
1 2 3 4 5 6
8 9 10 11 12 3 4 5 6 7 8 9 7 8 9 10 11 12 13
15 16 17 18 19 10 11 12 13 14 15 16 14 15 16 17 18 19 20
22 23 24 25 26 17 18 19 20 21 22 23 21 22 23 24 25 26 27
29 30 31
24 25 26 27 28 29 30 28 29 30
31
==ATO-G
Day
FAA Day
19
3 4 5
10 11 12 4 5
17 18 19 11 12
24 25 26 18 19
25 26
November
1 2 3
6 7 8 9 10 2
13 14 15 16 17 9
20 21 22 23 24 16
27 28 29 30
23
30
February
1 2
5 6 7 8 9 2
12 13 14 15 16 9
19 20 21 22 23 16
26 27 28 29
23
30
May
1 2 3 1
6 7 8 9 10 8
13 14 15 16 17 15
20 21 22 23 24 22
27 28 29 30 31 29
FAA selects a peak and offpeak day for each fiscal year
quarter producing an 8-day
sample
CAASD has added 28 more
days producing a 36-day
sample
CAASD’s sample attempts to
provide coverage over range
of
1. Airport delays
2. Airport weather
3. Center traffic levels
= Expanded CAASD Sample
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Pre-Processing for Merging and
Spacing Resource
In an earlier version of systemwideModeler, flights delayed due
to airport arrival capacity took the delay at the last possible
point – the “hurry up and wait” approach
Implemented first phase of modeling improvement
• Current version now delays flights farther upstream
• Flights requiring airport arrival delay spread their delay over their
“merge nodes”
MAS
Merge
Nodes
Airport
20
Pre-processing: In the demand
generation process, the arrival flow
merge nodes are identified from
flight trajectories
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
Pre-Processing for Departure Fix
Resource
Departure sectors could be presented with too much demand
due to a directional departure push over the same fix. Departing
flights were not constrained per directional flow
Implemented modeling improvement
• When necessary, the model delays flights due to departure fix
congestion
Pre-processing: In the demand
generation process, departure
fixes are identified from flight
trajectories
ORD Departure Fixes
after April 10, 2008
21
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256
This is the copyright work of The MITRE Corporation and was produced for the U.S. Government
under Contract Number DTFA01-01-C-00001 and is subject to Federal Aviation Administration
Acquisition Management System Clause 3.5-13, Rights in Data-General, Alt. III and Alt. IV (Oct.
1996). No other use other than that granted to the U.S. Government, or to those acting on behalf of
the U.S. Government, under that Clause is authorized without the express written permission of The
MITRE Corporation. For further information, please contact The MITRE Corporation, Contract
Office, 7515 Colshire Drive, McLean, VA 22102, (703) 983-6000.
The contents of this material reflect the views of the author and/or the Director of the Center for
Advanced Aviation System Development, and do not necessarily reflect the views of the Federal
Aviation Administration (FAA) or Department of Transportation (DOT). Neither the FAA nor the DOT
makes any warranty or guarantee, or promise, expressed or implied, concerning the content or
accuracy of the views expressed herein.
2009 The MITRE Corporation. As such, the Government retains a nonexclusive, royalty-free right
to publish or reproduce this document, or to allow others to do so, for “Government Purposes Only”.
For Release to All FAA. This document has been approved for public release. Distribution is unlimited.
22
© 2010 The MITRE Corporation. All rights reserved.
F066-B10-006 Public Release No.: 10-0256