1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Probabilistic NAS Platform
George Hunter, Fred Wieland
Ben Boisvert, Krishnakumar Ramamoorthy
Sensis Corporation
December 10, 2008
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Outline
•
•
•
•
•
What is PNP?
Team and development history
Example uses of the model
Software processes and testing
Validation
2
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Outline
•
•
•
•
•
What is PNP?
Team and development history
Example uses of the model
Software processes and testing
Validation
3
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
What is PNP?
• An fast-time and flexible NAS-wide simulation tool
– Real-time or fast-time modes
• Half-hour runtime on a laptop, to simulate a day in the NAS
– Physics-based: trajectories computed through integrating aerodynamic energy
balance equations by varying the time-step size
– System uncertainties (weather, security, operations …)
– Plug-and-play architecture
• Dynamic clients (TFM, DAC, AOC, …)
– An ATC community resource
– Formal software development processes in place
– Adaptable to current system or NextGen future concepts
• Uses
– Environment in which to design, build and test decision support tools
• TFM, DAC, AOC, …
• Fast-time, real-time, shadow-mode
– Potential NAS tool
• Service provider, operator, collaborative uses
– Benefits assessment tool
• Fast-time tool to evaluate improved
infrastructure, technology, procedures …
• Evaluates historic and future traffic
scenarios in weather
4
Graphical User Interface
Plan View Display
Reports
NAS
Database
Flight Data
MATLAB®
Scripting
Interface
Probabilistic
NAS Platform
(PNP)
Weather Data
NAS Simulation
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
PNP Architecture
Performance Data
A fast-time physics-based
(trajectory-based) NAS-wide
modeling tool
5
Graphical User Interface
Plan View Display
Reports
NAS
Database
Flight Data
Probabilistic
NAS Platform
(PNP)
MATLAB®
Scripting
Interface
Weather Data
NAS Simulation
Performance Data
SimObjects
MATLAB® Client
Java Client
Client
As Middleware
Prob-TFM
A fast-time physics-based
(trajectory-based) NAS-wide
modeling tool
External Client
(Any Language)
Decision making
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
PNP Architecture
6
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
PNP Client Development
• TFM client development
– ProbTFM (Sensis internal development)
• TFM client integration
– C2 (algorithms from and used with permission of Bob Hoffman,
Metron)
– Constrained LP (algorithms from and used with permission of
NASA, Joey Rios) in progress
• DAC client integration
– MxDAC (algorithms from and used with permission of Min Xue,
NASA/UARC)
• AOC client development
– Gaming behaviors (collaboration with GMU/Lance Sherry) in
progress
7
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Capabilities Summary
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Real-time
Fast-time
Airport weather impact models
Airspace weather impact models
Weather-integrated decision making
Probabilistic modeling / decision making
Traffic flow management
Dynamic airspace configuration
Surface traffic modeling
Terminal area modeling
Super density operations
Fuel burn modeling
Emissions modeling
Trajectory-based operations
Separation assurance
Plug-n-play
Fast run-time
Existing Can Support
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
8
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Outline
•
•
•
•
•
What is PNP?
Team and development history
Example uses of the model
Software processes and testing
Validation
A fast-time physics-based
(trajectory-based) NAS-wide
modeling tool
9
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Team and Development History
2003
George
Hunter
2004
2005
Krishnakumar
Ramamoorthy
2006
Ben
Boisvert
2007
Diego
Escala
2008
Tae
Lee
Michelle
Lu
Huina
Gao
People
Project
Env’nment
System lead
Projects
Users
Software
System
Software
Java/real-time
Matlab
Data
Funct’ality
Software lead
System
Web 2.0
WSI collaboration for real-time weather feed
NAS-wide,
probabilistic
JPDO
Internal
Wx modeling
and routing
Client
architecture
NASA NRAs
Dynamic
clients
FAA NASPAC
NWA
GMU
10
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Outline
•
•
•
•
•
What is PNP?
Team and development history
Example uses of the model
Software processes and testing
Validation
A fast-time physics-based
(trajectory-based) NAS-wide
modeling tool
11
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Example Project Uses
• JPDO Modeling and Analysis
– NextGen performance evaluation with weather
• FAA NASPAC Weather Modeling
– Convection impact modeling for NASPAC
• NASA Gaming NRA
– Evaluation of NextGen gaming with AOC clients
• NASA MetaSimulation NRA
– Investigation of TFM + DAC interactions
• NASA SLDAST RFA
– Evaluation of NextGen TFM concepts and models
• NASA Market-Based TFM NRA
– Evaluation of NextGen market-based TFM concepts
12
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
NextGen Sensitivity Studies
NextGen Performance Sensitivity Analysis
Benefit of Improved Wx Forecasts
Benefit of Using Clear Weather Forecasts
NAS Performance Sensitivity
Case 2: No distinction
between clear and
heavy weather
forecast accuracy
Persistence forecast
11/16/06
Case 1: Take
advantage of improved
forecast accuracy in
clear weather
George Hunter, Fred Wieland " Sensitivity of the National Airspace System
Performance to Weather Forecast Accuracy," Integrated Communications,
Navigation and Surveillance Conference (ICNS), Herndon, VA, May, 2008
Kris Ramamoorthy, George Hunter, "Evaluation of National Airspace System
Performance Improvement With Four Dimensional Trajectories," AIAA Digital Avionics
Systems Conference (DASC), Dallas, TX, October, 2007
13
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Market-Based TFM Studies
Delay
SCC
UAL233 Delay Cost
NAS Access Valuation Models
George Hunter, et. al., "Toward an Economic Model to Incentivize Voluntary Optimization
of NAS Traffic Flow," AIAA ATIO Conference, Anchorage, AK, September, 2008.
14
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Dynamic Airspace Configuration
Nov 12, 2006, LAT=2, #Gen=40
ZFW FAA sectors
George Hunter, "Preliminary Assessment of Interactions Between Traffic Flow Management and Dynamic Airspace
15
Configuration Capabilities," AIAA Digital Avionics Systems Conference (DASC), St. Paul, MN, October, 2008.
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
AOC Dispatch Use Case
Reroute with low probability of delay
16
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Outline
•
•
•
•
•
What is PNP?
Team and development history
Example uses of the model
Software processes and testing
Validation
A fast-time physics-based
(trajectory-based) NAS-wide
modeling tool
17
Processes and Testing Cycle
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Project Monitoring & Control
Development Tracking
Quantitative Project Management
Branch
Configuration
Management
Regression
Testing
Unit and System
Testing
Trunk
Configuration
Management
18
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Project Monitoring and Control
• JIRA is used to track issues
– Project Manager and Lead Software Engineer assign task priorities, due dates,
and personnel.
• Weekly telecoms keep distributed team apprised of
PNP and communications open
• Project Manager maintains a master schedule in MSProject
19
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Development Tracking
• Software engineers use JIRA to track and status
development efforts.
20
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Branch Configuration Management
• Software Engineers are responsible for creating
branches from the trunk to develop
fixes/enhancements.
• The Configuration Management of the software is
accomplished with Subversion
– Subversion is an open source version control system
(http://subversion.tigris.org/)
21
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Unit and System Testing
• Software Engineers are responsible for creating unit tests to
verify the correctness of their code. The JIRA issue number is to
be used throughout the code and unit tests for tracking
purposes.
• Software Engineers are responsible for running their own
system/function tests to verify their software.
• Once testing is validated, code is merged back on to the trunk.
22
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Trunk Configuration Management
• Once all validated JIRA issues are merged
unto the trunk, regression testing is
performed.
23
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Regression Testing
• Regression testing
• Aggregate results
–
–
–
–
–
Total delay
Total congestion
Traffic volume
#TFM initiatives
Runtime
• Different scenarios
– Truncated demand set
– Full demand set
– Weather
• Automated
– Weekly or as required
• Archived
• Graphical quick-look
24
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Quantitative Project Management
• Regression testing validation is performed and the
release letter is updated.
• Release is tagged in Subversion.
• JIRA issues are closed.
• Documentation is updated to reflect changes in
software.
25
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Outline
•
•
•
•
•
What is PNP?
Team and development history
Example uses of the model
Software processes and testing
Validation
A fast-time physics-based
(trajectory-based) NAS-wide
modeling tool
26
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
System-Level Engineering Validation
• ASPM / ETMS verification tests
√
– Compare ASPM/ETMS data with simulation data
• Calibrate concept to match aggregate field observations
– Models
• Trajectory data
• Airport capacities (VMC / IMC)
• Sector capacities in weather
– Aggregate performance
• Mean flight delay
• Sector and airport overloadings
– Detailed performance
• Flight delay by airport and time of day
• Overloading and delay patterns (Spatial and temporal)
 Delays by airport and time of day
 Sector and airport loading by time of day
 Spatial loading patterns
– Light and heavy weather days
27
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
System-Level Software Verification
• Cross check sums
–
–
–
–
–
SFlights = SOperations at all airports
SFlight time = SMinutes from sector loads
SSector load by sector = SSector load by time
SAirport ops = SFlights using the airport in demand set
SDelays by flight = SDelays by time; and reroutes
• Weather data checks
–
–
–
–
Compare PNP/Metar airport capacity with ASPM AAR/ADR
Compare PNP/Metar airport capacity with ASPM IFR periods
Ensure SEn route convection versus time of day is smooth
Ensure WxMAP ≤ MAP for all sector time bins
• Graphical
– Ensure reroutes overlaid on weather make sense
• TFM Performance
– Number of delays per flight, min and max flight delay
– Maximum airport and sector overloading (ensure are reasonable)
28
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
System-Level Engineering Validation
• ASPM / ETMS verification tests
– Compare ASPM/ETMS data with simulation data
• Calibrate concept to match aggregate field observations
– Models
√
• Trajectory data
• Airport capacities (VMC / IMC)
• Sector capacities in weather
– Aggregate performance
• Mean flight delay
• Sector and airport overloadings
– Detailed performance
• Flight delay by airport and time of day
• Overloading and delay patterns (Spatial and temporal)
 Delays by airport and time of day
 Sector and airport loading by time of day
 Spatial loading patterns
– Light and heavy weather days
29
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Trajectory Model Validation
• Compared to ETMS flight data (May 2008)
N: 316
Mean: 0.321 min
Std dev: 11.95 min
Detrended for Range
George Hunter, Ben Boisvert, Kris Ramamoorthy, "Advanced Traffic Flow
Management Experiments for National Airspace Performance Improvement,"
2007 Winter Simulation Conference, Washington, DC, December, 2007
Mean: 0.80 min
Std dev: 6.51 min
R2: 0.012
30
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
ProbTFM Performance
• ASPM / ETMS verification tests
– Compare ASPM/ETMS data with simulation data
• Calibrate concept to match aggregate field observations
– Models
• Trajectory data
• Airport capacities (VMC / IMC), actual and forecasted
• Sector capacities in weather, actual and forecasted
√
– Aggregate performance
• Mean flight delay
• Sector and airport loadings
– Detailed performance
• Flight delay by airport and time of day
• Overloading and delay patterns (Spatial and temporal)
 Delays by airport and time of day
 Sector and airport loading by time of day
 Spatial loading patterns
– Light and heavy weather days
31
• Compare to ETMS/ASPM
– Forecast accuracies, Decision
making horizon, Delay distribution
January 7 2007
3000
January 7, 2007
(Similar results
with other days)
2500
LAT = 60 minutes
Sector Congestion
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Compare With Field Observations
2000
(14.5,1657)
1500
LAT = 30 mins
1000
LAT = 0
500
0
0
5
10
15
20
25
Average Delay (minutes per aircraft)
32
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Verification of Results
• ASPM / ETMS verification tests
– Compare ASPM/ETMS data with simulation data
• Calibrate concept to match aggregate field observations
– Models
• Trajectory data
• Airport capacities (VMC / IMC), actual and forecasted
• Sector capacities in weather, actual and forecasted
– Aggregate performance
• Mean flight delay
• Sector and airport loadings
√
– Detailed performance
• Flight delay by airport and time of day
• Overloading and delay patterns (Spatial and temporal)
 Delays by airport and time of day
 Sector and airport loading by time of day
 Spatial loading patterns
– Light and heavy weather days
33
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
System Loading Patterns
ProbTFM predicted, 14:45 GMT
ETMS Actual, 14:45 GMT
ETMS
Underloading
Overloading
ETMS
ProbTFM
ProbTFM loading
34
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Verification of Results
• ASPM / ETMS verification tests
– Compare ASPM/ETMS data with simulation data
• Calibrate concept to match aggregate field observations
– Models
• Trajectory data
• Airport capacities (VMC / IMC), actual and forecasted
• Sector capacities in weather, actual and forecasted
– Aggregate performance
• Mean flight delay
• Sector and airport loadings
– Detailed performance
• Flight delay by airport and time of day
• Overloading and delay patterns (Spatial and temporal)
 Delays by airport and time of day
 Sector and airport loading by time of day
 Spatial loading patterns
√
– Light and heavy weather days, control days
35
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Conclusion
• The development of PNP has benefited from lessons
learned over past two decades in NAS system wide
modeling
– Plug and play simulation architecture
– Supports both analytical and HITL studies
– Adaptable to simulate current system as well as NextGen future
concepts
– Fast-time, physics-based
– Formal software development processes in place
– Probabilistic decision making and extensive weather modeling
explicitly incorporated in tool
36
Publications
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
George Hunter, "Preliminary Assessment of Interactions Between Traffic Flow Management and Dynamic Airspace
Configuration Capabilities," AIAA Digital Avionics Systems Conference (DASC), St. Paul, MN, October, 2008.
George Hunter, et. al., "Toward an Economic Model to Incentivize Voluntary Optimization of NAS Traffic Flow," AIAA ATIO
Conference, Anchorage, AK, September, 2008.
George Hunter, Fred Wieland " Sensitivity of the National Airspace System Performance to Weather Forecast Accuracy,"
Integrated Communications, Navigation and Surveillance Conference (ICNS), Herndon, VA, May, 2008.
George Hunter, Kris Ramamoorthy, "Integration of terminal area probabilistic meteorological forecasts in NAS-wide traffic
flow management decision making," 13th Conference on Aviation, Range and Aerospace Meteorology, New Orleans, LA,
January, 2008.
Kris Ramamoorthy, George Hunter, "The Integration of Meteorological Data in Air Traffic Management: Requirements and
Sensitivities," 46th AIAA Aerospace Sciences Meeting and Exhibit, Reno, NV, January, 2008.
George Hunter, Ben Boisvert, Kris Ramamoorthy, "Advanced Traffic Flow Management Experiments for National Airspace
Performance Improvement," 2007 Winter Simulation Conference, Washington, DC, December, 2007.
Kris Ramamoorthy, George Hunter, "Evaluation of National Airspace System Performance Improvement With Four
Dimensional Trajectories," AIAA Digital Avionics Systems Conference (DASC), Dallas, TX, October, 2007.
Kris Ramamoorthy, Ben Boisvert, George Hunter, "Sensitivity of Advanced Traffic Flow Management to Different Weather
Scenarios," Integrated Communications, Navigation and Surveillance Conference (ICNS), Herndon, VA, May, 2007.
George Hunter, Ben Boisvert, Kris Ramamoorthy, "Use of automated aviation weather forecasts in future NAS," The 87th
American Meteorological Society Annual Meeting, San Antonio, TX, January, 2007.
Kris Ramamoorthy, George Hunter, "Probabilistic Traffic Flow Management in the Presence of Inclement Weather and Other
System Uncertainties," INFORMS Annual Meeting, Pittsburgh, PA, November, 2006.
Kris Ramamoorthy, Ben Boisvert, George Hunter, "A Real-Time Probabilistic TFM Evaluation Tool," AIAA Digital Avionics
Systems Conference (DASC), Portland, OR, October, 2006.
George Hunter, Kris Ramamoorthy, Alexander Klein "Modeling and Performance of NAS in Inclement Weather," AIAA
Aviation Technology, Integration and Operations (ATIO) Forum, Wichita, KS, September 2006.
Kris Ramamoorthy, George Hunter, "A Trajectory-Based Probabilistic TFM Evaluation Tool and Experiment," Integrated
Communications, Navigation and Surveillance Conference (ICNS), Baltimore, MD, May, 2006.
Kris Ramamoorthy, George Hunter, "Avionics and National Airspace Architecture Strategies for Future Demand Scenarios in
Inclement Weather," AIAA Digital Avionics Systems Conference (DASC), Crystal City, VA, October, 2005.
George Hunter, Kris Ramamoorthy, Joe Post, "Evaluation of the Future National Airspace System in Heavy Weather," AIAA
Aviation Technology, Integration and Operations (ATIO) Forum, Arlington, VA, September 2005.
James D. Phillips, “An Accurate and Flexible Trajectory Analysis,” World Aviation Congress (SAE Paper 975599), Anaheim,
CA, October 13-16, 1997.
37
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Questions?
38
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Backup
39
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
PNP Systems Requirements
• System requirements
– PNP is a Java application
– Hardware
• Memory: minimum 1GB, preferred 2GB
• CPU: Pentium (4) 3.2 GHz or better
• Video card: 128MB memory, preferred 256MB
– Software
• Java JDK 6 http://java.sun.com/javase/downloads/index.jsp
• MySQL Server 5.0 http://dev.mysql.com
– Third party licenses
• Eurocontrol BADA usage license
40
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Weather Days
• Ten weather days, two control days
41
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Weather Days
• Weather days
– Spectrum of weather days
• Variation in weather type and intensity
• Variation in season
– Support real-world comparison
• Support same sector data
• Variation in traffic demand volume and structure
 Different days of week, holidays
• Control days
42
43
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
NextGen Performance
Sensitivity Analysis
44
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
En Route and Terminal Area Combined Sensitivities - 2025
45
46
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Benefit of Improved
Convection Forecasts
47
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Investment Analysis
48
49
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Benefit of Using Clear Weather Forecasts
50
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Benefit Evaluation
Case 2: No distinction
between clear and heavy
weather forecast accuracy
Case 1: Take advantage of
improved forecast
accuracy in clear weather
Persistence forecast, 11/16/06
51
52
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Market-Based TFM:
Valuation of NAS Access
53
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Congestion-Delay Relationship
• Unconstrained sector congestion cost (SCC) for
zero lookahead time (blue) and PNP-ProbTFM
simulated delay (black) time histories for all en route
NAS sectors and flights, respectively.
Delay
SCC
54
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Aggregate Delay Model
• Hypothesize a first-order lag transfer function
SCC(s)
K
1
s  1
Delay(s)
Simulated delay
Modeled delay
55
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Aggregate Delay Model
• Hypothesize a second-order transfer function
Simulated delay
Modeled delay
56
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Transfer Functions Summary
57
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Explicit Cost Model
• Evaluate cost of NAS access by removing the flight
• Remove one flight
– 11/16/06, UAL233, A320
– Morning departure from Bradley International (KBDL) to Chicago
O’Hare airport (KORD)
– Relatively high cost flight
• 90.02 SCC
58
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Remove UAL233
• Delay reduction by time bin in simulation run
– Delay reduction of 8141 minutes
59
60
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
NAS Performance Sensitivity Studies
• Performance sensitivity to:
• Delay distribution policy (most important factor)
• TFM system agility
• System forecasts (least important factor)
Nov 12, 2006
ETMS/ASPM
Non Agile
Delay Distribution
61
62
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Dynamic Airspace Configuration
63
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
NAS Sectorization
• Nov 12, 2006
64
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
MxDAC Afternoon Sectorization
• Nov 12, 2006, LAT=6, #Gen=20
65
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Coeff_peak_ac_var=0.0
Coeff_avg_ac_var=0.0
Coeff_crossings=0.0
Coeff_transition_time=0.0
Coeff_residual_capacity=1.0
MxDAC Midday Sectorization
• Nov 12, 2006, LAT=2, #Gen=40
66
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Delay-Congestion Performance
MxDAC on, LAT = 4 hrs
MxDAC on, LAT = 2 hrs
MxDAC off
67
68
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Equity Analysis:
Cost of Delay Distribution
69
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Cost of Distributing Delay
• RMS delay can be reduced by spreading delay to
more flights
– But at the cost of increased total delay
Nov 12, 2006
$65/minute
Increased delay distribution
70
71
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
AOC Dispatch Use Case
72
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Dispatcher Successfully Finds a Reroute
Reroute with low probability of delay
73
74
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Project Monitoring and Control
• JIRA is used to track issues
– Project Manager and Lead Software Engineer assign task priorities, due dates,
and personnel.
• Weekly telecoms keep distributed team apprised of
PNP and communications open
• Project Manager maintains a master schedule in MSProject
75
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Development Tracking
• Software engineers use JIRA to track and status
development efforts.
76
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Branch Configuration Management
• Software Engineers are responsible for creating
branches from the trunk to develop
fixes/enhancements.
• The Configuration Management of the software is
accomplished with Subversion
– Subversion is an open source version control system
(http://subversion.tigris.org/)
77
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Unit and System Testing
• Software Engineers are responsible for creating unit tests to
verify the correctness of their code. The JIRA issue number is to
be used throughout the code and unit tests for tracking
purposes.
• Software Engineers are responsible for running their own
system/function tests to verify their software.
• Once testing is validated, code is merged back on to the trunk.
78
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Trunk Configuration Management
• Once all validated JIRA issues are merged
unto the trunk, regression testing is
performed.
79
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Regression Testing
• Regression testing
• Aggregate results
–
–
–
–
–
Total delay
Total congestion
Traffic volume
#TFM initiatives
Runtime
• Different scenarios
– Truncated demand set
– Full demand set
– Weather
• Automated
– Weekly or as required
• Archived
• Graphical quick-look
80
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Quantitative Project Management
• Regression testing validation is performed and the
release letter is updated.
• Release is tagged in Subversion.
• JIRA issues are closed.
• Documentation is updated to reflect changes in
software.
81
1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08
Risk Management
• Lessons learned analysis
– A wrap up meeting is held to discuss all issues on a project in
which proactive steps can be documented to avoid the same
mistakes
82