Performance Review Commission ATM ATM Performance Performance Framework Framework NAS Performance Workshop 5 September 2007 Xavier FRON Performance Review Unit EUROCONTROL 2 Role of performance review Legislation Institutions Industrial Organisation Policy makers Advice on performance issues Policy objectives Performance information Airspace users Airport operators Air Navigation ANS Providers Supervisory authorities Common functions Performance Review Achieved Performance Consultation Raw information • Provide independent advice on ANS/ATM performance to policy makers and relevant information to all stakeholders (e.g. benchmarking and best practice), based on observation of achieved performance, consultation and information provided by relevant parties; • Performance review closes the performance loop; • It is the least intrusive form of regulation; Performance Review Commission www.eurocontrol.int/prc 3 Performance Review Commission (PRC) Independent advisory role to EUROCONTROL governing bodies Twelve commissioners supported by the PRU Objective “to introduce strong, transparent and independent performance review and target setting to facilitate more effective management of the European ATM system…” 37 States 9.6 M flights Annual Performance Review Reports (PRR) – Traffic – Key ATM Performance Areas • • • • Safety Cost-effectiveness Quality of service/ Environment Capacity ANSP benchmarking reports (ACE) Special reports – Evaluation of SES impact on ATM performance – Fragmentation – Punctuality drivers, etc. – US/Europe comparison – Comparison of aeronautical MET costs PRR 2006 just published Performance Review Commission www.eurocontrol.int/prc 4 Overview of ATM performance measurement framework Political & SocioEconomic Expectations SAFETY ECONOMY ENVIRONMENT SECURITY PRC Key performance areas Safety Airspace user needs and requirements Service Provider Perspective/ ATM contribution Service Quality User charges Safety •Efficiency •Predictability •Flexibility Security Cost Effectiveness Quality of service Safety management Costeffectiveness Capacity Security management Environment Capacity Performance affecting factors Technical innovations Network effects & fragmentation Traffic volume, variability, complexity) Weather Regulations/ restrictions (Political & Environmental) Prevailing economic conditions • Various perspectives on ATM performance (Political/ Social, User, Service provider); • The PRC focuses on: Safety, Cost-effectiveness, Service Quality (Delays, flight efficiency) , Capacity and Environment (Global aspects) • ATM performance is affected by trade-offs (capacity vs. delay, etc.) and a number of performance affecting factors (weather, complexity, etc.) which need to be captured in a balanced view Performance Review Commission www.eurocontrol.int/prc 5 How the PRC analyses ANS performance Performance Drivers Performance Indicators Legislative framework Safety maturity Culture Safety Management Systems Accidents En-route delays Capacity Airport delays Cost of living Fragmentation Airspace design & use Civil/Military use of airspace (Chapter 7) Safety (Chapter 3) Air Transport performance Airline Performance Incidents (a/c proximities) Traffic demand (Chapter 2) Complexity ANS Key Performance Areas Punctuality & Predictability (Chapter 4) ATFM Delays (Chapter 5) Productivity AIS, MET costs ATCO Unit Costs ANSP costs Support Costs Horizontal Flight efficiency ANS Performance Cost Effectiveness (Chapter 8) Airport Performance EUROCONTROL costs Flight Efficiency (Chapter 6) Vertical Flight Efficiency Performance Review Commission www.eurocontrol.int/prc 6 Air traffic demand in Europe Annual growth in IFR Movements 2006 10 8 Below 0% 2007 Forecas t Traffic 0% to 3% 6 4 2 3% to 6% 6% 6% 2% 2% 9% to 12% 3 % 5% 4 % 4 % 4 % 3 %3% 3%3 % 3 % 6% 4% 0% Year o n year variatio n 6% to 9% 2% 6 % 5% 6% 7% 4% 5% 5% Above 12% -2% 0 2012 2010 2008 2006 2004 2002 2000 1998 1996 1994 1992 11% 1990 13% 11% 2% 5% ( before 1997, estimation based on Euro 88 traffic variation) 3% 19% 6% 5% 4% so urce : EUR OC ON TR OL 13% 0% 6% Sustained growth continued in 2006 4% 4% 2% 7% 5% 1% 4%4% • Average annual growth +4.1%; 6% 5% 9% Azores • Between 0% and 19% at State level; 4% 6% 2% 7% 2% 5% • +24% for “low fare” airlines (16% of traffic) 8% 3% 5% • +11% for business aviation (7% of traffic) Canarias 3% 0% Produced by the D ata Information & Analysis (DIA) Business Division. © EURO C O NT RO L 2006. million flights per year 12 data source : EUROCONTROL/STATFOR Performance Review Commission www.eurocontrol.int/prc 7 Traffic forecasts • Short, medium , long term forecasts from EUROCONTROL STATFOR • Challenges to Growth study (2004) being updated • Suppressed demand due to airport capacity limitations Heathrow traffic has grown by filling the gaps Performance Review Commission www.eurocontrol.int/prc 8 Measuring Safety Performance Airspace Users Airports Safety performance of air transport ANS Safety Performance Accidents Incidents (a/c proximities) Reactive Measurement (Outcome) Safety maturity Legislative framework Culture Proactive Measurement (Process, safety Safety Enablers) Management • Approach to measuring safety is wider than merely focusing on the level of achieved safety of the existing ATM system under certain conditions (accidents, incidents); • Maturity of safety processes are as important as the measurement of achieved safety; Performance Review Commission www.eurocontrol.int/prc 9 Accidents/incidents 20 Total number of acccidents 16 Direct ATM Accidents Accidents Trend CLassified events 18 14 12 10 8 6 4 2 0 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 Data source: Flight Safety Foundation - Aviation Safety Net A posteriori check Lagging indicator It’s too late! Airspace Users 1400 1200 Incident reports Severity C,D,E 1000 800 600 400 200 0 Severity B Severity A 38% 2002 15 States 28% 2003 21% 2004 23% % :Proportion of Severity A+B 2005P Data so urce : EUROCONTROL/SRU Leading indicator Informs safety management Airports Safety performance of air transport Voluntary reports: Useful, but reliable? Systematic screening Loss of separation, runway incursions, etc ANS Safety Performance Accidents Incidents (a/c proximities) Reactive Measurement (Outcome) Safety maturity Legislative framework Culture Proactive Measurement (Process, safety Safety Enablers) Management Performance Review Commission www.eurocontrol.int/prc 10 Safety maturity ANSPs 100 Legislation Culture Austria █ █ Belgium █ █ Bulgaria █ █ Cyprus █ █ Czech Rep. █ █ Denmark █ █ Finland █ █ France █ █ Germany █ █ Greece █ █ Hungary █ █ Ireland █ █ Italy █ █ Luxembourg █ █ Netherlands █ █ Norway █ █ Poland █ █ Portugal █ █ Romania █ █ Slovakia █ █ Slovenia █ █ Spain █ █ Sweden █ █ Switzerland █ █ UK █ █ 2002 90 2004 2006 80 70 Maturity Score 60 50 40 30 20 Maturity surveys 10 0 0 5 10 15 20 25 30 35 40 Are safety processes, legislation, culture in place? Airspace Users Airports Safety performance of air transport ANS Safety Performance Accidents Incidents (a/c proximities) Reactive Measurement (Outcome) Safety maturity Legislative framework Culture Proactive Measurement (Process, safety Safety Enablers) Management Performance Review Commission www.eurocontrol.int/prc 11 Safety Performance targets CLassified events TODAY Performance to date 1400 1200 Severity C,D,E 1000 800 Severity B 600 400 Severity A 38% 200 0 2002 15 States 28% 2003 23% 21% 2004 % :Proportion of Severity A+B Data so urce : EUROCONTROL/SRU OUTLOOK System risk is quadratic: Incidents/hour x4 when traffic x2 3 2 1 0 8 9 10 11 12 13 14 Traffic (million flights) Risk trend Risk target 15 16 Target (s) • European (ATM 2000+) objective: Number of accidents and serious incidents not to increase: Very challenging! 2005P 4 • Increasing number of incidents reports: more opportunities for learning, prevention • Severe incidents (A&B) don’t appear to increase in sample of 15 “mature” European States Aircraft risk is linear: Incidents per flight-hour x2 when traffic x2 SESAR target System risk does not increase (no more accidents) 2020: Traffic: +73% => Safety x3 vs trend Later: Traffic x3 => Safety x10 vs trend • Current target: <1.55.10-8 accident per flight hour No corresponding indicator so far • PRC proposed interim target (maturity) Future system • Safety may be the most challenging • Safety needs to be engineered in next generation design from the start Performance Review Commission www.eurocontrol.int/prc 12 Service quality Political & SocioEconomic Expectations SAFETY Airspace user needs and requirements Service Provider Perspective/ ATM contribution Ambient performance affecting factors ECONOMY Safety Costeffectiveness Safety management Costeffectiveness Technical innovations Network effects & fragmentation ENVIRONMENT Service Quality •Flight efficiency •Predictability •Flexibility Capacity Traffic volume, variability, complexity) SECURITY Weather Security Security management Regulations/ restrictions (Political & Environmental) Prevailing economic conditions Performance Review Commission www.eurocontrol.int/prc 13 Analysis of Air transport delays 12% 5% 3% 5% 3% 1% 1% 1% >30 28-30 26-28 24-26 22-24 20-22 18-20 16-18 0% 14-16 Early arrival 11% 10% 12-14 Buffer 15% 10-12 Taxi In 18% 20% 6-8 Airborne ON Time 8-10 Taxi Out IN 4-6 ON 2-4 Schedule OFF 22% 22% 0-2 OUT % of operated services (2004) "Time to Take-off" distribution - Charles de Gaulle Airport 25% Time bands in minutes Data source: EUROCONTROL/ eCODA Late arrival Optimising performance Ahead of schedule Delay Behind schedule High added-value: Compressing half of European flight schedules by 5 minutes is worth some €1,000M p.a. Closer to optimum Mean (50%) Punctuality % On-time 80% Std Dev Time Number of observations (2) (2) (1) (1) Predictability Spread (Standard deviation) (1) (1) Reduce variability Time of operation Performance Review Commission www.eurocontrol.int/prc 14 69.0% 74.4% 75.2% 82.5% 83.6% 81.8% 79.8% Departure Punctuality 78.1% 35% source: AEA source: eCODA 30% 25% Punctuality 13% 10% 20% 15% 10% 9% 10% 5% 7.9% 5.6% 2000 7% 9% 10% 7% 6% 8% 8% 8% 9% 10% 1.1% 1.2% 1.2% 1.1% 10% 4.5% 0% 1999 Primary delay distribution (2006) 9% 2001 9% 79% 2.1% 1.6% 1.4% 1.5% 1.6% 2002 2003 2004 2005 2006 12% En-route ATFM regulations due to ATC Airport ATFM regulations (weather, capacity) Turn around delays (airline, airport, etc.) Reactionary delays • 21.4% of flights arrived late in 2006 (23.1% in US) • Departure delays originate principally from turnaround processes (79% of primary dep. delays) • Reactionary delays are increasing ratio reactionary to primary delay % of flights with departure delay > 15 min. Departure punctuality Sensitivity of the European air transport network to primary delays 0.80 0.75 Reactionary/primary delays 0.70 0.65 0.60 0.55 0.50 1999 2000 AEA data 2001 2002 2003 2004 2005 2006 eCODA data Primary delay includes local turn around delays and en-route and airport ATFM delays Performance Review Commission www.eurocontrol.int/prc 15 Arrival and departure delays Departure punctuality of inbound flights at departure airports Departure punctuality of outbound flights at analysed airports Arrival punctuality of inbound flights at arrival airports FLIGHT PERSPECTIVE 25% Dep. to Arr. at Dep. from Dep. to Arr. at Dep. from FCO FRA LHR MAD MUC MXP 20% Dep. to Arr. at Dep. from CDG 19% Dep. to Arr. at Dep. from 20% 20% Dep. to Arr. at Dep. from BCN 10% 18% 20% Dep. to Arr. at Dep. from 23% 29% 20% Dep. to Arr. at Dep. from 15% 25% 20% Dep. to Arr. at Dep. from 5% VIE Dep. to Arr. at Dep. from 0% Dep. to Arr. at Dep. from % of flights departing/ arriving more than 15 min. behind schedule (2005) 30% ZRH Drivers of departure delay: En-route ATFM delay Airport ATFM delay Local turn-around delay (airlines, airport, etc.) Reactionary delay source: eCODA Arrival delays are mostly driven by departure delays Departure delays mostly from airlines/airports processes Amplification of delays at some airports (Departure delays > arrival delays) Performance Review Commission www.eurocontrol.int/prc 16 Air transport predictability Flights from/to Europe (11%) 17.5 15.0 15.0 12.5 12.5 10.0 10.0 7.5 7.5 5.0 5.0 2.5 2.5 0.0 0.0 2003 (1) (1) (1) (1) Reduce variability Taxi out + holding Arrival time Gate-to-gate variability 2002 (2) (2) Number of observations 17.5 Closer to optimum Time of operation Arrival time 20.0 Taxi in + waiting for gate 20.0 Flight times (cruising + terminal) 22.5 Departure time 22.5 Taxi in + waiting for gate 25.0 Flight times (cruising + terminal) 27.5 25.0 Taxi out + holding 27.5 Departure time Standard deviation (minutes) Intra European flights (89%) Gate-to-gate variability 2004 2005 2006 • Standard deviations of departure and arrival times reached 18 and 20 minutes respectively • Pre-departure processes play a main role in this poor predictability, and ATM only a minor role. • Lower punctuality and predictability negatively impacts the ability of airlines and airports to build and operate reliable and efficient schedules. Performance Review Commission www.eurocontrol.int/prc 17 Airport capacity/ delay trade-off QUEUE LENGTH (units of landing interval) 5 4.5 Variability of TMA entry flow (std. dev.) =20 MIN =2 MIN 4 Reduced variability of landing interval 3.5 3 Reduced variability of TMA entry flow 2.5 2 1.5 1 Variability of landing interval (std. dev.) 0.5 0 0.8 0.82 0.84 0.86 0.88 0.9 0.92 39 Sec. 30 Sec. 18 Sec. 0 Sec. 0.94 0.96 0.98 1 Runway utilisation ratio • Trade-off airport capacity / airborne delay • Airport scheduling impacts ATM performance (TMA holding, environmental impact) • Smoothing arrival flows and landing rates significantly improves the trade-off Performance Review Commission www.eurocontrol.int/prc 18 Improving air transport network performance Network delivery (volume and variability of TMA entry flow) TMA Departure time variability Reactionary delays Local turnaround delays Arrival time variability Departure airport Arrival airport Pre-departure delays En-route ATFM delays Airborne holding Landing interval (actual throughput) Management of arrival flows Airport scheduling (utilisation ratio) Airport ATFM delays • Air transport punctuality and predictability could be improved in several ways: - Improving adherence to scheduled departure times - Maximising the use of airport capacity whilst minimising delays. - Optimising the ground vs. airborne holding trade-off. • SESAR places emphasis on flexibility (ability to recover from non nominal situations) • Comparable metrics to be developed and agreed (Punctuality, ATM ground and airborne delay, etc). Performance Review Commission www.eurocontrol.int/prc 19 Ground delays managed by ATM 6 TODAY Summer (May-October) 6.000 ATFM delays (ground) 5.000 Target: 1.0min/flight 4 Actual: 1.4min/flight 3 En route target En route optimum 2 4.000 3.000 2.000 1 1.000 0 2,9 4,1 5,5 3,6 3,1 1,8 1,2 1,2 1,3 1,4 0.8 0.9 0.8 0.9 0.7 0.6 0.6 0.7 0.9 0,8 Flights in Summer ('000) minutes per flight 5 0 -1 • Flows essentially controlled through ground delays in Europe, MIT in US • Major improvement in Europe since 1999 • 2002-05 ATFM en-route target met; • ATFM delays increasing again since 2004; • Estimated en-route ATFM delay costs: € 550 M in 2006 -1.000 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 En-route ATFM delay per flight (May-Oct.) Airport ATFM delay per flight (May-Oct.) OUTLOOK 300 Traffic volume (km) Effective Capacity En-route ATFM delay 275 source : EUROCONTROL/CFMU European delay target ACC capacity targets European delay forecast ACC capacity commitments 30 25 225 million minutes 20 200 175 15 150 • Trade-off delay/cost of capacity; • Set with reference to optimum capacity/delay • En-route ATFM: 1 minute per flight • Others (i.e. ACC) to be developed Management • Co-operative capacity management; 10 125 100 5 75 Closing capacity gap Widening capacity gap Closing capacity gap 2010 2009 2008 2007 2006 2005 2004 2003 2002 2001 2000 1999 1998 1997 1996 1995 1994 1993 1992 0 1991 50 1990 km index 1990=100 250 Target (s) source : EUROCONTROL Performance Review Commission www.eurocontrol.int/prc 20 ATFM Delays: Target setting Trade-off delay/cost of capacity Target setting based on understanding of optimum Optimum changes with improved cost-effectiveness (dynamic efficiency) Yearly costs • • • Static economic optimum Total economic costs Cost of capacity Dynamic economic optimum Delay costs Capacity/demand ratio Performance Review Commission www.eurocontrol.int/prc 21 ATFM delay causes Annual 12 000 600 10 000 500 8 000 400 6 000 300 4 000 200 2 000 100 MAR APR ATC CAPACITY (2006) ATC STAFFING (2006) OTHER (2006) MAY JUN JUL AUG SEP OCT NOV DEC ATC IND ACTION (2006) WEATHER (2006) Total en-route ATFM delay (2005) 2006 FEB 2005 JAN 2004 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 En-route ATFM delay ('000) Weekly en-route ATFM delay distribution 700 source: EUROCONTROL Performance Review Commission www.eurocontrol.int/prc 22 Service quality (continued) Political & SocioEconomic Expectations SAFETY Airspace user needs and requirements Service Provider Perspective/ ATM contribution Ambient performance affecting factors ECONOMY Safety Costeffectiveness Safety management Costeffectiveness Technical innovations Network effects & fragmentation ENVIRONMENT Service Quality •Flight efficiency •Predictability •Flexibility Capacity Traffic volume, variability, complexity) SECURITY Weather Security Security management Regulations/ restrictions (Political & Environmental) Prevailing economic conditions Performance Review Commission www.eurocontrol.int/prc 23 Flight Efficiency Actual Route (A) G Direct route Extension D Route Extension Direct Course (D) A TMA Interface 30 NM Great Circle (G) ro u t e e x t e n s io n ( k m /f lig h t ) 60 50 - 2 km per flight 40 30 PRC uses same framework as ICAO ANSEP 20 Direct en route TMA Interface 10 Proposed target 0 2004 2005 2006 2007 2008 2009 2010 Performance Review Commission www.eurocontrol.int/prc 24 Flight efficiency (horizontal) Fuel price 60 40 20 2001 2002 2000 1999 1998 1997 1995 1996 Direct link with environmental impact 1994 4.7 M tons Additional CO2 emissions 1993 Flight-efficiency is a main issue 1992 € 2 230 M 1990 1991 Estimated costs to airspace users 1989 441 M km 1988 Additional distance 1987 48.6 km 2006 0 Extension per flight 2005 5.9% 80 2004 Route Extension (%) 100 2003 Total 2006 US$ 2006 per barrel 120 data so urce : U.S. Department o f Energy (Ro tterdam Jet Fuel Spo t P rice) • Cost of horizontal route inefficiencies is estimated at 2.2 billion euro, to which vertical and TMA inefficiencies have to be added. • Significant environmental impact (4.7 million tons of CO2 per annum) • Costs increased further due to higher fuel prices in 2006 Performance Review Commission www.eurocontrol.int/prc 25 Improving Flight-efficiency Actual route (A) 874.6 km ATC routing - 8.2 km Filed Route (F) -1.0% 882.8 km En-route Extension Route selection + 8.7 km Shortest Route (S) 1.1% + 48.6 km 5.9% 874.1 km En-route design + 48.0 km Great Circle (G) ATC routing Route selection En-route design Total 826.0 km 5.8% Strategic design and use of airspace are ATC routing Route theselection main origins of En-route routedesign inefficiencies Total Performance Review Commission www.eurocontrol.int/prc 26 2000 city pairs or 150 Most constraining point most constraining points B Cumulative share of extra miles generated En-route design 100% 90% 80% 70% 60% 50% 50% of the flight inefficiencies are generated by 2100 origin-destination 40% 30% 20% 10% 0% A 0 2000 4000 6000 Cumulative share of extra miles generated Number of origin-destination 100% 90% 80% 70% 60% 50% 50% of the flight inefficiencies are generated by 150 points 40% 30% 20% 10% 0% 0 200 400 600 800 1000 Number of constraining points Performance Review Commission www.eurocontrol.int/prc 27 Flight efficiency (horizontal) There appears to be room for significant improvements: 5% 4.1% 4.1% 3.8% 4% 3.8% 3% 2005 2006 2% •Only minimal improvements in flight efficiency during week-ends (essentially no airspace restrictions) => 130 million Euro could be saved every year if the route network was one third more efficient during week-ends. 1% 0% W eek Days W eek End Direct route efficiency Domestic Intra-European Domestic-AVG Intra-European-AVG Direct routeextension (%) 7% •Intra-European routes are significantly less efficient than domestic routes. 6% 5% 4% 3% 2% 1% 0% 0-400 km 400-800 km 800-1200 km 1200-1600 km 1600 -2000 km Great Circle Distance (between TMA) >2000 km => If the European route network was as efficient as the domestic networks, as one would expect under the SES, 150 to 300 million Euro could be saved every year. Performance Review Commission www.eurocontrol.int/prc 28 Flight efficiency (horizontal) • Trade-off capacity – flight-efficiency: don’t jeopardize capacity where little margin • Objective: a more efficient Trans-European network of upper airways Demand Performance Review Commission www.eurocontrol.int/prc 29 Flight-efficiency: Route selection Example: Amsterdam to Torino Airspace structure Actual route (A) 874.6 km ATC routing - 8.2 km Filed Route (F) En-route congestion -1.0% + 8.7 km En-route Extension 1.1% + 48.6 km 5.9% 874.1 km En-route design + 48.0 km Great Circle (G) ATC routing Route selection En-route design Total 882.8 km Route selection Shortest Route (S) 826.0 km Route charge differentials 5.8% ATC routing Route selection En-route design Total Performance Review Commission www.eurocontrol.int/prc 30 Flight Efficiency (horizontal): Conclusions Target Performance to date • Horizontal en-route flight efficiency is a major ATM performance issue; route extension (km/flight) 60 • Cost of horizontal en-route route inefficiencies is estimated at 2.2 billion Euro; 50 - 2 km per flight 40 • Significant environmental impact (4.7 million tons of CO2 per annum); 30 20 • Costs increased further due to higher fuel prices in 2006; Direct en route TMA Interface 10 Indicators and trade-offs Proposed target 9.6 10.0 11.2 • Safety and capacity gains require a certain level of “inefficiency” in the route network; • Focus has been on safety and capacity so far; • Need to develop indicators to measure vertical flight efficiency and TMA inefficiencies (airborne holdings); 48.6 46.6 40.6 Target Distance saved (million km) 0 -20 -90 -216 Cost savings (million euro) 0 -100 -450 -1 080 CO2 savings (million tons) 0 -0.2 -1.0 -2.3 0 2004 2005 2006 2007 2008 2009 2010 OUTLOOK 2006 Number of flights (million) Target (km per flight) 2007 2010 Total • Agreed target is to reduce the additional distance flown due to route extension by 2 km per flight each year until 2010 Performance Review Commission www.eurocontrol.int/prc 31 Framework for analysis of ATM performance Political & SocioEconomic perspective SAFETY ECONOMY ENVIRONMENT Gaseous emissions ANS provision costs Strategic costs (buffer) Costs due to suboptimal operations MET costs Airspace users perspective Reg. costs Costeffectiveness Eurocontrol costs Safety ATM/CNS Provision cost Safety management Service provider perspective Runway incursions Airspace events Safety culture (reporting, etc.) Ambient performance affecting factors SECURITY Technical innovations Predictability Flight efficiency Quality of service Efficiency Ground delay Flexibility Cost management Employment costs Support costs Airport/TMA capacity Use of airspace En-route capacity To be developed ATFM/ Network capacity ATCO productivity Traffic (volume, complexity) Capacity management Weather Network effects & fragmentation Political/ Environ. Regulations/ restrictions Performance Review Commission www.eurocontrol.int/prc