E.02.14-CASSIOPEIA-D1.1-High-Level Functional Specification – Model Inputs Document information Project title CASSIOPEIA Project N° E.02.14 Project Manager Fundación Instituto de Investigación Innaxis High-Level Functional Specification – Model Inputs Deliverable Name Deliverable ID 1.1 Edition 01.01.00 Template version 02.00.00 Task contributors University of Westminster; Fundación Instituto de Investigación Innaxis. Abstract This document reports on a survey of ATM stakeholders, executed in order to establish priorities for the selection of future scenarios (SESAR-focused, and more secular) and case studies to be adopted as part of a new ATM performance model. The survey questions and their responses are described in detail. Further consultation with the stakeholder community will inform the development of the model and shape the evolution of a number of supporting case studies. Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 Authoring & Approval Prepared By Name & company Position / Title Date Andrew Cook / University of Westminster 28OCT11 Graham Tanner / University of Westminster Member of the Consortium Board Consortium Member Reviewed By Name & company Position / Title Date Graham Tanner / University of Westminster Consortium Member 31OCT11 Approved By Name & company Position / Title Date David Pérez / Innaxis Project Leader 21NOV11 Document History Edition Date Status Author Justification 01.00.00 31OCT11 Deliverable A. Cook 01.01.00 11NOV11 Deliverable A. Cook New document for review by EUROCONTROL Minor revisions following comments from EUROCONTROL IPR (foreground) This Deliverable consists of Foreground owned by one or several Members or their Affiliates. 2 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 Table of Contents EXECUTIVE SUMMARY ...................................................................................................................................5 1 INTRODUCTION...........................................................................................................................................6 1.1 1.2 1.3 1.4 1.5 BRIEF OVERVIEW OF DOCUMENT ................................................................................................................6 INTENDED READERSHIP ...............................................................................................................................6 OBJECTIVES OF DELIVERABLES 1.1 – 1.3....................................................................................................6 FOCUS OF DELIVERABLE 1.1.......................................................................................................................7 OVERVIEW OF THE STAKEHOLDER CONSULTATION PROCESS ......................................................................7 2 STAKEHOLDER CONSULTATION AND SURVEY IMPLEMENTATION ........................................8 2.1 STAKEHOLDER CONSULTATION ..................................................................................................................8 2.1.1 Overview of the stakeholder groups....................................................................................................8 2.1.2 Sampling and representativeness........................................................................................................9 2.2 STAKEHOLDER PERSPECTIVES...................................................................................................................10 2.2.1 Technical versus non-technical interests ..........................................................................................10 2.2.2 Inclusion of the passenger perspective .............................................................................................11 2.2.3 Inclusion of the industry perspective ................................................................................................13 2.3 SURVEY IMPLEMENTATION .......................................................................................................................14 2.3.1 Timing and duration..........................................................................................................................14 2.3.2 Survey content ...................................................................................................................................14 2.3.3 On-line questionnaire .......................................................................................................................15 2.3.4 Comments on specific questions .......................................................................................................23 2.3.5 On-line development, testing and piloting ........................................................................................24 2.3.6 Mailing of invitations to participate in survey..................................................................................24 3 SURVEY RESULTS AND MODEL INPUTS............................................................................................26 3.1 3.2 3.3 3.4 OVERVIEW OF RESPONSE AND DATA CLEANING ........................................................................................26 RESPONDENT PROFILE...............................................................................................................................26 TECHNICAL SCENARIOS (SESAR-RELATED).............................................................................................30 NON-TECHNICAL SCENARIOS (GENERIC)...................................................................................................31 4 NEXT STEPS AND A LOOK AHEAD TO FUTURE DELIVERABLES..............................................34 4.1 NEXT STEPS...............................................................................................................................................34 4.2 FUTURE DELIVERABLES ............................................................................................................................34 5 REFERENCES..............................................................................................................................................35 3 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 List of tables Table 1. Stakeholder groups.................................................................................................................. 8 Table 2. Comments on specific questions ........................................................................................... 23 Table 3. Data cleaning summary ......................................................................................................... 26 Table 4. Company/institution categories.............................................................................................. 27 Table 5. Re-coded company/institution categories .............................................................................. 27 Table 6. ICAO region in which respondent based ............................................................................... 28 Table 7. Involved in PRB/PRU stakeholder consultation ..................................................................... 28 Table 8. Familiarity with SESAR KPIs ................................................................................................. 29 Table 9. Job function ........................................................................................................................... 29 Table 10. Importance of more information – technical scenarios (SESAR LoCs) ............................... 30 Table 11. Importance of more information – technical scenarios (SESAR LoCs); ANSPs.................. 31 Table 12. Importance of more information – non-technical scenarios (generic) .................................. 32 Table 13. Importance of more information – non-technical scenarios (generic); ANSPs .................... 32 List of figures Figure 1. Stakeholder interest in types of change ............................................................................... 10 4 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 Executive summary This report presents the results of a major stakeholder survey on ATM performance assessment, as part of a Delphi process, to analyse and define the high-level functional requirements of a new ATM performance model. The model aims to provide policy-makers and stakeholders with a toolset capable of delivering meaningful answers to questions of particular interest in the European ATM context. It will evaluate the impact of different technological, operational, economic, or policy options on a number of KPAs/KPIs. The stakeholder consultation and survey implementation processes, including sampling and bias, are discussed in detail. The relative stakeholder interest in exploring ATM performance under two general headings, ‘technical’ (related to SESAR Lines of Change) and ‘non-technical’ (less technology-driven, not focusing on SESAR and including secular changes), is discussed. At the aggregate level, no particular SESAR Line of Change dominates the demand for further performance modelling over and above the others, whilst interest levels in such performance assessment was rated fairly high. Further consultation with the stakeholder community will inform the development of the model and shape the evolution of a number of supporting case studies. 5 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 1 Introduction 1.1 Brief overview of document This document reports on a survey of ATM stakeholders, executed in order to establish priorities for the selection of future scenarios and case studies to be adopted as part of a new ATM performance model. The survey questions and their responses are described in detail. 1.2 Intended readership Based on a cross-sectional survey of 157 ATM stakeholders, this report identifies scenarios whose evaluation is of interest to stakeholders, in the context of developing ATM performance assessment models. These scenarios have been examined separately as ‘technical’ and ‘non-technical’. The report also details the methodology of the stakeholder consultation process. It is written for the professional reader and assumes an understanding of air transport and ATM. Without detriment to appropriate referencing and delineation, the text is not cluttered with explanations of common acronyms or principles. 1.3 Objectives of Deliverables 1.1 – 1.3 The objective of Workpackage 1 is to analyse and define the high-level functional and non-functional requirements of CASSIOPEIA. As described in the proposal, the CASSIOPEIA platform aims to provide policy-makers and stakeholders with a toolset capable of delivering meaningful answers to questions of particular interest in the European ATM context. The platform should be able to evaluate the impact of different technological, operational, economic, or policy options on a number of KPAs/KPIs. The selection of relevant features and the degree of detail to be included in the model is determined, in part, by the type of scenarios to be assessed (model inputs – D1.1) and by the type of answers to be delivered (model outputs – D1.2). The functional requirements will be therefore expressed in terms of these inputs and outputs. The usability of the model, in-built modularity and growth capability, are expressed through a number of non-functional requirements (D1.3). Deliverables 1.1 – 1.3 are thus to be considered as an integrated report on Workpackage 1, although we have reported on the particular aspects identified above, separately. In particular, D1.1 and D1.2 report on the stakeholder survey, with much of the background material on the survey reported in D1.1 (and thus not repeated in D1.2). The survey itself is part of the wider consultation process of WP5. Note. During early project implementation, it was decided to bring forward some aspects of the case study selection and development, with an additional deliverable (D2.0) to draft the formulation of the regulation, ATM network, agent and exogenous factors models. It has further been agreed that D2.0 will also, and jointly with Workpackage 1, further develop the case study selection process and specify the elements that will be parameterised in order to define certain scenarios (the latter task originally planned for D1.1), with these processes in D2.0 to be informed by the stakeholder survey results presented in D1.1 and D1.2. Mostly due to some late mailings by third parties of the respondent invitations to participate in the survey, outside of our control, the time available to report on the survey results was reduced to several days. However, further analyses will be possible on these data going forward in CASSIOPEIA, and the results reported herein form just part of the consultation process and its reporting (see also Section 4). 6 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 1.4 Focus of Deliverable 1.1 The primary objective of this Deliverable is to identify scenarios whose evaluation is of interest to stakeholders, which we have examined separately as ‘technical’ and ‘non-technical’ scenarios. The former are based on the SESAR Lines of Change described in the Master Plan (SESAR Consortium, 2008). The latter are other major themes, not driven by technological advances under SESAR per se, but which may impact on ATM performance; these themes were discussed and short-listed by the project team. 1.5 Overview of the stakeholder consultation process The CASSIOPEIA stakeholder consultation process is what is often termed a ‘Delphi’ survey. Two key characteristics of such an approach is that it relates to the process of collecting opinion from experts, particularly regarding potential future developments around a given issue, and that it is an iterative approach, with one phase building on the results of another. Since, by definition, such respondents have little time for such surveys, we have used an on-line instrument for the first phase, which respondents can complete at a time which best suits them. The second phase of the consultation process will be a stakeholder workshop, hosted at the University of Westminster’s headquarters in January 2012. By design, the vast majority of stakeholders invited to the workshop will have received this invitation at the same time as the survey, and be invited to attend at the end of the on-line questionnaire. This serves two purposes. Firstly, it enables respondents to acquire a more detailed understanding of the scope of the consultation process. By this mechanism, they will better appreciate whether the workshop is something they would be particularly interested in. This allows us to filter out stakeholders from the workshop for whom attendance would be less mutually beneficial. Secondly, it enables us to build up a rapport with certain stakeholders and to hopefully motivate them to attend the workshop to develop the discussion further. This will be enhanced by the fact that the survey results will be presented at the workshop, although not dominating this event, in order to allow an open development of ideas. 7 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 2 Stakeholder consultation and survey implementation 2.1 Stakeholder consultation 2.1.1 Overview of the stakeholder groups † The stakeholder groups shown in Table 1 are adapted from current SESAR definitions . Those which represent a focus area for the project are indicated in the penultimate column. The sampling methods, including issues of bias and inclusion, are discussed in the next section. Table 1. Stakeholder groups Group SESAR stakeholder group* Included in SESAR group Focus of project - scheduled airlines - charter airlines Where not performing a primarily regulatory and/or political role (see group 6), airline associations are also included in this group. Please see Section 2.2.3. - airfreight service providers 1 Airspace users Comments - business and leisure aviation, incl. GA pilots - all other non-military: other leisure (e.g. hot air balloons) & non-leisure (e.g. police, rescue) - military 2 Airport operators 3 ANSPs 4 Suppliers 5 Airline, airport and air traffic navigation staff Several large consortia of airport operators, 1 such as SEAC (expected to include six large 2 European airports), AENA and NORACON , are already SESAR members. SESAR refers both to the more disaggregate level of ATC and more aggregate level of NORACON. Please see Section 2.2.3. In future, we note that this user group may be refined to refer 3 explicitly to the European (regional), sub-regional, and local network managers. For now, our level of engagement is at the ANSP level, and with EUROCONTROL. - aircraft manufacturers - ground and airborne equipment manufacturers SESAR includes this category to capture important human factors issues. - Not within our remit. 1 SEAC, the SESAR European Airports Consortium, comprises six major European airports: Aéroports de Paris, BAA Airports, Flughafen München, Fraport, Schiphol and Flughafen Zürich. 2 NORACON, the NORth European and Austrian CONsortium, consists of eight European ANS providers: Austro Control (Austria) and the North European ANS Providers (NEAP) including AVINOR (Norway), EANS (Estonia), Finavia (Finland), IAA (Ireland), ISAVIA (Iceland), LFV (Sweden) and Naviair (Denmark). 3 See Section 2.2.3 regarding the role of EUROCONROL as “Network Manager”. 8 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Group 6 Passengers and citizens 8 Media, NGOs and pressure groups* 9 Transport infrastructure* 10 Manufacturing and services industry* 12 * Regulatory and political 4 framework 7 11 † SESAR stakeholder group* Research / consultancy* Research funding body* Included in SESAR group Edition: 01.01.00 Focus of project SESAR refers to the environmental, security, safety and competition aspects of regulations, and to European governments. Presumably, the European Commission would be included here. () - passengers: SESAR details how, through improved air traffic navigation, information and positioning, passengers stand to benefit through shorter and more reliable journeys, lower costs and improved safety. Comments This scope could be extended to include local government and land-use planning, and also to all air transport organisations, which perform regulatory and/or political roles in particular, such as IATA an ICAO. Air transport unions also belong to this grouping. National enforcement bodies / units for Regulation 261 are separately identified in the responses to Question 1 (see Section 2.3.3). Whilst SESAR groups passengers and citizens † together , we split these groups for our analysis. We will focus on passengers. See Section 2.2.2 for details. - citizens: SESAR details how society at large, whether or not they use air transport, will also gain from a more competitive European air industry, less noise around airports, more efficient and convenient travel and a contribution to cutting greenhouse gases and reducing climate change impacts. - The complementary groupings and associations of those affiliated to the air transport industry. - Providers of complementary transport (airport connecting transport, ‘flights’ actually operated by rail services) and competitive transport (mostly high-speed rail). - Users of (e.g. shippers) and suppliers to (excluding aircraft manufacturers, manufacturers of ground/ airborne equipment) air transport industry. - Includes SMEs primarily engaged in research / consultancy, and research institutions / universities. These will be primarily included through contact with the ComplexWorld network and ad hoc inclusions in other bespoke lists used in the survey mailing. - This group is shown to acknowledge that such respondents may be included within other categories of respondent listed in Question 1 (see Section 2.3). However, these are not likely to be many in number and it would severely complicate the responses to Question 1 to separate these respondents out from other groups. SESAR stakeholder group source: http://www.sesarju.eu/players/stakeholders Additional groups not specified at ‘†’ are marked with an asterisk. 2.1.2 Sampling and representativeness In market research we often attempt to minimise most types of bias, sampling bias being key among them. In this stakeholder consultation we are more interested in collecting a range of opinions, 4 Described as “regulators and administrations” at: http://www.sesarju.eu/players/stakeholders. 9 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 however. This range of opinions is intended to challenge and extend our thinking regarding the application and use of KPAs in air transport. More specifically to CASSIOPEIA, the objective is to investigate: • which types of change should be evaluated in terms of their impact on performance (as specified for D1.1); • which specific KPAs/KPIs will be explored in the modelling, and at what spatial/temporal scale (as specified for D1.2). We discussed in Section 1 that this was a Delphi approach, eliciting expert stakeholder feedback. If this survey were to be representative of all stakeholders, clearly the passengers and citizens would easily outnumber all the other stakeholders, yet it is undesirable for the feedback to be heavily dominated in this way. However, the passenger context, in particular, is very important, and the specific capture of expert opinion in relation to this perspective is discussed in detail in Section 2.2.2. Within the timescale of the project, and in the context of this survey phase being implemented quite early in the project, we have focused on what may be termed the core airspace stakeholders on the supply and demand side. Important in the context of CASSIOPEIA, we have specific scope to include other stakeholders through the case studies. These may be selected to complement some of the stakeholder groups included in the survey, by furnishing more detailed interaction in some already-included groups and/or specifically developing contacts within new stakeholder groups, not included in this survey. One objective of the project is to explore new KPA modelling, which may mean focusing on areas for which only a specific sub-section of the stakeholders or case study partners are supportive of further research and development. It is important within WP E not only that we focus on new KPA investigations, but also that we understand how the results of this work may be useful to the ATM community. We thus need to avoid adopting case studies that are inadvisable due to satisfactory existing coverage, or due to profound impracticalities due to severe KPA immaturity or the absence of supporting data. 2.2 Stakeholder perspectives 2.2.1 Technical versus non-technical interests Stakeholder Technical Non-technical Types of change Technical Non-technical Figure 1. Stakeholder interest in types of change Figure 1 shows, figuratively, how what we might term ‘non-technical’ stakeholders (such as passengers) will tend to have a much more substantial interest in non-technical types of change (such as increases in emissions taxes on fuel, which may well drive up ticket prices). Their interest in technical types of change (such as UDPP) will be far less prominent (as represented by the peak of the triangle on the right). Some of the ‘non-technical’ stakeholders will not have a depth of expertise in assessing the effects of various types of change on ATM, and the corresponding KPAs best suited to measure performance. However, these effects may be estimated in certain example cases (such as the impact on ticket 10 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 prices of emissions taxes, as mentioned) by either presenting such stakeholders with proxy variables (e.g. seat prices instead of emissions charges) in surveys, or through examination of secondary research. Whilst it will be important to estimate some of these effects in other parts of the project (e.g. the agent-based models of WP2), a passenger survey (or, indeed, a survey of citizens not using air transport) is outside the scope. In contrast, more technical stakeholders, with an active engagement in technical types of change (such as the Lines of Change defined by SESAR: see Section 2.3) will also need to take a strong interest in ‘non-technical’ types of changes (such as emissions taxes). This is represented by the trapezoid on the left-hand side of Figure 1. Its tapering towards ‘non-technical’ types of change is not pronounced. The zone of stakeholder / type of change interaction that we cannot explore in CASSIOPEIA is thus that represented by the top of the triangle on the right-hand side. 2.2.2 Inclusion of the passenger perspective On 17 February 2005, the European Union’s air passenger compensation and assistance scheme (Regulation (EC) No 261/2004) was introduced. In addition to affording passengers additional rights in cases of flight disruption (denied boarding, cancellation and delay), the Regulation also requires airlines to inform passengers of their rights when a flight is disrupted. This includes giving the contact details of a national enforcement body, designated by the member state to receive complaints. To take the examples of the UK and Spain, this body is the corresponding civil aviation authority in each case – the Civil Aviation Authority (CAA) and Dirección General de Aviación Civil (DGAC), respectively. There is no pan-European organisation of the national enforcement bodies. When complaints concern European flights booked and operated outside of the passenger’s country of domicile, the European 5 Consumer Centre Network (ECC-Net) should be contacted. This is a broad-scope consumer protection network, for residents in a dispute with a trader located in a different country to their country of domicile, but also in the European Union. The UK European Consumer Centre is part of the ECC-Net, which consist of 29 centres (one in each EU country plus Iceland and Norway). Some of these sites have links to pages offering support with air travel (in the UK it is the first link), others do not. The optimum inclusion of the passenger perspective in this project is to be achieved through expert interface with the national enforcement bodies, rather than through passenger interviews. Given the bases of the University of Westminster and Innaxis, we have liaised individually with the national 6 enforcement bodies for the UK and Spain. We have approached the UK CAA and the Agencia Estatal de Seguridad Aérea (State Aviation Safety Agency, the DGAC agency responsible for this function in Spain), and invited their participation in the survey and the January 2012 workshop. For other countries, we have sent invitations to the corresponding national contact points, where these email addresses were listed on the European Commission’s website. Plans are currently underway at the Commission to review Regulation 261, regarding improved enforcement and clarification of responsibilities. A new legislative proposal is planned for 2012, looking at key issues such as: […] limitation of liability in cases of exceptional circumstances, compensation thresholds, as well as re-routing and sharing the burden of risk. European Commission (2011a) 5 Each state has its own website. In the UK it is: http://www.ukecc.net Later, this liaison may move to a dialogue with the Aviation Consumer Advocacy Panel. This should be up and running by mid-2012. It is a new body in the UK, part of the Civil Aviation Authority, undertaking consumer representation and replacing the Air Transport Users Council (AUC). The Panel, and thus the former AUC’s complaints handling role, will become part of the CAA’s newly created Regulatory Policy Group (which itself replaces the old Economic Regulation Group). The RPG contains a new sub-group focused on consumer enforcement and market issues, in addition to collecting and analysing aviation statistics and survey responses, as two of its four key functions. 6 11 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 The review has been prompted by the experience of the ash cloud and severe snow in 2010. It may be that governments will take on some of the burden during future events similar to the ash cloud, and airports in the case of uncleared snow, for example. A public consultation will also consider: […] commercial practices linked to rescheduling (passenger rights in cases of flight rescheduling) and no-shows (clarifying the situation for airlines in the case of connecting flights where there are "no show" passengers). European Commission (2011a) In the short term, the Commission plans to improve pan-European enforcement of passenger rights, including a strengthening of the mandate of the network of national enforcement bodies (NEBs) to adopt decisions on a common interpretation, and regarding enforcement. This reflects a wider European policy to consolidate coherent enforcement across all modes of transport, with increased commonality (Regulation (EC) No 1371/2007, for example, offers rail passengers improved information, assistance in the event of delays of more than one hour, and compensation). The Commission's new roadmap (White Paper) to a Single European Transport Area for 2050, opens as follows: The EU has already established a comprehensive set of passengers’ rights which will be further consolidated. Following the ash cloud crisis and the experience of extreme weather events in 2010, it has become evident that Mobility Continuity Plans may be required to preserve the mobility of passengers and goods in a crisis situation. These events also demonstrated the need for the increased resilience of the transport system through scenario development and disaster planning. European Commission (2011b) A complementary, high-level document (Flightpath 2050 - Europe’s Vision for Aviation) from the European Commission, addressing European transportation, specifically aviation, cites resilience both in its foreword and many times in the main document, as well as emphasising the importance of integration: The strategy addresses customer orientation and market needs as well as industrial competitiveness and the need to maintain an adequate skills and research infrastructure base in Europe. By 2050, passengers and freight should enjoy efficient and seamless travel services, based on a resilient air transport system thoroughly integrated with other transport modes and well connected to the rest of the world. This will be necessary in order to meet the growing demand for travel and to cope more easily with unforeseeable events. European Commission (2011c) Finally, in the context of the above changes and drives to improve passenger legislation, the Commission has identified a problem regarding the lack of supporting data, for example regarding the number of flights within the scope of Regulation 261: In particular, the assessment, on many issues, has shown a critical lack of data on which to base analysis. The provision of data from industry and national authorities as well as consumer groups is a central underpinning proportionate and good regulation. This will be an important issue for the consultation and subsequent revision. European Commission (2011a) To summarise, what are the important lessons for CASSIOPEIA, specifically regarding the stakeholder consultation, and more generally for the project? As in the project: • the passenger perspective is becoming increasingly important in the evolving European policy context • resilience and scenario modelling are also considered important by the Commission • complementary transport modes are meaningful considerations • the importance of supporting data is to be considered 12 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 2.2.3 Inclusion of the industry perspective 7 Implementing a European Network Manager was an important step foreseen in the second package of measures of the Single European Sky (European Commission, 2010). In February 2011, the Single Sky Committee agreed the Implementing Rule on network functions and voted in favour of EUROCONTROL being nominated as the SES Network Manager (EUROCONTROL, 2011a). The successful completion of the nomination process was announced in September 2011 (EUROCONTROL, 2011b).The Network Manager is the body in charge of managing ATM functions centralised at a European level, such as the design of the European route network and the air traffic flow management function (currently executed by CFMU). As described in the corresponding Regulation (EU) No 677/2011 (Official Journal of the European Union, 2011), the Network Manager will: […] detail the Network Strategy Plan through a Network Operations Plan, as further specified in Article 6, addressing in particular European Union-wide performance targets covering 3 to 5 year, annual, seasonal, weekly and daily periods The Network Operations Plan, the Regulation further details (ibid.), shall be based on a structure that includes: Consolidated forecast and analysis of the operational performance of the network • network, air navigation service provider, functional airspace block and ACC ATM delay/capacity targets and forecast, • airport operational performance, • network environment/flight efficiency performance target and forecast, • impact of special events, • analysis of the operational performance targets and forecast. and will be tailored to the various individual functions and to the time horizons identified above. This thus underlines the important point that the Network Manager is responsible for both delivering and monitoring performance with respect to the targets. Also important for CASSIOPEIA, is the particular connection, flagged by EUROCONTROL’s Performance Review Unit, between delay propagation and the Network Manager function: Reactionary delays are by definition a network issue and a better understanding of the contribution of airports, airlines and ANS towards those network effects and possible measured to mitigate those effects would be desirable, particularly with a view to the network manager that will be established under the SES II initiative. However such a study is complex as it requires linking the individual legs of aircraft (i.e. linking gate-to-gate and turn around phases of aircraft) at European scale. EUROCONTROL (2011a) 2.2.3.1 Airspace users and airports In addition to survey invitations sent to the University of Westminster’s airline (and ANSP) working group on delay cost management, plus our mailings to passenger’s rights National Enforcement bodies (see Section 2.2.2), plus ComplexWorld members and participants, and extensive work on EUROCONTROL and University of Westminster proprietary lists (with airline, ANSP, airport, regulator and local government contacts), we would like to express our deep gratitude to the following for their help in promoting the survey and/or workshop (in the majority of cases, both) to their members, with some particularly strong and helpful support received from certain associations (list in alphabetical order): 7 Note that we capitalise “Network Manager” to indicate that we are referring to the European Network Manager, EUROCONTROL (in ICAO terminology, the “Regional” Manager – Europe (and the North Atlantic) is a region), thus differentiating this from lower-level roles such as the sub-regional (FAB), and local network managers (ACC, or group of ACCs). 13 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs • • • • • • • • • • • • Edition: 01.01.00 ACI Europe AGIFORS Association for the Scientific Development of ATM in Europe (ASDA) Association of European Airlines CANSO EUROCONTROL (selected Organisation members, arranged by EEC) European Regions Airline Association International Air Carrier Association NATS UBM Aviation news (e-newsletter distribution) UK Aerospace, Aviation & Defence Knowledge Transfer Network UK Sustainable ATM National Technical Committee 2.3 Survey implementation 2.3.1 Timing and duration Important in the context of any survey, is that the respondent is not asked to complete a survey which is too long, as this may reduce response/completion rates and/or respondent compliance (they may become bored and/or tired). This sets a natural constraint on the amount of material that can be covered in the survey. As noted, this survey in any case forms part of a wider consultation process, for example the development of certain issues in the January 2012 workshop. To encourage the respondents to bear with the survey, a status bar indicated the approximate percentage completion at the foot of each page. It is worthwhile being candid with the respondent at the outset regarding the length of time the survey takes to complete (determined during test runs) and to stress the value of their cooperation, again to encourage engagement. It is also important that a reasonably narrow and specific time window for survey completion is given. If the completion date is set too far in the future, this increases the likelihood of the respondent postponing completion, and not returning to it. The first survey invitations were sent out on 30SEP11. The survey closure date was initially set at 19OCT11, in order to allow time for the initial analysis of the corresponding data for D1.1 and D1.2, but had to be extended to 21OCT11 due to some late invitation mailings via third parties. 2.3.2 Survey content The content of the survey, and its development, is described in the following sections. It was important to the objectives of the consultation, as described, to include a range of stakeholder views and not to introduce ‘pressure bias’ into the survey, whereby those with less expertise were discouraged from answering. (Indeed, Table 8 suggests a desirable cross-section was achieved). The questionnaire focuses on the use and importance of KPAs, etc, to the respondent, and not some abstract benefit to ‘ATM’ in general, which has a far less meaningful interpretation. It is a common error in surveys to ask respondents about ‘importance’ of a product, or service, or of certain information, but without setting that ‘importance’ in any appropriate context. 14 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 2.3.3 On-line questionnaire Key Performance Indicators in Air Transport Stakeholder Survey Most people don’t read the introduction page of a survey, but please spare half a minute to read this! Thank you. We are inviting a wide cross-section of stakeholders to complete this survey. We are hoping to collect a range of views from all types of stakeholder. The survey takes about 20 - 25 minutes to complete. Whilst this is a significant amount of your time, we have spent several months designing the survey and ask only questions that we will use. The results will be used to identify priorities for new Key Performance Indicators (KPIs), which we will design and test over the next 18 months, intended to benefit stakeholders in an operationally meaningful way, for example in delay management. As far as possible, we would like your responses to represent your considered opinion of the views and interests of your company or institution, rather than your personal views. All questions refer to Key Performance Areas (KPAs) and KPIs in the European context, although we welcome views from outside Europe. The survey is anonymous. Our sincere thanks for your valuable time The University of Westminster, London The Innaxis Foundation & Research Institute, Madrid Question content Question no. 1 [Responses are radio buttons, unless specified otherwise] At which type of company/institution are you based? Airport operator Airport-supporting organisation Air navigation service provider (ANSP) ANSP-supporting organisation Full-service carrier Low-cost carrier Charter airline Military, air force Other aircraft operator or airline Airline-supporting organisation Aircraft manufacturer Ground/airborne equipment manufacturer National civil aviation authority / regulator (excluding Reg 261 enforcement unit) National enforcement body/unit for Regulation 261 National / local government (including agencies) EUROCONTROL European Commission Researcher / consultant Other (please specify)[text box] [drop-down list] 2 In which ICAO region are you based? Africa-Indian Ocean (AFI) Region Asia (ASIA) Region Caribbean (CAR) Region European (EUR) Region Middle East (MID) Region North American (NAM) Region North Atlantic (NAT) Region Pacific (PAC) Region South American (SAM) Region 3 Please enter your group ID code here.[text box] Each collective group of respondents was sent a group ID code in the e-mail inviting participation in this survey. This will not be used to identify respondents, only to track response rates from each group. 15 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs 4 Edition: 01.01.00 Have you been involved in the stakeholder consultation process this year with the Performance Review Body / Performance Review Unit (EUROCONTROL, Brussels) regarding Reference Period 1 and/or 2? Yes No Not sure 5 We hope to interview a range of people with varying degrees of familiarity with Key Performance Indicators (KPIs). How would you describe your familiarity at this time, with the SESAR KPIs for the eleven ICAO Key Performance Areas (KPAs)? I am not currently familiar with the KPAs and do not refer to related documentation I am not currently familiar with the KPAs but I do refer to related documentation I could quote several KPAs without looking them up, but no specific KPI targets I could quote several specific KPI targets without looking them up I could quote most specific KPI targets without looking them up I am very familiar with the KPIs and their on-going development [Please select the option as far down the list as applies to you] 6 How useful to your company/institution are the following types of KPA? If you have any additional comments for any of these KPAs, please use the corresponding text boxes. Key Performance Area [Pop-up text:] Click on each KPA for further details. These texts refer to SES existing suggestions and adopted targets. Usefulness [“not at all”, “not very”, “somewhat”, “very”, “extremely”; “unclear”] Capacity [text box] Daily and annual IFR flights in Europe; declared airport capacities; annual average en-route ATFM delay; airport ATFM delays; additional time in taxi-out phase; additional time in arrival sequencing and metering area (ASMA). Cost-effectiveness [text box] En-route and terminal ANS rates in Europe. Efficiency [text box] Scheduled flights departing on time (as planned), average delay of remaining scheduled flights; flights with block-to-block time as planned, average block-to-block time extension of remaining flights; flights with fuel consumption as planned, average additional fuel consumption of remaining flights. Predictability [text box] Flights arriving on time (as planned), average arrival delay of remaining flights; coefficient of variation for actual block-to-block times for repeatedly flown routes; total reactionary delay; reactionary flight cancellation rate; total service disruption delay; percentage of diversions caused by service disruption. 16 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 Flexibility [text box] Accommodation of VFR-IFR change requests; unscheduled flights departing on time (as requested), average delay of remaining unscheduled flights; scheduled flights with departure time as requested (after change request), average delay of remaining scheduled flights. Environment [text box] Average fuel savings per flight as result of ATM improvements; average CO2 emission per flight as result of ATM improvements; reduction of horizontal en-route flight extension; effective use of civil/military airspace structures; compliance with local environmental rules; number of proposed environmentally-related ATM constraints subjected to transparent assessment with environmental and socio-economic scope. Safety [text box] Annual European-wide absolute number of ATM induced accidents and serious or risk-bearing incidents; effectiveness of safety management (‘maturity’); applying severity classifications to allow harmonised reporting of severity assessment for: separation minima infringements, runway incursions, ATM special technical events. Reporting of Just Culture. Security [text box] This refers to protection against direct/indirect threats, attacks/interference to ATM system. There are no quantified KPIs yet developed. Access & equity [text box] This refers to the right of access for all users to necessary ATM resources to fulfil requirements. There are no quantified KPIs yet developed. Participation [text box] This refers to keeping all stakeholders involved in ATM development. There are no quantified KPIs yet developed. Interoperability [text box] This refers to maintaining global standards and uniform principles. There are no quantified KPIs yet developed. 17 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs 7 Edition: 01.01.00 For such KPAs, over what geographical scale do you consider it would be useful to your company/institution for them to be reported upon? An example might be delays. (Each level assumes the ones lower in the list would also be reported upon). By specific airports By airport regions (e.g. ‘all London’) By specific routes (airport-to-airport) By general routes (e.g. Scandinavia to Mediterranean) By country By groups of country (e.g. FABs in future) At the European level Difficult to generalise across KPAs Any comments on specific KPAs, or other remarks [text box] 8 For such KPAs, at what level do you consider it would be useful to your company/institution for them to be reported upon? An example might be delays. (Each level assumes the ones lower in the list would also be reported upon). By the hour By day By week By month By summer/winter timetable season By year Difficult to generalise across KPAs Any comments on specific KPAs, or other remarks [text box] 9 … and by flight / airline level of detail? Please tick all boxes for which reporting would be useful. By each phase of flight (including turnaround) By individual flights By individual airlines By low-cost and full-service airlines By type of haul Difficult to generalise across KPAs Any comments on specific KPAs, or other remarks [text box] 10 Please outline briefly how KPAs / KPIs are relevant to / used in your line of work. [text box] 18 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs 11 Edition: 01.01.00 Of the changes listed below, for which ones do you feel you need more data/information, on how these changes will affect future ATM performance? We are specifically talking about the importance to your company/institution of getting more data (e.g. ‘this change will reduce average flight times by 5%’). Take the first example, SWIM. If you already have enough data about SWIM, or feel that its impact on ATM performance does not affect you anyway, getting more data would be of low importance to you (so you might select “1” or “2”). If you think that SWIM will have a significant impact on ATM performance and you do not have enough data about this, getting more data would be important to you (so you might select “4” or “5”). Type of change [Pop-up text:] Click on each type of change for further details. (i) Importance of getting more data [1-5; 5 = high] [U = unclear] System-wide information management (SWIM) Underpins collaborative planning. Enabler of end-user applications (EUAs); not itself an EUA. Providing relevant information to all parties. Removing constraints on implementation of EUAs, otherwise imposed by full deployment requirements providing benefits earlier. Creating conditions for advanced EUAs by finding most appropriate source of information. Migrating simpler EUAs first; not dependent on deployment of ATM changes. Benefits available even in largely legacy environment. (ii) Moving from airspace to trajectory based operations Deploying dynamic collaborative management of airspace/route structures. Environmentally friendly procedures (e.g. steep and curved approaches, supported by approach procedures with vertical guidance (APV)). Formalisation and modelling of traffic demand and capacity balancing (DCB). Advanced RNP1 in terminal airspace (reduced spacing between routes, tactical parallel offset as alternative to vectoring). Advanced RNP1 SIDs and STARs, Advanced CDAs. Free routing cruise in upper airspace (enabled by FABs). (iii) Collaborative planning using Network Operations Planner Use of ATFM slot swapping. Coordination between more than one ACC, airport operations and CFMU. Manual user-driven prioritization process (UDPP), then implementing the network management function to assist airspace users in the UDPP process. Publication of Shared Business Trajectories by airspace users. Reference Business Trajectories agreed through collaborative flight planning. (iv) Managing the ATM network Network performance assessment; ATFCM scenarios; management of critical events. Improved operations at airport in adverse conditions using ATFCM techniques. Implement interactive Network Operation Plan providing an overview of the ATFCM situation from strategic planning to real-time operations, available online to stakeholders for consultation/update. Analyse climate factors impact on route planning. Managing the Business Trajectory in real time (v) Automated support for traffic load management. En-route controller-pilot comms complemented by datalink. Optimised en-route cruise-climb setting between pilot and controllers. Automated support to predict congestion and support dynamic airspace management. Automated support for ATCOs to de-conflict or synchronise trajectories. Automated ATCO support for multi-sector planning. Assessing costs and benefits of RBT process. Define responsibilities between regional and sub-regional/local network management functions. 19 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs (vi) Edition: 01.01.00 Collaborative ground and airborne decision making tools Automated support for near-term conflict detection & resolution, and trajectory conformance monitoring. In TMA, introduction of ground system route allocation tools to assist controller in managing potentially large number of interacting routes; automatic selection of optimum conflict-free route when triggered by a specific event. Deploy ground system situation monitoring to assist controller in detecting and assessing impact of deviations from clearances. (vii) Queue management tools DMAN synchronised with pre-departure sequencing. AMAN extended to en-route airspace. AMAN/DMAN integration, including assistance to controller within TMA to manage mixed-mode runway operations, and to resolve complex interacting traffic. Integrate surface management with AMAN and DMAN. DMAN in the queue management process. AMAN and DMAN from multiple airports. CTA optimisation through use of datalink. Coordination with (airport-) CDM. (viii) New separation modes Deploying cockpit airborne traffic situation awareness (ATSAW) on-ground & airborne (incl. Oceanic in-trail procedures). Deploy airborne separation assistance system (ASAS), manually controlled arrival sequencing & merging (applicable TMAs) requiring pilot to follow speed commands manually; analyse runway throughput impact of relative (ASAS) versus absolute (RTA) time-based separation. Deploy precision trajectory clearances in two dimensions based on pre-defined 2D routes (i.e. absolute time-based separation). (ix) Independent cooperative ground and airborne safety nets Enhanced airborne collision avoidance system (ACAS): linked to autopilot / flight director display. Short-term conflict alert (STCA) using enriched surveillance information: improve ground-based safety net performance using widely shared aircraft position and intent data. Display ACAS resolution advisories (RAs) to controllers: introduce RA downlink informing controllers automatically when ACAS generates an RA. (x) Airport throughput, safety and environment Improved turnaround through CDM; aircraft fuel and emissions management – airports and en-route. Improved LVPs; deploying final approaches with vertical guidance procedures (to enable Cat-I-like ops). Introduce global navigation satellite system (GNSS) / ground-based augmentation system (GBAS) for precision approaches. Enhanced visual separation on approach. Discontinue wake vortex minima under certain crosswind conditions. Dynamic surface navigation for aircraft and conflicts resolution (automated assistance to ATCO). We are extremely grateful for your cooperation. Two further, (simpler) grid questions follow, before the survey is complete. 12 The response format to this question is the same as the previous question. Of the changes listed below, for which ones do you feel you need more data/information, on how these changes will affect future ATM performance? We are specifically talking about the importance to your company/institution of getting more data. These types of change are more general than the previous set. Type of change Please consider these as very much open in future scope, i.e. not only based on current plans and research. Importance of getting more data [1-5; 20 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 5 = high] [U = unclear] a Environmental regulations and taxes (for airlines and airports) b Patterns of European traffic flows & route growth; market maturity & demand saturation c Airline ability to reaccommodate passengers with missed connections d Flight prioritisation rules and flight management (e.g. based on delay costs) e ATFM slot trading / swapping / market-based (peak) pricing f Airline alliance structures and cooperation models g Aircraft size and design h Increased airport regionalisation (passengers, cargo) i Competition from / cooperation with high-speed rail j Dynamic adaption of airport runway configurations k Your own suggestion[text box] (with rating …) l Your own suggestion[text box] (with rating …) 13 How useful to your company/institution are the following types of KPI / ways of measuring performance? Please respond in terms of usefulness, without being concerned about how practical / easy some of these may be to collect at the moment. a Average delay of delayed flights b % of flights delayed by more than a certain amount of time (long delays, e.g. 60 minutes) c Average delay of delayed passengers d % of passengers delayed by more than a certain amount of time (long delays, e.g. 60 minutes) e Passenger arrival delay at final destination (compared with flight delay by segment) f Measures of delay propagation (‘spreading’ between flights) g Number of cancelled flights h Number of passengers: with missed connections; with cancelled flights i Delays measured in minutes j Delays measured in costs k Measures of delay equity (e.g. are all airlines / airspace areas, affected to the same extent?) l Your own suggestion[text box] (with rating …) Usefulness [“not at all”, “not very”, “somewhat”, “very”, “extremely”; “unclear”] 21 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs m Q14 Edition: 01.01.00 Your own suggestion[text box] (with rating…) Please indicate which of the following best describes your job function. Senior manager, not directly active in ATM/flight operations Other role, not directly active in ATM/flight operations Senior manager, directly active in ATM/flight operations Other role, directly active in ATM/flight operations Research / consultancy Other (please specify if none of above approximately describes your job function)[text box] Q15 Is there anything further, especially regarding KPAs, that you would like to mention? [text box] Our sincere thanks for your valuable time, which we genuinely appreciate. After you click on ‘Finish’, there is a final screen giving you an opportunity to engage in, or just hear more about, our on-going ATM performance research. This could take a few seconds to appear, please do not click ‘Finish’ twice. Thank you. (Q16, although not marked as in web survey) Please select what may be of interest to you below, with no commitments at this stage. Workshop On 10 January 2012, we will be holding a (free) workshop in central London to discuss and develop these results. We will hold open discussions about ATM performance and how its measurement might be developed in the future, with presentations from on-going SESAR JU projects in this area. Fuller details of the programme for the day will be available soon. Please select this box if you are interested in attending. [check box] Case studies Below is a selection of case studies which we are considering. Please select any which you would like us to contact you further about (regarding participation or just being sent information, for those we develop). Newly designed medium/short-range aircraft, flying lower/slower in order to save fuel / reduce CO2 Local environmental restrictions limiting airport and terminal airspace capacity Changes in delays under scenarios of price bidding for FLs/ slot times, by airlines to ANSPs Investigating CDM performance variations with different levels of information sharing ATFM slot allocations as a tradable commodity (airport to airline, or airline to airline) Tactical, dynamic exchange of departure slots between airlines Peak and off-peak pricing in air navigation charges and slots The use of biofuels giving airlines ATC/slot preferences and/or price differentials Different ATFM prioritisation strategies / algorithms Technological improvements reducing uncertainty in 4D trajectories - network level impact FMS with a Cost Index that includes environmental and variable slot/ATC costs Turbofans replaced by propfan engines, with different trajectory optimisations How the timing of sharing actual aircraft operating details influences 4D trajectory planning and execution, and system performance [check boxes] Further information / reports To be sent to you every few months by e-mail 22 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 [check box] After you have selected the appropriate check-boxes above, please enter your e-mail address in the box below and click ‘submit’. [text box] Important! This e-mail address will not be stored with your survey reply and cannot be used to identify you in the survey. It will only be used for the purposes stated above. 2.3.4 Comments on specific questions Table 2. Comments on specific questions Question(s) Comments 1, 2, 3 Data from these questions will allow us to carry out disaggregate analyses of the data. It is important that respondents are assured that their responses are anonymous. Question 2 is simply to differentiate which respondents are from Europe (by design, the vast majority, and the focus of our work). 2 There is a lack of consistent reporting on the definition and number of ICAO regions. This list was compiled from various ICAO sources, several of which refer to nine ICAO regions. 4 This question is asked to flag any potential bias resulting from respondent participation in this parallel consultation process. 5 We may consider responses elsewhere by levels of familiarity / expertise, expressed here. The phrasing is designed to make respondents feel comfortable expressing non-expert status. 6 These pop-up texts were developed largely from the SESAR Master Plan (SESAR Consortium, 2008) and Performance Review Report 2010 (EUROCONTROL, 2011a), and includes the EU-wide performance targets for the first reference period of the Single European Sky performance scheme. 8 Note that the OJEU, with reference to performance targets, sets forward: “ […] detail the Network Strategy Plan through a Network Operations Plan, as further specified in Article 6, addressing in particular European Union-wide performance targets covering 3 to 5 year, annual, seasonal, weekly and daily periods” Official Journal of the European Union (2011). 10 This question may capture unexpected uses by stakeholders types, or identify lack of such use. 11 This question is more complicated than the others and would not be employed in a survey of the general public. It conflates two responses into one ‘importance’ scale, based on the perceived impact on ATM performance and whether the respondent considers that there is already sufficient data/information. If either no impact is perceived, or sufficient data is considered to be available, the need for further information is low. This conflation technique obviates the need for two response scales for each element in the list of changes. We use the word “data” intentionally instead of “modelling”, as the latter may have negative connotations for some respondents, and the survey is not about seeking views on the value of modelling ATM performance. Options (i) to (x) are based on the Master Plan (ibid.) LoCs 1 to 10, respectively. We have focused on the earlier Service Levels (SL0 to SL2 ), as we can’t capture all of them here. The earlier SLs are generally reasonably tangible (nearer-term) to the respondents/stakeholders. An attempt has been made to capture, in particular, those LoCs that might be more useful/interesting to model. (We note that SWIM as a methodology of sharing information can apply to all ATM capability and service levels. The Capability Level may relate to an extension of geographical/spatial availability, although different ATM SLs may equally need more advanced and/or widespread implementation of SWIM.) The pop-up texts (developed from the Master Plan texts) are only visible when deployed by the respondent, to avoid clutter on the page (which therefore looks a lot simpler on-line than it does in this document). These texts have been edited down, over several iterations, to try to give unbiased and neutral summaries of the main LoC components, and such that each box contains an approximately similar number (50 – 65) of words (to reduce the appearance of greater importance 23 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Question(s) Edition: 01.01.00 Comments for those with longer entries). It is vital that the text “5=high” is clear to the respondents, as different cultures will use these scales in different ways, unless explicit instruction is given. 12 A number of these elements embrace a range of potential considerations. For example, element ‘b’ could include domestic infrastructural capacity to accommodate inbound tourism, and element ‘c’ could include scenarios such as dynamic scheduling of aircraft and changes in load factors. For simplicity and speed of completion, these are not explicitly presented to the respondent, since this question mainly serves to obtain an initial understanding of the perceived relative importance of these generic areas. 13 This question needs to strike a difficult balance between being too general and too specific. Since measures of delay and others types of disruption are central to the project, we have focused on such examples here, although the list could obviously have been very long. The idea is to compare the relative, perceived usefulness of these metrics within and across the stakeholder groups. To reduce the extent to which certain respondents might indicate that they are all pretty useful, or, not particularly useful (known as the ‘halo’ effect, or ‘straight-lining’), strong anchors have been used (at the scale extremes). By this point in the survey, it is hoped that respondents might be particularly ready to suggest (types of) metric of their own. 15 It is important to give respondents an opportunity to express ideas / concerns not covered by the (limited number of) questions in the questionnaire. The frequencies of spontaneously given comments (e.g. possibly relating to lack of appropriate data) often affords valuable insights. 16 The survey data are saved first. This page operates separately so that the survey data are not lost if the respondent just closes this form. As a back-up, the workshop invitation is also included in the email inviting the respondent to complete the survey. 2.3.5 On-line development, testing and piloting Following early tests of some of the publicly available on-line survey hosts, the decision was taken to design and host a bespoke on-line survey on a university server, thereby reducing the constraints imposed by standard question layouts. Care was taken to ensure the resulting questionnaire was compatible with multiple browsers and operating systems, and that the layout and features appeared consistent. During the development stage, the Project Team and a number of interested parties completed a series of pilots / test runs on different versions of the following browsers: Internet Explorer, Firefox, Opera, Chrome and Safari; running on various releases of Windows, Mac OS and Linux operating systems. Examples of problems identified through testing included ‘pop-up’ information boxes not being displayed correctly in older versions of Internet Explorer, and the questionnaire ‘back’ and ‘next’ buttons not working whenever the JavaScript option was disabled. The ‘pop-up’ problem was rectified by using an alternative technique and JavaScript detection redirected respondents, when necessary, to a help page with step-by-step instructions showing how to re-enable it. For simplicity, responses were saved to text files. Tests were carried out to check that responses were saved accurately, that long text strings (including those with line feeds) were not curtailed and that the server could handle simultaneous response submissions. Daily back-ups of the response files were made for the duration of the survey. 2.3.6 Mailing of invitations to participate in survey The distribution lists used for the survey mailings were as described in Section 2.2.3.1. To maximise response rates, individualised messages were sent to each respondent group, although this was highly time consuming. Responses to each group were also tracked by means of collective ID codes (see Question3, Section 2.3.3). Whilst most groups were sent reminder messages after the initial invitations, those with no 24 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 responses after a week or so were particularly targeted for such remedial action. The survey was live from 30SEP11 to 21OCT11, as mentioned above. 25 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 3 Survey results and model inputs 3.1 Overview of response and data cleaning Responses were downloaded from the host server each day, in order to mitigate data loss in the event of a server malfunction or data corruption, and to monitor any other problems – none were observed. Due to the degree of control regarding the data input obtained through the coding and structure of the on-line survey, relatively little data cleaning and logic checking were required. No range errors were detected. Line feeds were removed from free text responses, recorded when respondents inserted new lines whilst writing within text boxes, prior to importing the data into the analysis package SPSS. A total of 170 responses were received for the main survey, and 126 showed an interest in the forthcoming workshop and/or the potential case studies (stored as a separate file for anonymity purposes). After data cleaning, which included the removal of blank data files (presumably due to respondents clicking through the survey in inspection mode) and duplicate entries (due to respondents not following the request not to double-click the final ‘submit’ button), the raw 170 main survey responses received furnished a clean data set of 157 responses, and 107 after similar cleaning in regard to the workshop and case studies data file (discussed in Deliverable 1.2). Table 3. Data cleaning summary Total responses Blank responses Duplicate responses Total responses after cleaning Stakeholder survey 170 7 6 157 Workshop and case studies 126 15 4 107 Dataset We should take some objective note that, considering the complex and specialist nature of the survey, this is an extremely satisfactory outcome. All supplementary comments relating to the survey and its objectives were also positive and supportive (with the exception of comments received from one airline questioning the sampling method, to which the University of Westminster offered a detailed reply). The nature of the free responses given to the open-ended questions also suggests a generally positive engagement with the survey process). 3.2 Respondent profile The raw respondent categories were re-coded into the new groupings indicated. The re-coding is to allow the breakdown of the responses in subsequent analyses into groupings that are not too sparsely populated, with no intention of suggesting any diminished value of the smaller groups. Some of the response categories were there to identify certain respondents in case they were included in the mailings not directly under our control (see Section 2.2.3.1), even though they were not a target response group. Despite the pleasing overall response to the survey, there has been an inevitable aggregation of some categories into slightly heterogeneous, notable the “collective” group. 26 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 Table 4. Company/institution categories Company / institution category Frequency Re-coded Airport operator 9 Airport Airport-supporting organisation 1 Airport 56 ANSP 6 ANSP Air navigation service provider (ANSP) ANSP-supporting organisation Full-service carrier 16 AO pax Low-cost carrier 4 AO pax Charter airline 1 AO pax Military, air force 4 AO non-pax Other aircraft operator or airline 6 AO non-pax Airline-supporting organisation 4 Collective Aircraft manufacturer 1 Collective Ground/airborne equipment manufacturer 0 N/A National civil aviation authority / regulator (excluding Reg 261 enforcement unit) 3 Collective National enforcement body/unit for Regulation 261 4 Collective National / local government (including agencies) 0 N/A EUROCONTROL 9 EUROCONTROL European Commission 0 N/A Researcher / consultant 7 Research 5 [see text] 21 Missing Other (please specify) Missing Total 157 This re-coding, to be used as a common basis for the rest of the analyses, does not prevent us from identifying groups into the raw response coding where this is useful. It remains somewhat of a disappointment that more NEBs did not respond, but the other key stakeholder groups have furnished a useful response. The responses are clearly strongly represented by ANSPs, not through an intentional over-sampling, and the reader is reminded that we have not controlled for more than one response from any given ANSP (due to the anonymity and the fact that attempting to collect a consolidated view from any given institution would have taken far more time than was available). Four of the five responses originally coded as ‘other’ fitted into one of the new categories (two into ‘research’, two into ‘collective). The remaining case was assigned to ‘missing’. Table 5. Re-coded company/institution categories Re-coded category Frequency AO pax 21 AO non-pax 10 ANSP 62 Airport 10 EUROCONTROL 9 Collective 14 Research 9 Missing 22 27 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Total Edition: 01.01.00 157 Table 6. ICAO region in which respondent based ICAO region Frequency Africa-Indian Ocean 1 Asia 3 Caribbean 0 European 135 Middle East 3 North American 4 North Atlantic 2 Pacific 5 South American 2 Missing 2 Total 157 As expected, the majority of respondents were from Europe. Just over half of the respondents were not involved in the PRB/PRU consultation (to be reported upon in future CASSIOPEIA deliverables, including our liaison therewith), thus representing a new respondent input to this WPE process and demonstrating the existing good level of coverage in the PRB/PRU consultation. All the respondents (with one exception) involved in the latter consultation were based in the European ICAO region, as would be expected. Table 8 shows a fairly even distribution of declared KPI expertise across the respondents, which will be a useful mechanism for analysing subsequent responses in this survey reporting. Table 7. Involved in PRB/PRU stakeholder consultation Involved in PRB/PRU stakeholder consultation Percent Yes 35% No 56% Unsure 8% Missing 1% Total 100% 28 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 Table 8. Familiarity with SESAR KPIs Familiarity with SESAR KPIs Percent I am not currently familiar with the KPAs and do not refer to related documentation 26% I am not currently familiar with the KPAs but I do refer to related documentation 18% I could quote several KPAs without looking them up, but no specific KPI targets 20% I could quote several specific KPI targets without looking them up 14% I could quote most specific KPI targets without looking them up I am very familiar with the KPIs and their on-going development Missing 6% 16% 1% Total 100% Table 9. Job function Job function (with eleven ‘other’ re-codes) Frequency Senior manager, not directly active in ATM/flight operations 24 Other role, not directly active in ATM/flight operations 24 Senior manager, directly active in ATM/flight operations 46 Other role, directly active in ATM/flight operations 38 Research/ consultancy 21 Other 0 Missing 4 Total 157 In the original data, eleven responses to job function were coded as ‘other’, although all eleven respondents gave further information regarding their job function (again indicating a good level of respondent engagement in the survey), such that it was possibly to assign them to one of the set categories (sometimes with reference to their other responses). Of the nine respondents in the ‘research (/consultancy)’ category for company/institution in Table 5, eight were classified under ‘research/consultancy’ under job function, and one as ‘senior manager, directly active in ATM/flight operations’, such that these responses were mutually consistent. The distribution of job functions also indicates a diverse respondent base. 29 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 3.3 Technical scenarios (SESAR-related) The development and presentation to stakeholders of the SESAR technical scenarios, each representing one of the SESAR Lines of Change, was detailed in Section 2.3. Question 11. Of the changes listed below, for which ones do you feel you need more data/information, on how these changes will affect future ATM performance? We are specifically talking about the importance to your company/institution of getting more data (e.g. ‘this change will reduce average flight times by 5%’). Table 10. Importance of more information – technical scenarios (SESAR LoCs) Technical scenario Mean (all) ‘Unclear’ (all) Mean (ANSPs) System-wide information management (SWIM) 3.8 10% 3.6 Moving from airspace to trajectory based operations 4.1 5% 4.1 Collaborative planning using Network Operations Planner 4.0 6% 4.1 Managing the ATM network 4.0 4% 4.1 Managing the Business Trajectory in real time 3.8 6% 4.0 Collaborative ground and airborne decision-making tools 4.0 5% 4.1 Queue management tools 3.8 10% 3.9 New separation modes 3.8 9% 3.9 Independent cooperative ground and airborne safety nets 3.7 10% 3.7 Airport throughput, safety and environment 3.8 8% 3.8 (n= 157 (all), 62 (ANSPs); means to 1d.p., per cent values to 0d.p.) From the responses to the questions, it is observed that the higher levels of uncertainty correspond fairly well with the lower means, although these differences are not highly pronounced and the means are quite closely clustered around the higher end of the 1 – 5 scale. When we look at the ANSP subgroup alone (right-hand column), it might have been expected that some more pronounced differences would be observed, but the means are broadly very similar to those of the whole group. Indeed, when we compare the means of the ANSP sub-groups with the rest of the stakeholders (Table 11), none of the differences between the means were significant (independent t-tests, all p > 0.05). It is, however, likely that further disaggregate analyses (in subsequent reporting) will discover differences between sub-groups to this question. At the aggregate level, however, it appears that no particular LoC dominates the demand for further information over and above the others, but we should remind ourselves once more that these means are fairly high. 30 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 Table 11. Importance of more information – technical scenarios (SESAR LoCs); ANSPs Technical scenario Mean (non-ANSPs) Mean (ANSPs) System-wide information management (SWIM) 3.9 3.6 Moving from airspace to trajectory based operations 4.0 4.1 Collaborative planning using Network Operations Planner 3.9 4.1 Managing the ATM network 3.9 4.1 Managing the Business Trajectory in real time 3.7 4.0 Collaborative ground and airborne decision-making tools 3.9 4.1 Queue management tools 3.7 3.9 New separation modes 3.8 3.9 Independent cooperative ground and airborne safety nets 3.6 3.7 Airport throughput, safety and environment 3.9 3.8 (n= 95 (non-ANSPs), 62 (ANSPs); means to 1d.p.) 3.4 Non-technical scenarios (generic) Non-technical scenarios were developed as described in Section 1.4 and presented to respondents for evaluation in the same format as Question 11 (please see previous section). In particular, and as described in the proposal, scenarios which may affect ATM performance were of particular interest. Question 12. Of the changes listed below, for which ones do you feel you need more data/information, on how these changes will affect future ATM performance? We are specifically talking about the importance to your company/institution of getting more data. These types of change are more general than the previous set. 31 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 Table 12. Importance of more information – non-technical scenarios (generic) Mean (all) ‘Unclear’ (all) Mean (ANSPs) 3.4 3% 3.0 3.7 3% 3.7 (c) Airline ability to reaccommodate passengers with missed connections 2.9 8% 2.5 (d) Flight prioritisation rules and flight management (e.g. based on delay costs) 3.9 3% 3.9 (e) ATFM slot trading / swapping / market-based (peak) pricing 3.6 5% 3.8 (f) Airline alliance structures and cooperation models 2.8 6% 2.6 (g) Aircraft size and design 2.9 6% 2.8 (h) Increased airport regionalisation (passengers, cargo) 3.0 6% 2.8 (i) Competition from / cooperation with high-speed rail 2.7 6% 2.4 (j) Dynamic adaption of airport runway configurations 3.5 6% 3.6 Non-technical scenario (a) Environmental regulations and taxes (for airlines and airports) (b) Patterns of European traffic flows & route growth; market maturity & demand sat N (n= 157 (all), 62 (ANSPs); means to 1d.p., per cent values to 0d.p.) The results of Table 12 are far more usefully discussed in the context of the case study selection process, so the main discussion on these data is to be found in Section 3.3 of Deliverable 1.2. Table 13. Importance of more information – non-technical scenarios (generic); ANSPs Mean (non-ANSP) Mean (ANSPs) **3.7 **3.0 3.7 3.7 **3.2 **2.5 (d) Flight prioritisation rules and flight management (e.g. based on delay costs) 4.0 3.9 (e) ATFM slot trading / swapping / market-based (peak) pricing 3.5 3.8 (f) Airline alliance structures and cooperation models 2.9 2.6 (g) Aircraft size and design 3.0 2.8 (h) Increased airport regionalisation (passengers, cargo) 3.1 2.8 **2.9 **2.4 3.5 3.6 Non-technical scenario (a) Environmental regulations and taxes (for airlines and airports) (b) Patterns of European traffic flows & route growth; market maturity & demand sat N (c) Airline ability to reaccommodate passengers with missed connections (i) Competition from / cooperation with high-speed rail (j) Dynamic adaption of airport runway configurations (n= 95 (non-ANSPs), 62 (ANSPs); means to 1d.p.) 32 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 When the scores are analysed for the ANSPs compared with a combined non-ANSP group, three means differ significantly across these groups (**; p < 0.01, independent t-test). The scenarios scored significantly lower by the ANSPs are those that are more likely to directly impact either airlines or airports. This logical outcome lends support to the engagement of respondents with the questionnaire and suggests that further disaggregate analyses may be insightful. 33 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 4 Next steps and a look ahead to future deliverables Section 4 is common to Deliverables 1.1 and 1.2. 4.1 Next steps The key next steps for the CASSIOPEIA project in terms of taking the Workpackage 1 results forward are to: • • • • consolidate the results of deliverables 1.1 – 1.3 decide upon the full implications for the model inform the final selection of the case studies in this consolidated context discuss and refine the model and case study development at the 10 January 2012 workshop, in London 4.2 Future deliverables The next steps outlined above feed specifically into: • Deliverable 2.0, a draft formulation of the regulation, ATM network, agent and exogenous factors models, plus the case study selection reporting – a short-term action for November 2011 • WP2.5 (indicators modelling) and WP2.7 (data requirements) – a medium-term, parallel action, not to be constrained by the case study scope • on-going work of the consultation, communication and dissemination processes of WP5 34 of 35 Project ID E.02.14 D1.1-High-Level Functional Specification – Model Inputs Edition: 01.01.00 5 References EUROCONTROL (2011a). Performance Review Report 2010: An Assessment of Air Traffic Management in Europe during the Calendar Year 2010, EUROCONTROL Performance Review Commission, Brussels, May 2011. EUROCONTROL (2011b). Media & Info Centre News, Network Manager: a new key function in European aviation sees the light today. http://www.eurocontrol.int/news/network-manager-new-key-function-european-aviation-sees-lighttoday European Commission (2010). Press release IP/10/363: http://europa.eu/rapid/pressReleasesAction.do?reference=IP/10/363 European Commission (2011a). Press release MEMO/11/232: http://europa.eu/rapid/pressReleasesAction.do?reference=MEMO/11/232 European Commission (2011b). White Paper: Roadmap to a Single European Transport Area – Towards a competitive and resource efficient transport system, Brussels, http://ec.europa.eu/transport/strategies/2011_white_paper_en.htm. European Commission (2011c). Flightpath 2050 - Europe’s Vision for Aviation (Report of the High Level Group on Aviation Research), ISBN 978-92-79-19724-6, DOI 10.2777/50266 http://www.acare4europe.com/html/documentation.asp Official Journal of the European Union (2011). Regulation (EU) No 677/2011 of 7 July 2011 laying down detailed rules for the implementation of air traffic management (ATM) network functions and amending Regulation (EU) No 691/2010. http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2011:185:0001:0029:EN:PDF SESAR (2011). Dynamic DCB: Bridging the gap (SESAR Project 7.6.5, Dynamic DCB). http://www.sesarju.eu/players/experts/dynamic-dcb-bridging-gap-857 SESAR Consortium (2008). SESAR Definition Phase: Milestone Deliverable 5, SESAR Master Plan, April 2008. 35 of 35