E.02.14-CASSIOPEIA-D1.1-High-Level
Functional Specification – Model Inputs
Document information
Project title
CASSIOPEIA
Project N°
E.02.14
Project Manager
Fundación Instituto de Investigación Innaxis
High-Level Functional Specification – Model Inputs
Deliverable
Name
Deliverable ID
1.1
Edition
01.01.00
Template version
02.00.00
Task contributors
University of Westminster; Fundación Instituto de Investigación Innaxis.
Abstract
This document reports on a survey of ATM stakeholders, executed in order to establish priorities for
the selection of future scenarios (SESAR-focused, and more secular) and case studies to be adopted
as part of a new ATM performance model. The survey questions and their responses are described in
detail. Further consultation with the stakeholder community will inform the development of the model
and shape the evolution of a number of supporting case studies.
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
Authoring & Approval
Prepared By
Name & company
Position / Title
Date
Andrew Cook / University of Westminster
28OCT11
Graham Tanner / University of Westminster
Member of the
Consortium Board
Consortium Member
Reviewed By
Name & company
Position / Title
Date
Graham Tanner / University of Westminster
Consortium Member
31OCT11
Approved By
Name & company
Position / Title
Date
David Pérez / Innaxis
Project Leader
21NOV11
Document History
Edition
Date
Status
Author
Justification
01.00.00
31OCT11
Deliverable
A. Cook
01.01.00
11NOV11
Deliverable
A. Cook
New document for review
by EUROCONTROL
Minor revisions following
comments from
EUROCONTROL
IPR (foreground)
This Deliverable consists of Foreground owned by one or several Members or their Affiliates.
2 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
Table of Contents
EXECUTIVE SUMMARY ...................................................................................................................................5 1 INTRODUCTION...........................................................................................................................................6 1.1 1.2 1.3 1.4 1.5 BRIEF OVERVIEW OF DOCUMENT ................................................................................................................6 INTENDED READERSHIP ...............................................................................................................................6 OBJECTIVES OF DELIVERABLES 1.1 – 1.3....................................................................................................6 FOCUS OF DELIVERABLE 1.1.......................................................................................................................7 OVERVIEW OF THE STAKEHOLDER CONSULTATION PROCESS ......................................................................7 2 STAKEHOLDER CONSULTATION AND SURVEY IMPLEMENTATION ........................................8 2.1 STAKEHOLDER CONSULTATION ..................................................................................................................8 2.1.1 Overview of the stakeholder groups....................................................................................................8 2.1.2 Sampling and representativeness........................................................................................................9 2.2 STAKEHOLDER PERSPECTIVES...................................................................................................................10 2.2.1 Technical versus non-technical interests ..........................................................................................10 2.2.2 Inclusion of the passenger perspective .............................................................................................11 2.2.3 Inclusion of the industry perspective ................................................................................................13 2.3 SURVEY IMPLEMENTATION .......................................................................................................................14 2.3.1 Timing and duration..........................................................................................................................14 2.3.2 Survey content ...................................................................................................................................14 2.3.3 On-line questionnaire .......................................................................................................................15 2.3.4 Comments on specific questions .......................................................................................................23 2.3.5 On-line development, testing and piloting ........................................................................................24 2.3.6 Mailing of invitations to participate in survey..................................................................................24 3 SURVEY RESULTS AND MODEL INPUTS............................................................................................26 3.1 3.2 3.3 3.4 OVERVIEW OF RESPONSE AND DATA CLEANING ........................................................................................26 RESPONDENT PROFILE...............................................................................................................................26 TECHNICAL SCENARIOS (SESAR-RELATED).............................................................................................30 NON-TECHNICAL SCENARIOS (GENERIC)...................................................................................................31 4 NEXT STEPS AND A LOOK AHEAD TO FUTURE DELIVERABLES..............................................34 4.1 NEXT STEPS...............................................................................................................................................34 4.2 FUTURE DELIVERABLES ............................................................................................................................34 5 REFERENCES..............................................................................................................................................35 3 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
List of tables
Table 1. Stakeholder groups.................................................................................................................. 8 Table 2. Comments on specific questions ........................................................................................... 23 Table 3. Data cleaning summary ......................................................................................................... 26 Table 4. Company/institution categories.............................................................................................. 27 Table 5. Re-coded company/institution categories .............................................................................. 27 Table 6. ICAO region in which respondent based ............................................................................... 28 Table 7. Involved in PRB/PRU stakeholder consultation ..................................................................... 28 Table 8. Familiarity with SESAR KPIs ................................................................................................. 29 Table 9. Job function ........................................................................................................................... 29 Table 10. Importance of more information – technical scenarios (SESAR LoCs) ............................... 30 Table 11. Importance of more information – technical scenarios (SESAR LoCs); ANSPs.................. 31 Table 12. Importance of more information – non-technical scenarios (generic) .................................. 32 Table 13. Importance of more information – non-technical scenarios (generic); ANSPs .................... 32 List of figures
Figure 1. Stakeholder interest in types of change ............................................................................... 10 4 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
Executive summary
This report presents the results of a major stakeholder survey on ATM performance assessment, as
part of a Delphi process, to analyse and define the high-level functional requirements of a new ATM
performance model. The model aims to provide policy-makers and stakeholders with a toolset
capable of delivering meaningful answers to questions of particular interest in the European ATM
context. It will evaluate the impact of different technological, operational, economic, or policy options
on a number of KPAs/KPIs.
The stakeholder consultation and survey implementation processes, including sampling and bias, are
discussed in detail. The relative stakeholder interest in exploring ATM performance under two general
headings, ‘technical’ (related to SESAR Lines of Change) and ‘non-technical’ (less technology-driven,
not focusing on SESAR and including secular changes), is discussed.
At the aggregate level, no particular SESAR Line of Change dominates the demand for further
performance modelling over and above the others, whilst interest levels in such performance
assessment was rated fairly high. Further consultation with the stakeholder community will inform the
development of the model and shape the evolution of a number of supporting case studies.
5 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
1 Introduction
1.1 Brief overview of document
This document reports on a survey of ATM stakeholders, executed in order to establish priorities for
the selection of future scenarios and case studies to be adopted as part of a new ATM performance
model. The survey questions and their responses are described in detail.
1.2 Intended readership
Based on a cross-sectional survey of 157 ATM stakeholders, this report identifies scenarios whose
evaluation is of interest to stakeholders, in the context of developing ATM performance assessment
models. These scenarios have been examined separately as ‘technical’ and ‘non-technical’. The
report also details the methodology of the stakeholder consultation process. It is written for the
professional reader and assumes an understanding of air transport and ATM. Without detriment to
appropriate referencing and delineation, the text is not cluttered with explanations of common
acronyms or principles.
1.3 Objectives of Deliverables 1.1 – 1.3
The objective of Workpackage 1 is to analyse and define the high-level functional and non-functional
requirements of CASSIOPEIA.
As described in the proposal, the CASSIOPEIA platform aims to provide policy-makers and
stakeholders with a toolset capable of delivering meaningful answers to questions of particular
interest in the European ATM context. The platform should be able to evaluate the impact of different
technological, operational, economic, or policy options on a number of KPAs/KPIs.
The selection of relevant features and the degree of detail to be included in the model is determined,
in part, by the type of scenarios to be assessed (model inputs – D1.1) and by the type of answers to
be delivered (model outputs – D1.2). The functional requirements will be therefore expressed in terms
of these inputs and outputs. The usability of the model, in-built modularity and growth capability, are
expressed through a number of non-functional requirements (D1.3).
Deliverables 1.1 – 1.3 are thus to be considered as an integrated report on Workpackage 1, although
we have reported on the particular aspects identified above, separately. In particular, D1.1 and D1.2
report on the stakeholder survey, with much of the background material on the survey reported in
D1.1 (and thus not repeated in D1.2). The survey itself is part of the wider consultation process of
WP5.
Note.
During early project implementation, it was decided to bring forward some aspects of the case study
selection and development, with an additional deliverable (D2.0) to draft the formulation of the
regulation, ATM network, agent and exogenous factors models. It has further been agreed that D2.0
will also, and jointly with Workpackage 1, further develop the case study selection process and
specify the elements that will be parameterised in order to define certain scenarios (the latter task
originally planned for D1.1), with these processes in D2.0 to be informed by the stakeholder survey
results presented in D1.1 and D1.2.
Mostly due to some late mailings by third parties of the respondent invitations to participate in the
survey, outside of our control, the time available to report on the survey results was reduced to
several days. However, further analyses will be possible on these data going forward in
CASSIOPEIA, and the results reported herein form just part of the consultation process and its
reporting (see also Section 4).
6 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
1.4 Focus of Deliverable 1.1
The primary objective of this Deliverable is to identify scenarios whose evaluation is of interest to
stakeholders, which we have examined separately as ‘technical’ and ‘non-technical’ scenarios. The
former are based on the SESAR Lines of Change described in the Master Plan (SESAR Consortium,
2008). The latter are other major themes, not driven by technological advances under SESAR per se,
but which may impact on ATM performance; these themes were discussed and short-listed by the
project team.
1.5 Overview of the stakeholder consultation process
The CASSIOPEIA stakeholder consultation process is what is often termed a ‘Delphi’ survey. Two key
characteristics of such an approach is that it relates to the process of collecting opinion from experts,
particularly regarding potential future developments around a given issue, and that it is an iterative
approach, with one phase building on the results of another.
Since, by definition, such respondents have little time for such surveys, we have used an on-line
instrument for the first phase, which respondents can complete at a time which best suits them. The
second phase of the consultation process will be a stakeholder workshop, hosted at the University of
Westminster’s headquarters in January 2012.
By design, the vast majority of stakeholders invited to the workshop will have received this invitation
at the same time as the survey, and be invited to attend at the end of the on-line questionnaire. This
serves two purposes.
Firstly, it enables respondents to acquire a more detailed understanding of the scope of the
consultation process. By this mechanism, they will better appreciate whether the workshop is
something they would be particularly interested in. This allows us to filter out stakeholders from the
workshop for whom attendance would be less mutually beneficial. Secondly, it enables us to build up
a rapport with certain stakeholders and to hopefully motivate them to attend the workshop to develop
the discussion further. This will be enhanced by the fact that the survey results will be presented at
the workshop, although not dominating this event, in order to allow an open development of ideas.
7 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
2 Stakeholder consultation and survey implementation
2.1 Stakeholder consultation
2.1.1 Overview of the stakeholder groups
†
The stakeholder groups shown in Table 1 are adapted from current SESAR definitions . Those which
represent a focus area for the project are indicated in the penultimate column. The sampling methods,
including issues of bias and inclusion, are discussed in the next section.
Table 1. Stakeholder groups
Group
SESAR
stakeholder
group*
Included in SESAR group
Focus
of
project
- scheduled airlines

- charter airlines

Where not performing a
primarily regulatory and/or
political role (see group 6),
airline associations are also
included in this group. Please
see Section 2.2.3.
- airfreight service providers
1
Airspace users
Comments
- business and leisure aviation, incl. GA pilots
- all other non-military: other leisure (e.g. hot
air balloons) & non-leisure (e.g. police, rescue)
- military
2
Airport
operators
3
ANSPs
4
Suppliers
5
Airline, airport
and air traffic
navigation
staff
Several large consortia of airport operators,
1
such as SEAC (expected to include six large
2
European airports), AENA and NORACON ,
are already SESAR members.
SESAR refers both to the more disaggregate
level of ATC and more aggregate level of
NORACON.

Please see Section 2.2.3.

In future, we note that this user
group may be refined to refer
3
explicitly to the European
(regional), sub-regional, and
local network managers. For
now, our level of engagement
is at the ANSP level, and with
EUROCONTROL.
- aircraft manufacturers
- ground and airborne equipment
manufacturers
SESAR includes this category to capture
important human factors issues.
-
Not within our remit.
1
SEAC, the SESAR European Airports Consortium, comprises six major European airports: Aéroports de Paris, BAA Airports,
Flughafen München, Fraport, Schiphol and Flughafen Zürich.
2
NORACON, the NORth European and Austrian CONsortium, consists of eight European ANS providers: Austro Control
(Austria) and the North European ANS Providers (NEAP) including AVINOR (Norway), EANS (Estonia), Finavia (Finland), IAA
(Ireland), ISAVIA (Iceland), LFV (Sweden) and Naviair (Denmark).
3
See Section 2.2.3 regarding the role of EUROCONROL as “Network Manager”.
8 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Group
6
Passengers
and citizens
8
Media, NGOs
and pressure
groups*
9
Transport
infrastructure*
10
Manufacturing
and services
industry*
12
*
Regulatory
and political
4
framework
7
11
†
SESAR
stakeholder
group*
Research /
consultancy*
Research
funding body*
Included in SESAR group
Edition: 01.01.00
Focus
of
project
SESAR refers to the environmental, security,
safety and competition aspects of regulations,
and to European governments. Presumably,
the European Commission would be included
here.
()
- passengers: SESAR details how, through
improved air traffic navigation, information and
positioning, passengers stand to benefit
through shorter and more reliable journeys,
lower costs and improved safety.

Comments
This scope could be extended
to include local government
and land-use planning, and
also to all air transport
organisations, which perform
regulatory and/or political roles
in particular, such as IATA an
ICAO. Air transport unions also
belong to this grouping.
National enforcement bodies /
units for Regulation 261 are
separately identified in the
responses to Question 1 (see
Section 2.3.3).
Whilst
SESAR
groups
passengers
and
citizens
†
together , we split these groups
for our analysis. We will focus
on passengers. See Section
2.2.2 for details.
- citizens: SESAR details how society at large,
whether or not they use air transport, will also
gain from a more competitive European air
industry, less noise around airports, more
efficient and convenient travel and a
contribution to cutting greenhouse gases and
reducing climate change impacts.
-
The complementary groupings
and associations of those
affiliated to the air transport
industry.
-
Providers of complementary
transport (airport connecting
transport,
‘flights’
actually
operated by rail services) and
competitive transport (mostly
high-speed rail).
-
Users of (e.g. shippers) and
suppliers to (excluding aircraft
manufacturers, manufacturers
of ground/ airborne equipment)
air transport industry.
-
Includes
SMEs
primarily
engaged
in
research
/
consultancy, and research
institutions / universities. These
will be primarily included
through contact with the
ComplexWorld network and ad
hoc inclusions in other bespoke
lists used in the survey mailing.

-
This group is shown to
acknowledge
that
such
respondents may be included
within other categories of
respondent listed in Question 1
(see Section 2.3). However,
these are not likely to be many
in number and it would
severely
complicate
the
responses to Question 1 to
separate these respondents
out from other groups.
SESAR stakeholder group source: http://www.sesarju.eu/players/stakeholders
Additional groups not specified at ‘†’ are marked with an asterisk.
2.1.2 Sampling and representativeness
In market research we often attempt to minimise most types of bias, sampling bias being key among
them. In this stakeholder consultation we are more interested in collecting a range of opinions,
4
Described as “regulators and administrations” at: http://www.sesarju.eu/players/stakeholders.
9 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
however. This range of opinions is intended to challenge and extend our thinking regarding the
application and use of KPAs in air transport. More specifically to CASSIOPEIA, the objective is to
investigate:
• which types of change should be evaluated in terms of their impact on performance (as
specified for D1.1);
• which specific KPAs/KPIs will be explored in the modelling, and at what spatial/temporal
scale (as specified for D1.2).
We discussed in Section 1 that this was a Delphi approach, eliciting expert stakeholder feedback. If
this survey were to be representative of all stakeholders, clearly the passengers and citizens would
easily outnumber all the other stakeholders, yet it is undesirable for the feedback to be heavily
dominated in this way. However, the passenger context, in particular, is very important, and the
specific capture of expert opinion in relation to this perspective is discussed in detail in Section 2.2.2.
Within the timescale of the project, and in the context of this survey phase being implemented quite
early in the project, we have focused on what may be termed the core airspace stakeholders on the
supply and demand side.
Important in the context of CASSIOPEIA, we have specific scope to include other stakeholders
through the case studies. These may be selected to complement some of the stakeholder groups
included in the survey, by furnishing more detailed interaction in some already-included groups and/or
specifically developing contacts within new stakeholder groups, not included in this survey.
One objective of the project is to explore new KPA modelling, which may mean focusing on areas for
which only a specific sub-section of the stakeholders or case study partners are supportive of further
research and development. It is important within WP E not only that we focus on new KPA
investigations, but also that we understand how the results of this work may be useful to the ATM
community. We thus need to avoid adopting case studies that are inadvisable due to satisfactory
existing coverage, or due to profound impracticalities due to severe KPA immaturity or the absence of
supporting data.
2.2 Stakeholder perspectives
2.2.1 Technical versus non-technical interests
Stakeholder
Technical
Non-technical
Types of
change
Technical
Non-technical
Figure 1. Stakeholder interest in types of change
Figure 1 shows, figuratively, how what we might term ‘non-technical’ stakeholders (such as
passengers) will tend to have a much more substantial interest in non-technical types of change (such
as increases in emissions taxes on fuel, which may well drive up ticket prices). Their interest in
technical types of change (such as UDPP) will be far less prominent (as represented by the peak of
the triangle on the right).
Some of the ‘non-technical’ stakeholders will not have a depth of expertise in assessing the effects of
various types of change on ATM, and the corresponding KPAs best suited to measure performance.
However, these effects may be estimated in certain example cases (such as the impact on ticket
10 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
prices of emissions taxes, as mentioned) by either presenting such stakeholders with proxy variables
(e.g. seat prices instead of emissions charges) in surveys, or through examination of secondary
research. Whilst it will be important to estimate some of these effects in other parts of the project (e.g.
the agent-based models of WP2), a passenger survey (or, indeed, a survey of citizens not using air
transport) is outside the scope.
In contrast, more technical stakeholders, with an active engagement in technical types of change
(such as the Lines of Change defined by SESAR: see Section 2.3) will also need to take a strong
interest in ‘non-technical’ types of changes (such as emissions taxes). This is represented by the
trapezoid on the left-hand side of Figure 1. Its tapering towards ‘non-technical’ types of change is not
pronounced. The zone of stakeholder / type of change interaction that we cannot explore in
CASSIOPEIA is thus that represented by the top of the triangle on the right-hand side.
2.2.2 Inclusion of the passenger perspective
On 17 February 2005, the European Union’s air passenger compensation and assistance scheme
(Regulation (EC) No 261/2004) was introduced. In addition to affording passengers additional rights in
cases of flight disruption (denied boarding, cancellation and delay), the Regulation also requires
airlines to inform passengers of their rights when a flight is disrupted. This includes giving the contact
details of a national enforcement body, designated by the member state to receive complaints. To
take the examples of the UK and Spain, this body is the corresponding civil aviation authority in each
case – the Civil Aviation Authority (CAA) and Dirección General de Aviación Civil (DGAC),
respectively.
There is no pan-European organisation of the national enforcement bodies. When complaints concern
European flights booked and operated outside of the passenger’s country of domicile, the European
5
Consumer Centre Network (ECC-Net) should be contacted. This is a broad-scope consumer
protection network, for residents in a dispute with a trader located in a different country to their
country of domicile, but also in the European Union. The UK European Consumer Centre is part of
the ECC-Net, which consist of 29 centres (one in each EU country plus Iceland and Norway). Some of
these sites have links to pages offering support with air travel (in the UK it is the first link), others do
not.
The optimum inclusion of the passenger perspective in this project is to be achieved through expert
interface with the national enforcement bodies, rather than through passenger interviews. Given the
bases of the University of Westminster and Innaxis, we have liaised individually with the national
6
enforcement bodies for the UK and Spain. We have approached the UK CAA and the Agencia
Estatal de Seguridad Aérea (State Aviation Safety Agency, the DGAC agency responsible for this
function in Spain), and invited their participation in the survey and the January 2012 workshop. For
other countries, we have sent invitations to the corresponding national contact points, where these email addresses were listed on the European Commission’s website.
Plans are currently underway at the Commission to review Regulation 261, regarding improved
enforcement and clarification of responsibilities. A new legislative proposal is planned for 2012,
looking at key issues such as:
[…] limitation of liability in cases of exceptional circumstances, compensation thresholds, as well as re-routing and
sharing the burden of risk.
European Commission (2011a)
5
Each state has its own website. In the UK it is: http://www.ukecc.net
Later, this liaison may move to a dialogue with the Aviation Consumer Advocacy Panel. This should be up and running by
mid-2012. It is a new body in the UK, part of the Civil Aviation Authority, undertaking consumer representation and replacing
the Air Transport Users Council (AUC). The Panel, and thus the former AUC’s complaints handling role, will become part of the
CAA’s newly created Regulatory Policy Group (which itself replaces the old Economic Regulation Group). The RPG contains a
new sub-group focused on consumer enforcement and market issues, in addition to collecting and analysing aviation statistics
and survey responses, as two of its four key functions.
6
11 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
The review has been prompted by the experience of the ash cloud and severe snow in 2010. It may
be that governments will take on some of the burden during future events similar to the ash cloud, and
airports in the case of uncleared snow, for example. A public consultation will also consider:
[…] commercial practices linked to rescheduling (passenger rights in cases of flight rescheduling) and no-shows
(clarifying the situation for airlines in the case of connecting flights where there are "no show" passengers).
European Commission (2011a)
In the short term, the Commission plans to improve pan-European enforcement of passenger rights,
including a strengthening of the mandate of the network of national enforcement bodies (NEBs) to
adopt decisions on a common interpretation, and regarding enforcement. This reflects a wider
European policy to consolidate coherent enforcement across all modes of transport, with increased
commonality (Regulation (EC) No 1371/2007, for example, offers rail passengers improved
information, assistance in the event of delays of more than one hour, and compensation). The
Commission's new roadmap (White Paper) to a Single European Transport Area for 2050, opens as
follows:
The EU has already established a comprehensive set of passengers’ rights which will be further consolidated.
Following the ash cloud crisis and the experience of extreme weather events in 2010, it has become evident that
Mobility Continuity Plans may be required to preserve the mobility of passengers and goods in a crisis situation.
These events also demonstrated the need for the increased resilience of the transport system through scenario
development and disaster planning.
European Commission (2011b)
A complementary, high-level document (Flightpath 2050 - Europe’s Vision for Aviation) from the
European Commission, addressing European transportation, specifically aviation, cites resilience both
in its foreword and many times in the main document, as well as emphasising the importance of
integration:
The strategy addresses customer orientation and market needs as well as industrial competitiveness and the need to
maintain an adequate skills and research infrastructure base in Europe. By 2050, passengers and freight should
enjoy efficient and seamless travel services, based on a resilient air transport system thoroughly integrated with other
transport modes and well connected to the rest of the world. This will be necessary in order to meet the growing
demand for travel and to cope more easily with unforeseeable events.
European Commission (2011c)
Finally, in the context of the above changes and drives to improve passenger legislation, the
Commission has identified a problem regarding the lack of supporting data, for example regarding the
number of flights within the scope of Regulation 261:
In particular, the assessment, on many issues, has shown a critical lack of data on which to base analysis. The
provision of data from industry and national authorities as well as consumer groups is a central underpinning
proportionate and good regulation. This will be an important issue for the consultation and subsequent revision.
European Commission (2011a)
To summarise, what are the important lessons for CASSIOPEIA, specifically regarding the
stakeholder consultation, and more generally for the project? As in the project:
• the passenger perspective is becoming increasingly important in the evolving European
policy context
• resilience and scenario modelling are also considered important by the Commission
• complementary transport modes are meaningful considerations
• the importance of supporting data is to be considered
12 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
2.2.3 Inclusion of the industry perspective
7
Implementing a European Network Manager was an important step foreseen in the second package
of measures of the Single European Sky (European Commission, 2010). In February 2011, the Single
Sky Committee agreed the Implementing Rule on network functions and voted in favour of
EUROCONTROL being nominated as the SES Network Manager (EUROCONTROL, 2011a). The
successful completion of the nomination process was announced in September 2011
(EUROCONTROL, 2011b).The Network Manager is the body in charge of managing ATM functions
centralised at a European level, such as the design of the European route network and the air traffic
flow management function (currently executed by CFMU). As described in the corresponding
Regulation (EU) No 677/2011 (Official Journal of the European Union, 2011), the Network Manager
will:
[…] detail the Network Strategy Plan through a Network Operations Plan, as further specified in Article 6, addressing
in particular European Union-wide performance targets covering 3 to 5 year, annual, seasonal, weekly and daily
periods
The Network Operations Plan, the Regulation further details (ibid.), shall be based on a structure that
includes:
Consolidated forecast and analysis of the operational performance of the network
•
network, air navigation service provider, functional airspace block and ACC ATM delay/capacity targets
and forecast,
•
airport operational performance,
•
network environment/flight efficiency performance target and forecast,
•
impact of special events,
•
analysis of the operational performance targets and forecast.
and will be tailored to the various individual functions and to the time horizons identified above. This
thus underlines the important point that the Network Manager is responsible for both delivering and
monitoring performance with respect to the targets. Also important for CASSIOPEIA, is the particular
connection, flagged by EUROCONTROL’s Performance Review Unit, between delay propagation and
the Network Manager function:
Reactionary delays are by definition a network issue and a better understanding of the contribution of airports,
airlines and ANS towards those network effects and possible measured to mitigate those effects would be desirable,
particularly with a view to the network manager that will be established under the SES II initiative. However such a
study is complex as it requires linking the individual legs of aircraft (i.e. linking gate-to-gate and turn around phases of
aircraft) at European scale.
EUROCONTROL (2011a)
2.2.3.1 Airspace users and airports
In addition to survey invitations sent to the University of Westminster’s airline (and ANSP) working
group on delay cost management, plus our mailings to passenger’s rights National Enforcement
bodies (see Section 2.2.2), plus ComplexWorld members and participants, and extensive work on
EUROCONTROL and University of Westminster proprietary lists (with airline, ANSP, airport, regulator
and local government contacts), we would like to express our deep gratitude to the following for their
help in promoting the survey and/or workshop (in the majority of cases, both) to their members, with
some particularly strong and helpful support received from certain associations (list in alphabetical
order):
7
Note that we capitalise “Network Manager” to indicate that we are referring to the European Network Manager,
EUROCONTROL (in ICAO terminology, the “Regional” Manager – Europe (and the North Atlantic) is a region), thus
differentiating this from lower-level roles such as the sub-regional (FAB), and local network managers (ACC, or group of ACCs).
13 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
•
•
•
•
•
•
•
•
•
•
•
•
Edition: 01.01.00
ACI Europe
AGIFORS
Association for the Scientific Development of ATM in Europe (ASDA)
Association of European Airlines
CANSO
EUROCONTROL (selected Organisation members, arranged by EEC)
European Regions Airline Association
International Air Carrier Association
NATS
UBM Aviation news (e-newsletter distribution)
UK Aerospace, Aviation & Defence Knowledge Transfer Network
UK Sustainable ATM National Technical Committee
2.3 Survey implementation
2.3.1 Timing and duration
Important in the context of any survey, is that the respondent is not asked to complete a survey which
is too long, as this may reduce response/completion rates and/or respondent compliance (they may
become bored and/or tired). This sets a natural constraint on the amount of material that can be
covered in the survey. As noted, this survey in any case forms part of a wider consultation process,
for example the development of certain issues in the January 2012 workshop. To encourage the
respondents to bear with the survey, a status bar indicated the approximate percentage completion at
the foot of each page. It is worthwhile being candid with the respondent at the outset regarding the
length of time the survey takes to complete (determined during test runs) and to stress the value of
their cooperation, again to encourage engagement.
It is also important that a reasonably narrow and specific time window for survey completion is given.
If the completion date is set too far in the future, this increases the likelihood of the respondent
postponing completion, and not returning to it. The first survey invitations were sent out on 30SEP11.
The survey closure date was initially set at 19OCT11, in order to allow time for the initial analysis of
the corresponding data for D1.1 and D1.2, but had to be extended to 21OCT11 due to some late
invitation mailings via third parties.
2.3.2 Survey content
The content of the survey, and its development, is described in the following sections. It was
important to the objectives of the consultation, as described, to include a range of stakeholder views
and not to introduce ‘pressure bias’ into the survey, whereby those with less expertise were
discouraged from answering. (Indeed, Table 8 suggests a desirable cross-section was achieved). The
questionnaire focuses on the use and importance of KPAs, etc, to the respondent, and not some
abstract benefit to ‘ATM’ in general, which has a far less meaningful interpretation. It is a common
error in surveys to ask respondents about ‘importance’ of a product, or service, or of certain
information, but without setting that ‘importance’ in any appropriate context.
14 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
2.3.3 On-line questionnaire
Key Performance Indicators in Air Transport
Stakeholder Survey
Most people don’t read the introduction page of a survey, but please spare half a minute to read this! Thank you.
We are inviting a wide cross-section of stakeholders to complete this survey. We are hoping to collect a range of views from all
types of stakeholder.
The survey takes about 20 - 25 minutes to complete. Whilst this is a significant amount of your time, we have spent several
months designing the survey and ask only questions that we will use. The results will be used to identify priorities for new Key
Performance Indicators (KPIs), which we will design and test over the next 18 months, intended to benefit stakeholders in an
operationally meaningful way, for example in delay management.
As far as possible, we would like your responses to represent your considered opinion of the views and interests of your
company or institution, rather than your personal views. All questions refer to Key Performance Areas (KPAs) and KPIs in the
European context, although we welcome views from outside Europe. The survey is anonymous.
Our sincere thanks for your valuable time
The University of Westminster, London
The Innaxis Foundation & Research Institute, Madrid
Question content
Question
no.
1
[Responses are radio buttons, unless specified otherwise]
At which type of company/institution are you based?
Airport operator
Airport-supporting organisation
Air navigation service provider (ANSP)
ANSP-supporting organisation
Full-service carrier
Low-cost carrier
Charter airline
Military, air force
Other aircraft operator or airline
Airline-supporting organisation
Aircraft manufacturer
Ground/airborne equipment manufacturer
National civil aviation authority / regulator (excluding Reg 261 enforcement unit)
National enforcement body/unit for Regulation 261
National / local government (including agencies)
EUROCONTROL
European Commission
Researcher / consultant
Other (please specify)[text box]
[drop-down list]
2
In which ICAO region are you based?
Africa-Indian Ocean (AFI) Region
Asia (ASIA) Region
Caribbean (CAR) Region
European (EUR) Region
Middle East (MID) Region
North American (NAM) Region
North Atlantic (NAT) Region
Pacific (PAC) Region
South American (SAM) Region
3
Please enter your group ID code here.[text box]
Each collective group of respondents was sent a group ID code in the e-mail inviting participation in this
survey. This will not be used to identify respondents, only to track response rates from each group.
15 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
4
Edition: 01.01.00
Have you been involved in the stakeholder consultation process this year with the Performance Review
Body / Performance Review Unit (EUROCONTROL, Brussels) regarding Reference Period 1 and/or 2?
Yes
No
Not sure
5
We hope to interview a range of people with varying degrees of familiarity with Key Performance
Indicators (KPIs). How would you describe your familiarity at this time, with the SESAR KPIs for the eleven
ICAO Key Performance Areas (KPAs)?
I am not currently familiar with the KPAs and do not refer to related documentation
I am not currently familiar with the KPAs but I do refer to related documentation
I could quote several KPAs without looking them up, but no specific KPI targets
I could quote several specific KPI targets without looking them up
I could quote most specific KPI targets without looking them up
I am very familiar with the KPIs and their on-going development
[Please select the option as far down the list as applies to you]
6
How useful to your company/institution are the following types of KPA?
If you have any additional comments for any of these KPAs, please use the corresponding text boxes.
Key Performance Area
[Pop-up text:]
Click on each KPA for further details.
These texts refer to SES existing suggestions and adopted targets.
Usefulness
[“not at all”,
“not very”,
“somewhat”,
“very”,
“extremely”;
“unclear”]
Capacity [text box]
Daily and annual IFR flights in Europe; declared airport capacities; annual average
en-route ATFM delay; airport ATFM delays; additional time in taxi-out phase;
additional time in arrival sequencing and metering area (ASMA).
Cost-effectiveness [text box]
En-route and terminal ANS rates in Europe.
Efficiency [text box]
Scheduled flights departing on time (as planned), average delay of remaining
scheduled flights; flights with block-to-block time as planned, average block-to-block
time extension of remaining flights; flights with fuel consumption as planned, average
additional fuel consumption of remaining flights.
Predictability [text box]
Flights arriving on time (as planned), average arrival delay of remaining flights;
coefficient of variation for actual block-to-block times for repeatedly flown routes; total
reactionary delay; reactionary flight cancellation rate; total service disruption delay;
percentage of diversions caused by service disruption.
16 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
Flexibility [text box]
Accommodation of VFR-IFR change requests; unscheduled flights departing on time
(as requested), average delay of remaining unscheduled flights; scheduled flights with
departure time as requested (after change request), average delay of remaining
scheduled flights.
Environment [text box]
Average fuel savings per flight as result of ATM improvements; average CO2
emission per flight as result of ATM improvements; reduction of horizontal en-route
flight extension; effective use of civil/military airspace structures; compliance with local
environmental rules; number of proposed environmentally-related ATM constraints
subjected to transparent assessment with environmental and socio-economic scope.
Safety [text box]
Annual European-wide absolute number of ATM induced accidents and serious or
risk-bearing incidents; effectiveness of safety management (‘maturity’); applying
severity classifications to allow harmonised reporting of severity assessment for:
separation minima infringements, runway incursions, ATM special technical events.
Reporting of Just Culture.
Security [text box]
This refers to protection against direct/indirect threats, attacks/interference to ATM
system. There are no quantified KPIs yet developed.
Access & equity [text box]
This refers to the right of access for all users to necessary ATM resources to fulfil
requirements. There are no quantified KPIs yet developed.
Participation [text box]
This refers to keeping all stakeholders involved in ATM development.
There are no quantified KPIs yet developed.
Interoperability [text box]
This refers to maintaining global standards and uniform principles.
There are no quantified KPIs yet developed.
17 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
7
Edition: 01.01.00
For such KPAs, over what geographical scale do you consider it would be useful to your
company/institution for them to be reported upon? An example might be delays. (Each level assumes the
ones lower in the list would also be reported upon).
By specific airports
By airport regions (e.g. ‘all London’)
By specific routes (airport-to-airport)
By general routes (e.g. Scandinavia to Mediterranean)
By country
By groups of country (e.g. FABs in future)
At the European level
Difficult to generalise across KPAs
Any comments on specific KPAs, or other remarks [text box]
8
For such KPAs, at what level do you consider it would be useful to your company/institution for them to be
reported upon? An example might be delays. (Each level assumes the ones lower in the list would also be
reported upon).
By the hour
By day
By week
By month
By summer/winter timetable season
By year
Difficult to generalise across KPAs
Any comments on specific KPAs, or other remarks [text box]
9
… and by flight / airline level of detail?
Please tick all boxes for which reporting would be useful.
By each phase of flight (including turnaround)
By individual flights
By individual airlines
By low-cost and full-service airlines
By type of haul
Difficult to generalise across KPAs
Any comments on specific KPAs, or other remarks [text box]
10
Please outline briefly how KPAs / KPIs are relevant to / used in your line of work.
[text box]
18 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
11
Edition: 01.01.00
Of the changes listed below, for which ones do you feel you need more data/information, on how these
changes will affect future ATM performance? We are specifically talking about the importance to your
company/institution of getting more data (e.g. ‘this change will reduce average flight times by 5%’).
Take the first example, SWIM. If you already have enough data about SWIM, or feel that its impact on
ATM performance does not affect you anyway, getting more data would be of low importance to you (so
you might select “1” or “2”).
If you think that SWIM will have a significant impact on ATM performance and you do not have enough
data about this, getting more data would be important to you (so you might select “4” or “5”).
Type of change
[Pop-up text:]
Click on each type of change for further details.
(i)
Importance
of getting
more data
[1-5;
5 = high]
[U = unclear]
System-wide information management (SWIM)
Underpins collaborative planning. Enabler of end-user applications (EUAs); not itself
an EUA. Providing relevant information to all parties. Removing constraints on
implementation of EUAs, otherwise imposed by full deployment requirements providing benefits earlier. Creating conditions for advanced EUAs by finding most
appropriate source of information. Migrating simpler EUAs first; not dependent on
deployment of ATM changes. Benefits available even in largely legacy environment.
(ii)
Moving from airspace to trajectory based operations
Deploying dynamic collaborative management of airspace/route structures.
Environmentally friendly procedures (e.g. steep and curved approaches, supported by
approach procedures with vertical guidance (APV)). Formalisation and modelling of
traffic demand and capacity balancing (DCB). Advanced RNP1 in terminal airspace
(reduced spacing between routes, tactical parallel offset as alternative to vectoring).
Advanced RNP1 SIDs and STARs, Advanced CDAs. Free routing cruise in upper
airspace (enabled by FABs).
(iii)
Collaborative planning using Network Operations Planner
Use of ATFM slot swapping. Coordination between more than one ACC, airport
operations and CFMU. Manual user-driven prioritization process (UDPP), then
implementing the network management function to assist airspace users in the UDPP
process. Publication of Shared Business Trajectories by airspace users. Reference
Business Trajectories agreed through collaborative flight planning.
(iv)
Managing the ATM network
Network performance assessment; ATFCM scenarios; management of critical events.
Improved operations at airport in adverse conditions using ATFCM techniques.
Implement interactive Network Operation Plan providing an overview of the ATFCM
situation from strategic planning to real-time operations, available online to
stakeholders for consultation/update. Analyse climate factors impact on route
planning.
Managing the Business Trajectory in real time
(v)
Automated support for traffic load management. En-route controller-pilot comms
complemented by datalink. Optimised en-route cruise-climb setting between pilot and
controllers. Automated support to predict congestion and support dynamic airspace
management. Automated support for ATCOs to de-conflict or synchronise trajectories.
Automated ATCO support for multi-sector planning. Assessing costs and benefits of
RBT process. Define responsibilities between regional and sub-regional/local network
management functions.
19 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
(vi)
Edition: 01.01.00
Collaborative ground and airborne decision making tools
Automated support for near-term conflict detection & resolution, and trajectory
conformance monitoring. In TMA, introduction of ground system route allocation tools
to assist controller in managing potentially large number of interacting routes;
automatic selection of optimum conflict-free route when triggered by a specific event.
Deploy ground system situation monitoring to assist controller in detecting and
assessing impact of deviations from clearances.
(vii)
Queue management tools
DMAN synchronised with pre-departure sequencing. AMAN extended to en-route
airspace. AMAN/DMAN integration, including assistance to controller within TMA to
manage mixed-mode runway operations, and to resolve complex interacting traffic.
Integrate surface management with AMAN and DMAN. DMAN in the queue
management process. AMAN and DMAN from multiple airports. CTA optimisation
through use of datalink. Coordination with (airport-) CDM.
(viii)
New separation modes
Deploying cockpit airborne traffic situation awareness (ATSAW) on-ground & airborne
(incl. Oceanic in-trail procedures). Deploy airborne separation assistance system
(ASAS), manually controlled arrival sequencing & merging (applicable TMAs)
requiring pilot to follow speed commands manually; analyse runway throughput
impact of relative (ASAS) versus absolute (RTA) time-based separation. Deploy
precision trajectory clearances in two dimensions based on pre-defined 2D routes (i.e.
absolute time-based separation).
(ix)
Independent cooperative ground and airborne safety nets
Enhanced airborne collision avoidance system (ACAS): linked to autopilot / flight
director display. Short-term conflict alert (STCA) using enriched surveillance
information: improve ground-based safety net performance using widely shared
aircraft position and intent data. Display ACAS resolution advisories (RAs) to
controllers: introduce RA downlink informing controllers automatically when ACAS
generates an RA.
(x)
Airport throughput, safety and environment
Improved turnaround through CDM; aircraft fuel and emissions management –
airports and en-route. Improved LVPs; deploying final approaches with vertical
guidance procedures (to enable Cat-I-like ops). Introduce global navigation satellite
system (GNSS) / ground-based augmentation system (GBAS) for precision
approaches. Enhanced visual separation on approach. Discontinue wake vortex
minima under certain crosswind conditions. Dynamic surface navigation for aircraft
and conflicts resolution (automated assistance to ATCO).
We are extremely grateful for your cooperation. Two further,
(simpler) grid questions follow, before the survey is complete.
12
The response format to this question is the same as the previous question.
Of the changes listed below, for which ones do you feel you need more data/information, on how these
changes will affect future ATM performance? We are specifically talking about the importance to your
company/institution of getting more data. These types of change are more general than the previous set.
Type of change
Please consider these as very much open in future scope,
i.e. not only based on current plans and research.
Importance
of getting
more data
[1-5;
20 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
5 = high]
[U = unclear]
a
Environmental regulations and taxes (for airlines and airports)
b
Patterns of European traffic flows & route growth; market maturity &
demand saturation
c
Airline ability to reaccommodate passengers with missed connections
d
Flight prioritisation rules and flight management (e.g. based on delay costs)
e
ATFM slot trading / swapping / market-based (peak) pricing
f
Airline alliance structures and cooperation models
g
Aircraft size and design
h
Increased airport regionalisation (passengers, cargo)
i
Competition from / cooperation with high-speed rail
j
Dynamic adaption of airport runway configurations
k
Your own suggestion[text box] (with rating …)
l
Your own suggestion[text box] (with rating …)
13
How useful to your company/institution are the following types of KPI
/ ways of measuring performance?
Please respond in terms of usefulness, without being concerned about
how practical / easy some of these may be to collect at the moment.
a
Average delay of delayed flights
b
% of flights delayed by more than a certain amount of time (long delays, e.g. 60 minutes)
c
Average delay of delayed passengers
d
% of passengers delayed by more than a certain amount of time (long delays, e.g. 60 minutes)
e
Passenger arrival delay at final destination (compared with flight delay by segment)
f
Measures of delay propagation (‘spreading’ between flights)
g
Number of cancelled flights
h
Number of passengers: with missed connections; with cancelled flights
i
Delays measured in minutes
j
Delays measured in costs
k
Measures of delay equity (e.g. are all airlines / airspace areas, affected to the same
extent?)
l
Your own suggestion[text box] (with rating …)
Usefulness
[“not at all”,
“not very”,
“somewhat”,
“very”,
“extremely”;
“unclear”]
21 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
m
Q14
Edition: 01.01.00
Your own suggestion[text box] (with rating…)
Please indicate which of the following best describes your job function.
Senior manager, not directly active in ATM/flight operations
Other role, not directly active in ATM/flight operations
Senior manager, directly active in ATM/flight operations
Other role, directly active in ATM/flight operations
Research / consultancy
Other (please specify if none of above approximately describes your job function)[text box]
Q15
Is there anything further, especially regarding KPAs, that you would like to mention?
[text box]
Our sincere thanks for your valuable time, which we genuinely appreciate.
After you click on ‘Finish’, there is a final screen giving you an opportunity
to engage in, or just hear more about, our on-going ATM performance research.
This could take a few seconds to appear, please do not click ‘Finish’ twice. Thank you.
(Q16,
although not
marked as in
web survey)
Please select what may be of interest to you below, with no commitments at this
stage.
Workshop
On 10 January 2012, we will be holding a (free) workshop in central London to discuss
and develop these results. We will hold open discussions about ATM performance and
how its measurement might be developed in the future, with presentations from on-going
SESAR JU projects in this area. Fuller details of the programme for the day will be
available soon. Please select this box if you are interested in attending.
[check box]
Case studies
Below is a selection of case studies which we are considering. Please select any which
you would like us to contact you further about (regarding participation or just being sent
information, for those we develop).
Newly designed medium/short-range aircraft, flying lower/slower in order to save fuel / reduce CO2
Local environmental restrictions limiting airport and terminal airspace capacity
Changes in delays under scenarios of price bidding for FLs/ slot times, by airlines to ANSPs
Investigating CDM performance variations with different levels of information sharing
ATFM slot allocations as a tradable commodity (airport to airline, or airline to airline)
Tactical, dynamic exchange of departure slots between airlines
Peak and off-peak pricing in air navigation charges and slots
The use of biofuels giving airlines ATC/slot preferences and/or price differentials
Different ATFM prioritisation strategies / algorithms
Technological improvements reducing uncertainty in 4D trajectories - network level impact
FMS with a Cost Index that includes environmental and variable slot/ATC costs
Turbofans replaced by propfan engines, with different trajectory optimisations
How the timing of sharing actual aircraft operating details influences 4D trajectory planning and execution, and system
performance
[check boxes]
Further information / reports
To be sent to you every few months by e-mail
22 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
[check box]
After you have selected the appropriate check-boxes above, please enter your e-mail
address in the box below and click ‘submit’.
[text box]
Important! This e-mail address will not be stored with your survey reply and cannot be
used to identify you in the survey. It will only be used for the purposes stated above.
2.3.4 Comments on specific questions
Table 2. Comments on specific questions
Question(s)
Comments
1, 2, 3
Data from these questions will allow us to carry out disaggregate analyses of the data. It is
important that respondents are assured that their responses are anonymous. Question 2 is simply to
differentiate which respondents are from Europe (by design, the vast majority, and the focus of our
work).
2
There is a lack of consistent reporting on the definition and number of ICAO regions. This list was
compiled from various ICAO sources, several of which refer to nine ICAO regions.
4
This question is asked to flag any potential bias resulting from respondent participation in this
parallel consultation process.
5
We may consider responses elsewhere by levels of familiarity / expertise, expressed here. The
phrasing is designed to make respondents feel comfortable expressing non-expert status.
6
These pop-up texts were developed largely from the SESAR Master Plan (SESAR Consortium,
2008) and Performance Review Report 2010 (EUROCONTROL, 2011a), and includes the EU-wide
performance targets for the first reference period of the Single European Sky performance scheme.
8
Note that the OJEU, with reference to performance targets, sets forward: “ […] detail the Network
Strategy Plan through a Network Operations Plan, as further specified in Article 6, addressing in
particular European Union-wide performance targets covering 3 to 5 year, annual, seasonal, weekly
and daily periods” Official Journal of the European Union (2011).
10
This question may capture unexpected uses by stakeholders types, or identify lack of such use.
11
This question is more complicated than the others and would not be employed in a survey of the
general public. It conflates two responses into one ‘importance’ scale, based on the perceived
impact on ATM performance and whether the respondent considers that there is already sufficient
data/information. If either no impact is perceived, or sufficient data is considered to be available, the
need for further information is low. This conflation technique obviates the need for two response
scales for each element in the list of changes. We use the word “data” intentionally instead of
“modelling”, as the latter may have negative connotations for some respondents, and the survey is
not about seeking views on the value of modelling ATM performance.
Options (i) to (x) are based on the Master Plan (ibid.) LoCs 1 to 10, respectively. We have focused
on the earlier Service Levels (SL0 to SL2 ), as we can’t capture all of them here. The earlier SLs are
generally reasonably tangible (nearer-term) to the respondents/stakeholders. An attempt has been
made to capture, in particular, those LoCs that might be more useful/interesting to model. (We note
that SWIM as a methodology of sharing information can apply to all ATM capability and service
levels. The Capability Level may relate to an extension of geographical/spatial availability, although
different ATM SLs may equally need more advanced and/or widespread implementation of SWIM.)
The pop-up texts (developed from the Master Plan texts) are only visible when deployed by the
respondent, to avoid clutter on the page (which therefore looks a lot simpler on-line than it does in
this document). These texts have been edited down, over several iterations, to try to give unbiased
and neutral summaries of the main LoC components, and such that each box contains an
approximately similar number (50 – 65) of words (to reduce the appearance of greater importance
23 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Question(s)
Edition: 01.01.00
Comments
for those with longer entries).
It is vital that the text “5=high” is clear to the respondents, as different cultures will use these scales
in different ways, unless explicit instruction is given.
12
A number of these elements embrace a range of potential considerations. For example, element ‘b’
could include domestic infrastructural capacity to accommodate inbound tourism, and element ‘c’
could include scenarios such as dynamic scheduling of aircraft and changes in load factors. For
simplicity and speed of completion, these are not explicitly presented to the respondent, since this
question mainly serves to obtain an initial understanding of the perceived relative importance of
these generic areas.
13
This question needs to strike a difficult balance between being too general and too specific. Since
measures of delay and others types of disruption are central to the project, we have focused on
such examples here, although the list could obviously have been very long. The idea is to compare
the relative, perceived usefulness of these metrics within and across the stakeholder groups. To
reduce the extent to which certain respondents might indicate that they are all pretty useful, or, not
particularly useful (known as the ‘halo’ effect, or ‘straight-lining’), strong anchors have been used (at
the scale extremes). By this point in the survey, it is hoped that respondents might be particularly
ready to suggest (types of) metric of their own.
15
It is important to give respondents an opportunity to express ideas / concerns not covered by the
(limited number of) questions in the questionnaire. The frequencies of spontaneously given
comments (e.g. possibly relating to lack of appropriate data) often affords valuable insights.
16
The survey data are saved first. This page operates separately so that the survey data are not lost if
the respondent just closes this form. As a back-up, the workshop invitation is also included in the email inviting the respondent to complete the survey.
2.3.5 On-line development, testing and piloting
Following early tests of some of the publicly available on-line survey hosts, the decision was taken to
design and host a bespoke on-line survey on a university server, thereby reducing the constraints
imposed by standard question layouts. Care was taken to ensure the resulting questionnaire was
compatible with multiple browsers and operating systems, and that the layout and features appeared
consistent. During the development stage, the Project Team and a number of interested parties
completed a series of pilots / test runs on different versions of the following browsers: Internet
Explorer, Firefox, Opera, Chrome and Safari; running on various releases of Windows, Mac OS and
Linux operating systems.
Examples of problems identified through testing included ‘pop-up’ information boxes not being
displayed correctly in older versions of Internet Explorer, and the questionnaire ‘back’ and ‘next’
buttons not working whenever the JavaScript option was disabled. The ‘pop-up’ problem was rectified
by using an alternative technique and JavaScript detection redirected respondents, when necessary,
to a help page with step-by-step instructions showing how to re-enable it. For simplicity, responses
were saved to text files. Tests were carried out to check that responses were saved accurately, that
long text strings (including those with line feeds) were not curtailed and that the server could handle
simultaneous response submissions. Daily back-ups of the response files were made for the duration
of the survey.
2.3.6 Mailing of invitations to participate in survey
The distribution lists used for the survey mailings were as described in Section 2.2.3.1. To maximise
response rates, individualised messages were sent to each respondent group, although this was
highly time consuming.
Responses to each group were also tracked by means of collective ID codes (see Question3, Section
2.3.3). Whilst most groups were sent reminder messages after the initial invitations, those with no
24 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
responses after a week or so were particularly targeted for such remedial action. The survey was live
from 30SEP11 to 21OCT11, as mentioned above.
25 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
3 Survey results and model inputs
3.1 Overview of response and data cleaning
Responses were downloaded from the host server each day, in order to mitigate data loss in the
event of a server malfunction or data corruption, and to monitor any other problems – none were
observed. Due to the degree of control regarding the data input obtained through the coding and
structure of the on-line survey, relatively little data cleaning and logic checking were required. No
range errors were detected. Line feeds were removed from free text responses, recorded when
respondents inserted new lines whilst writing within text boxes, prior to importing the data into the
analysis package SPSS.
A total of 170 responses were received for the main survey, and 126 showed an interest in the
forthcoming workshop and/or the potential case studies (stored as a separate file for anonymity
purposes). After data cleaning, which included the removal of blank data files (presumably due to
respondents clicking through the survey in inspection mode) and duplicate entries (due to
respondents not following the request not to double-click the final ‘submit’ button), the raw 170 main
survey responses received furnished a clean data set of 157 responses, and 107 after similar
cleaning in regard to the workshop and case studies data file (discussed in Deliverable 1.2).
Table 3. Data cleaning summary
Total
responses
Blank
responses
Duplicate
responses
Total responses
after cleaning
Stakeholder survey
170
7
6
157
Workshop and case studies
126
15
4
107
Dataset
We should take some objective note that, considering the complex and specialist nature of the survey,
this is an extremely satisfactory outcome. All supplementary comments relating to the survey and its
objectives were also positive and supportive (with the exception of comments received from one
airline questioning the sampling method, to which the University of Westminster offered a detailed
reply). The nature of the free responses given to the open-ended questions also suggests a generally
positive engagement with the survey process).
3.2 Respondent profile
The raw respondent categories were re-coded into the new groupings indicated. The re-coding is to
allow the breakdown of the responses in subsequent analyses into groupings that are not too sparsely
populated, with no intention of suggesting any diminished value of the smaller groups.
Some of the response categories were there to identify certain respondents in case they were
included in the mailings not directly under our control (see Section 2.2.3.1), even though they were
not a target response group. Despite the pleasing overall response to the survey, there has been an
inevitable aggregation of some categories into slightly heterogeneous, notable the “collective” group.
26 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
Table 4. Company/institution categories
Company / institution category
Frequency
Re-coded
Airport operator
9
Airport
Airport-supporting organisation
1
Airport
56
ANSP
6
ANSP
Air navigation service provider (ANSP)
ANSP-supporting organisation
Full-service carrier
16
AO pax
Low-cost carrier
4
AO pax
Charter airline
1
AO pax
Military, air force
4
AO non-pax
Other aircraft operator or airline
6
AO non-pax
Airline-supporting organisation
4
Collective
Aircraft manufacturer
1
Collective
Ground/airborne equipment manufacturer
0
N/A
National civil aviation authority / regulator
(excluding Reg 261 enforcement unit)
3
Collective
National enforcement body/unit for Regulation 261
4
Collective
National / local government (including agencies)
0
N/A
EUROCONTROL
9
EUROCONTROL
European Commission
0
N/A
Researcher / consultant
7
Research
5
[see text]
21
Missing
Other (please specify)
Missing
Total
157
This re-coding, to be used as a common basis for the rest of the analyses, does not prevent us from
identifying groups into the raw response coding where this is useful. It remains somewhat of a
disappointment that more NEBs did not respond, but the other key stakeholder groups have furnished
a useful response. The responses are clearly strongly represented by ANSPs, not through an
intentional over-sampling, and the reader is reminded that we have not controlled for more than one
response from any given ANSP (due to the anonymity and the fact that attempting to collect a
consolidated view from any given institution would have taken far more time than was available).
Four of the five responses originally coded as ‘other’ fitted into one of the new categories (two into
‘research’, two into ‘collective). The remaining case was assigned to ‘missing’.
Table 5. Re-coded company/institution categories
Re-coded category
Frequency
AO pax
21
AO non-pax
10
ANSP
62
Airport
10
EUROCONTROL
9
Collective
14
Research
9
Missing
22
27 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Total
Edition: 01.01.00
157
Table 6. ICAO region in which respondent based
ICAO region
Frequency
Africa-Indian Ocean
1
Asia
3
Caribbean
0
European
135
Middle East
3
North American
4
North Atlantic
2
Pacific
5
South American
2
Missing
2
Total
157
As expected, the majority of respondents were from Europe. Just over half of the respondents were
not involved in the PRB/PRU consultation (to be reported upon in future CASSIOPEIA deliverables,
including our liaison therewith), thus representing a new respondent input to this WPE process and
demonstrating the existing good level of coverage in the PRB/PRU consultation. All the respondents
(with one exception) involved in the latter consultation were based in the European ICAO region, as
would be expected. Table 8 shows a fairly even distribution of declared KPI expertise across the
respondents, which will be a useful mechanism for analysing subsequent responses in this survey
reporting.
Table 7. Involved in PRB/PRU stakeholder consultation
Involved in PRB/PRU
stakeholder consultation
Percent
Yes
35%
No
56%
Unsure
8%
Missing
1%
Total
100%
28 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
Table 8. Familiarity with SESAR KPIs
Familiarity with SESAR KPIs
Percent
I am not currently familiar with the KPAs
and do not refer to related documentation
26%
I am not currently familiar with the KPAs
but I do refer to related documentation
18%
I could quote several KPAs without looking
them up, but no specific KPI targets
20%
I could quote several specific KPI targets
without looking them up
14%
I could quote most specific KPI targets
without looking them up
I am very familiar with the KPIs and their
on-going development
Missing
6%
16%
1%
Total
100%
Table 9. Job function
Job function (with eleven ‘other’ re-codes)
Frequency
Senior manager, not directly active in ATM/flight operations
24
Other role, not directly active in ATM/flight operations
24
Senior manager, directly active in ATM/flight operations
46
Other role, directly active in ATM/flight operations
38
Research/ consultancy
21
Other
0
Missing
4
Total
157
In the original data, eleven responses to job function were coded as ‘other’, although all eleven
respondents gave further information regarding their job function (again indicating a good level of
respondent engagement in the survey), such that it was possibly to assign them to one of the set
categories (sometimes with reference to their other responses).
Of the nine respondents in the ‘research (/consultancy)’ category for company/institution in Table 5,
eight were classified under ‘research/consultancy’ under job function, and one as ‘senior manager,
directly active in ATM/flight operations’, such that these responses were mutually consistent. The
distribution of job functions also indicates a diverse respondent base.
29 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
3.3 Technical scenarios (SESAR-related)
The development and presentation to stakeholders of the SESAR technical scenarios, each
representing one of the SESAR Lines of Change, was detailed in Section 2.3.
Question 11. Of the changes listed below, for which ones do you feel you
need more data/information, on how these changes will affect future ATM
performance?
We are specifically talking about the importance to your company/institution
of getting more data (e.g. ‘this change will reduce average flight times by 5%’).
Table 10. Importance of more information – technical scenarios (SESAR LoCs)
Technical scenario
Mean (all)
‘Unclear’ (all)
Mean (ANSPs)
System-wide information management (SWIM)
3.8
10%
3.6
Moving from airspace to trajectory based operations
4.1
5%
4.1
Collaborative planning using Network Operations Planner
4.0
6%
4.1
Managing the ATM network
4.0
4%
4.1
Managing the Business Trajectory in real time
3.8
6%
4.0
Collaborative ground and airborne decision-making tools
4.0
5%
4.1
Queue management tools
3.8
10%
3.9
New separation modes
3.8
9%
3.9
Independent cooperative ground and airborne safety nets
3.7
10%
3.7
Airport throughput, safety and environment
3.8
8%
3.8
(n= 157 (all), 62 (ANSPs); means to 1d.p., per cent values to 0d.p.)
From the responses to the questions, it is observed that the higher levels of uncertainty correspond
fairly well with the lower means, although these differences are not highly pronounced and the means
are quite closely clustered around the higher end of the 1 – 5 scale. When we look at the ANSP subgroup alone (right-hand column), it might have been expected that some more pronounced
differences would be observed, but the means are broadly very similar to those of the whole group.
Indeed, when we compare the means of the ANSP sub-groups with the rest of the stakeholders
(Table 11), none of the differences between the means were significant (independent t-tests, all
p > 0.05). It is, however, likely that further disaggregate analyses (in subsequent reporting) will
discover differences between sub-groups to this question. At the aggregate level, however, it appears
that no particular LoC dominates the demand for further information over and above the others, but
we should remind ourselves once more that these means are fairly high.
30 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
Table 11. Importance of more information – technical scenarios (SESAR LoCs); ANSPs
Technical scenario
Mean (non-ANSPs)
Mean (ANSPs)
System-wide information management (SWIM)
3.9
3.6
Moving from airspace to trajectory based operations
4.0
4.1
Collaborative planning using Network Operations Planner
3.9
4.1
Managing the ATM network
3.9
4.1
Managing the Business Trajectory in real time
3.7
4.0
Collaborative ground and airborne decision-making tools
3.9
4.1
Queue management tools
3.7
3.9
New separation modes
3.8
3.9
Independent cooperative ground and airborne safety nets
3.6
3.7
Airport throughput, safety and environment
3.9
3.8
(n= 95 (non-ANSPs), 62 (ANSPs); means to 1d.p.)
3.4 Non-technical scenarios (generic)
Non-technical scenarios were developed as described in Section 1.4 and presented to respondents
for evaluation in the same format as Question 11 (please see previous section). In particular, and as
described in the proposal, scenarios which may affect ATM performance were of particular interest.
Question 12. Of the changes listed below, for which ones do you feel you
need more data/information, on how these changes will affect future ATM
performance?
We are specifically talking about the importance to your company/institution
of getting more data. These types of change are more general than the
previous set.
31 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
Table 12. Importance of more information – non-technical scenarios (generic)
Mean
(all)
‘Unclear’
(all)
Mean
(ANSPs)
3.4
3%
3.0
3.7
3%
3.7
(c) Airline ability to reaccommodate passengers with missed connections
2.9
8%
2.5
(d) Flight prioritisation rules and flight management (e.g. based on delay costs)
3.9
3%
3.9
(e) ATFM slot trading / swapping / market-based (peak) pricing
3.6
5%
3.8
(f) Airline alliance structures and cooperation models
2.8
6%
2.6
(g) Aircraft size and design
2.9
6%
2.8
(h) Increased airport regionalisation (passengers, cargo)
3.0
6%
2.8
(i) Competition from / cooperation with high-speed rail
2.7
6%
2.4
(j) Dynamic adaption of airport runway configurations
3.5
6%
3.6
Non-technical scenario
(a) Environmental regulations and taxes (for airlines and airports)
(b) Patterns of European traffic flows & route growth; market maturity & demand sat
N
(n= 157 (all), 62 (ANSPs); means to 1d.p., per cent values to 0d.p.)
The results of Table 12 are far more usefully discussed in the context of the case study selection
process, so the main discussion on these data is to be found in Section 3.3 of Deliverable 1.2.
Table 13. Importance of more information – non-technical scenarios (generic); ANSPs
Mean
(non-ANSP)
Mean
(ANSPs)
**3.7
**3.0
3.7
3.7
**3.2
**2.5
(d) Flight prioritisation rules and flight management (e.g. based on delay costs)
4.0
3.9
(e) ATFM slot trading / swapping / market-based (peak) pricing
3.5
3.8
(f) Airline alliance structures and cooperation models
2.9
2.6
(g) Aircraft size and design
3.0
2.8
(h) Increased airport regionalisation (passengers, cargo)
3.1
2.8
**2.9
**2.4
3.5
3.6
Non-technical scenario
(a) Environmental regulations and taxes (for airlines and airports)
(b) Patterns of European traffic flows & route growth; market maturity & demand sat
N
(c) Airline ability to reaccommodate passengers with missed connections
(i) Competition from / cooperation with high-speed rail
(j) Dynamic adaption of airport runway configurations
(n= 95 (non-ANSPs), 62 (ANSPs); means to 1d.p.)
32 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
When the scores are analysed for the ANSPs compared with a combined non-ANSP group, three means differ
significantly across these groups (**; p < 0.01, independent t-test). The scenarios scored significantly lower by
the ANSPs are those that are more likely to directly impact either airlines or airports. This logical outcome lends
support to the engagement of respondents with the questionnaire and suggests that further disaggregate
analyses may be insightful.
33 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
4 Next steps and a look ahead to future deliverables
Section 4 is common to Deliverables 1.1 and 1.2.
4.1 Next steps
The key next steps for the CASSIOPEIA project in terms of taking the Workpackage 1 results forward
are to:
•
•
•
•
consolidate the results of deliverables 1.1 – 1.3
decide upon the full implications for the model
inform the final selection of the case studies in this consolidated context
discuss and refine the model and case study development at the 10 January 2012
workshop, in London
4.2 Future deliverables
The next steps outlined above feed specifically into:
• Deliverable 2.0, a draft formulation of the regulation, ATM network, agent and exogenous
factors models, plus the case study selection reporting – a short-term action for November
2011
• WP2.5 (indicators modelling) and WP2.7 (data requirements) – a medium-term, parallel
action, not to be constrained by the case study scope
• on-going work of the consultation, communication and dissemination processes of WP5
34 of 35
Project ID E.02.14
D1.1-High-Level Functional Specification – Model Inputs
Edition: 01.01.00
5 References
EUROCONTROL (2011a). Performance Review Report 2010: An Assessment of Air Traffic
Management in Europe during the Calendar Year 2010, EUROCONTROL Performance Review
Commission, Brussels, May 2011.
EUROCONTROL (2011b). Media & Info Centre News, Network Manager: a new key function in
European aviation sees the light today.
http://www.eurocontrol.int/news/network-manager-new-key-function-european-aviation-sees-lighttoday
European Commission (2010). Press release IP/10/363:
http://europa.eu/rapid/pressReleasesAction.do?reference=IP/10/363
European Commission (2011a). Press release MEMO/11/232:
http://europa.eu/rapid/pressReleasesAction.do?reference=MEMO/11/232
European Commission (2011b). White Paper: Roadmap to a Single European Transport Area –
Towards
a
competitive
and
resource
efficient
transport
system,
Brussels,
http://ec.europa.eu/transport/strategies/2011_white_paper_en.htm.
European Commission (2011c). Flightpath 2050 - Europe’s Vision for Aviation (Report of the High
Level Group on Aviation Research), ISBN 978-92-79-19724-6, DOI 10.2777/50266
http://www.acare4europe.com/html/documentation.asp
Official Journal of the European Union (2011). Regulation (EU) No 677/2011 of 7 July 2011 laying
down detailed rules for the implementation of air traffic management (ATM) network functions and
amending Regulation (EU) No 691/2010.
http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2011:185:0001:0029:EN:PDF
SESAR (2011). Dynamic DCB: Bridging the gap (SESAR Project 7.6.5, Dynamic DCB).
http://www.sesarju.eu/players/experts/dynamic-dcb-bridging-gap-857
SESAR Consortium (2008). SESAR Definition Phase: Milestone Deliverable 5, SESAR Master Plan,
April 2008.
35 of 35