Incremental Safety Assurance
of an
Autonomous Decision-making System
for UAS
IET International System Safety Conference 2010
C.A. Lucas*, A. Lyons†, W. Glover*, J.S. Cross*
*Agent Oriented Software Limited, †Aero Synergy
19/10/2010
Unclassified
1
Outline
Role of safety-critical software in autonomous UAS
The challenge of clearing incremental changes to this software
Evolving regulatory environment
Modular safety case
Icing case study
Conclusions
2
Background
Modern aircraft and engines rely on software for primary air
vehicle control, no manual reversion
Consequently FCS or FADEC software is safety critical
Substantial cost and timescale required to introduce software
changes, once safety assured
Modifications incur re-certification costs that approach cost of
original clearance
Desirable to have a modular software safety assurance process
relating software re-assurance effort to scale of change introduced
3
Implications for Autonomous UAS
Have decision-making software that replaces many functions
performed by human crew of a manned or remotely piloted vehicle
Scope extends beyond simply flying the vehicle to a wide range of
other critical roles, such as:
detect and avoid
mission management and re-routing
power management
failure detection, analysis and resolution, to Prognostic Health Management
centre of gravity and fuel management
ATC and communications
ground handling
4
Challenge
Many successful manned A/C recognised for their versatility
e.g., C130, dH Mosquito, Tornado
Breadth of capability results from inherent human versatility:
take on new missions if correctly briefed
train as necessary for the new mission
Autonomous UAS must be endowed with versatile behaviours
If adapted to new or different missions then on-board Autonomous Decisionmaking System (ADMS) has to be upgraded with necessary additional or
revised behaviours
Challenge – how can these be safely introduced without
requiring the re-generation of evidence for parts that remain
unchanged?
5
Approach used and relevance
Strategy to achieve incremental approval of autonomous software
systems
Start with clearance of the decision-making software engine (ADMS)
Then incrementally adding behaviours and data to reason with
Critical aspects:
Reasoning engine
Behaviours
Data
Relevant programmes:
SUAV(E) & Taranis
ASTRAEA
Ministry of Defence SSEI
6
Evolving Regulatory Environment
Civil airworthiness regulators starting to address UAS certification
Most UAS are currently operated in combat zones under war-like conditions
Not cleared to any formal set of regulations
Australia’s CASA one of first regulators to introduce civil UAS regulation
CASR, 1998, Part 101, Subpart F specific to UAS
NATO nations recently adopted STANAG 4671
However 4671 has a number of critical limitations when applied to autonomous
vehicles:
Introduction: “These requirements may not be sufficient for the certification of UAV
Systems with unconventional, novel or extremely complex features”, and further: “….the
following areas are not covered by this airworthiness code:
Airspace integration and segregation of aircraft (including “detect and avoid”),
Non-deterministic flight, in the sense that UAV flight profiles are not pre-determined or
UAV actions are not predictable to the UAV crew,…”
7
UK Regulatory Initiatives
2008 CAA updated CAP722 “Unmanned Aircraft System Operations
in UK Airspace – Guidance”
Chapter on autonomy introduced for the first time in a regulatory document
2009 CAA first released a draft of AMC UAS.1309: “Guidance
material for UAS system safety requirements”:
(a) “This Acceptable Means of Compliance (AMC) is designed to set minimum
acceptable UAS systems integrity levels in order to protect persons from collisions
with UAS.
(b) This AMC is applicable to the Special Conditions defining the system
requirements for the collection of systems that perform the functions usually assigned
to an aircraft located pilot, in other words the ‘Synthetic Pilot’ (SP)….”
(c) This AMC, a UAS certification document, has been specifically tailored for use
with UAS of all sizes …. to ensure the protection of persons in the air and on the
ground from the effects of UAS….
(d) This AMC generally follows the ethos of AMC CS/FAR-25.1309.
(e) The certification basis for UAS will be similar to those for manned aircraft
however there will be differences related to the absence of an aircraft-located pilot.”
8
ADMS Basis of Certification –
In-flight Icing
Evaluation of requirements for a UAS to fly into
known icing conditions chosen as the basis of the study
This case study then used as the vehicle for the study of
incrementally certifiable software system
9
In-Flight Icing
Comprised of two scenarios – baseline and an
incremented case
Baseline is UAS not cleared for flight into known icing
conditions, ADMS will include reasoning to:
Detect and identify possible icing conditions
Take necessary actions to avoid these conditions
Ensure effect of the icing conditions is minimised
Continue mission if possible
10
In-Flight Icing – Increment
Incremented case describes the changes made so that UAS
cleared for flight into known icing conditions
Autonomous software will require modification to:
Manage and maintain anti-icing and de-icing mechanisms
Assess the severity of icing conditions and determine if UAS can fly
through them or if it should attempt to exit (cf Rex SAAB 340)
Continually monitor ice sensors to determine level of ice accretion
Determine preferred routes both through and out of icing conditions
Process will be developed to ensure that evidence for core
autonomous system does not need to be reproduced
Only modified or new modules affected by the changed functionality
11
Software Approach
Approach for incremental certification is:
Modularisation of safety argumentation, allowing re-use of base
modules (e.g. compiler and Run-time Infrastructure (RTI)) for each new
system
Classification of incremental changes leading to different, and
potentially limited, needs for re-certification of applications whose base
version is itself certified
A language is developed to:
simplify modularisation of multi-agent systems and individual agents,
provide support for formal analysis of application
Additional tools help evidence collection suitable for
incremental certification (non-regression of unchanged
functionality)
12
Example
reasoning
plan
13
Proposed Safety Assurance Process
Establish a basis of certification, agreed with the CAA
Draft a Certification Plan, review with CAA
Functional Hazard Assessment
FMECA
Reliability assessment
Safety targets per AMC (UAS).1309
14
ADMS Basis of Certification –
How Established?
Existing code clauses applicable without alteration
Clauses wholly inapplicable to UAS
Clauses with a valid safety objective but which need to
be re-written
Wholly new clauses unique to UAS
15
Establish Basis of Safety Assurance
Source Documents:
CS-23
STANAG 4671
EU-OPS
Part Ops
Part FCL
AMC (UAS)1309
16
Establish Basis of Safety AssuranceAirframe Codes
CS23 –
EASA code for manned aeroplanes
Normal, Utility, Aerobatic, Commuter
17
Establish Basis of Safety AssuranceAirframe Codes
STANAG 4671
NATO code for Unmanned Air Systems
Based upon CS23, but 2 issues:
Failure mode severities, no compatibility to manned codes
Ignores any issue that affects a pilot in the manned code
18
UAS Airframe Codes Issues with the
STANAG – Example 1
CS23.773 Pilot’s View:
Clear and undistorted view
Free from Glare
Protected against fog or frost
STANAG says: Not Applicable
19
UAS Airframe Codes Issues with the
STANAG – Example 1
CS(UAS)23.773 Sensor View:
Clear and undistorted view of intended field
Free from Glare
Protected against fog or frost
20
UAS Airframe Codes Issues with
the STANAG – Example 2
CS23.775 (h)(i) Windshields and Windows
(1) Windshield panes directly in front of the pilot(s) in the normal
conduct of their duties, and the supporting structures for these
panes must withstand, without penetration, the impact of a 0∙91 kg
(2 lb) bird when the velocity of the aeroplane relative to the bird
along the aeroplane’s flight path is equal to the aeroplane’s
maximum approach flap speed.
21
UAS Airframe Codes Issues with
the STANAG – Example 2
USAR 775 says Not applicable
CS (UAS) 23.775 Clause devised by Aerosynergy:
(a) Forward-facing structure, covers and lenses, which directly
protect systems and equipment essential for detect and avoid and
flight control functions must withstand, without penetration, the
impact of a 0∙91 kg (2 lb) bird when the velocity of the aeroplane
relative to the bird along the aeroplane’s flight path is equal to the
aeroplane’s maximum approach flap speed.
22
What is the Problem with the Existing
Codes? - Example
AMC 25.1309 7 (a) 5 Classification of Severity of
failure mode:
Catastrophic: Failure Conditions, which would result in
multiple fatalities, usually with the loss of the
aeroplane. (Note: A “Catastrophic” Failure Condition
was defined in previous versions of the rule and the
advisory material as a Failure Condition which would
prevent continued safe flight and landing.)
23
Hence
Discussions with CAA
Regulatory discussions at the JARUS forum, NAAs
plus FAA
Led to:
CAA document AMC UAS 1309
24
AMC UAS 1309 Failure Mode
Severities Example:
Catastrophic
Failure Conditions that could result in multiple fatalities
25
AMC UAS 1309 Main Points
Failure Mode Severities vs. Probability:
Classification
of failure
conditions
Allowable
qualitative
probability
Allowable
Quantitative
probability per
Flight Hour on
the order of.
No safety
Effect
No Probability
Requirement
No Probability
Requirement
Minor
Major
Hazardous
Catastrophic
Probable
Remote
Extremely
Remote
Extremely
Improbable
<10-3
Note 1
<10-5
<10-7
<10-9
Non Detect and Avoid Equipped.
Detect and Avoid Equipped. Note 2
Software/CEH
Dev Level D
Software/CEH
Dev Level B
Software/CEH
Dev Level C
Software/CEH
Dev Level A
26
AMC UAS 1309 Main Points:
With D & A; Integrity of Airframe and Complex
Flight Mgt System:
Aircraft
Category
CS/FAR-25
CS/FAR-23
CFMS Integrity
including DA
requirement (Fig 1.0)
Airframe Integrity
(analogous CS/FAR
to UAS code)
requirement similar
to
AMCAMCUAS
1309
Points:
UAS.1309
(10 ) Main
CS/FAR-25.1309
(10 )
-9
-9
Integrity delta
No CFMS/ Airframe systems integrity delta
Increasing CFMS/ airframe systems integrity
With D & A; Integrity of AC-23.1309-1D
Airframe and Complex
Flight Mgt System:
delta dependent on class
AMC UAS.1309 (10-9)
CS/FAR-23
AMC UAS.1309 (10-7)
Mitigated case
AC-23.1309-1D
Only allowed if mitigation is accepted by the
authorities
CS/FAR-29
AMC UAS.1309 (10-9)
CS/FAR-29.1309
Medium CFMS/ airframe systems integrity
delta
CS/FAR-27
AMC UAS.1309 (10-9)
CS/FAR-27.1309
Large CFMS / airframe systems integrity
delta
CS/FAR-27
AMC UAS.1309 (10-7)
Mitigated case
CS/FAR-27.1309
Only allowed if mitigation is accepted by the
authorities
CS-VLA / VLR and
others
AMC UAS.1309 (10-9)
CS-VLA / VLR.1309
Very large CFMS / airframe systems integrity
delta
CS-VLA / VLR and
others
AMC UAS.1309 (10-7)
Mitigated case
CS-VLA / VLR.1309
Only allowed if mitigation is accepted by the
authorities
27
Other Codes:
EU-OPS Part Ops
Operational requirements
Part FCL
Looking at requirements for Flight Crew Licensing to understand
what the human pilot is expected to do
28
UAS Airframe Codes Issues with
the STANAG - Example 3
USAR1772 Part time data
USAR1725 Powerplant data etc..
Puts a lot of emphasis on the UAS Pilot, and this is a
problem because:
Estimated Reliability of the data link is 10-3
Not sufficient for critical functions
29
Firescout Incident
According to Capt. Tim Dunigan, Fire Scout
program manager, the helicopter was at 1,700 feet
AGL 75 minutes into a test flight from its base,
Naval Air Station Pax River in Southern Maryland,
when operators temporarily lost communications
with the unmanned rotary aircraft.
“The MQ-8 Fire Scout experienced lost link and proceeded 23 miles
north/northwest out of Pax River, still about 40 miles south of DC area in
northern St. Mary's county, Maryland, into National Capital Region
airspace,” he said. “The operator team shifted to other Ground Control
Station, restoring link and successfully commanding vehicle to recover at
Webster Field. The aircraft returned to Webster Field safely without
injuries, and without damage to the aircraft or vessel.”
30
Firescout Incident
He added that the MQ-8B Fire Scout has flown more
than 1,000 flight hours since December 2006. i.e. 10-3
Conclusions:
The integrity of the UAS must be on board the UAV
The relevance of the pilot is limited
Autonomous logic is necessary...
...including ‘interrupts’ to prevent pilot error
31
Concluding
Firescout incident highlights the risks
This paper proposes a means for dealing with autonomous
UAS regulation and ADMS safety assurance
Cross fertilisation of results from SSEI Programme to
ASTRAEA, with opportunity to flight test ADMS
behaviours on Cranfield A/C
Flow through to civil and military UAS programmes
worldwide
Civil regulators, CAA and FAA, developing regulations
with industry cooperation
What will MoD and the MAA do?
32