SBAS and GBAS Integrity for NonAviation Users: Moving Away from
"Specific Risk"
Sam Pullen, Todd Walter, and Per Enge
Stanford University
spullen@stanford.edu
ION ITM 2011
San Diego, CA.
25 January 2011
Motivation (1): SBAS and GBAS for NonAviation Users
• Where augmentation signals can be received,
SBAS and GBAS benefits are available to all
users.
• However, integrity algorithms in airborne MOPS
are designed to support specific aviation
applications.
– Resulting integrity protection levels are not wellsuited for other classes of users
• Correcting this would increase the attractiveness
of SBAS and GBAS to non-aviation transport
users (auto, rail, marine) and others.
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
2
Motivation (2): Accuracy and Integrity
Illustrative
example – not
to scale or
direction
HPL (nonaviation
application)
95%
HPE
HPL (per
MOPS)
• Accuracy bounds (e.g., 95% vertical position error, or VPE) can
be measured and modeled with high precision
• Integrity bounds (e.g., 10-7 vertical protection level, or VPL)
cannot be
–
–
–
–
25 January 2011
Lack of sufficient measurements
Flaws in Gaussian extrapolations to low probabilities
Dependence on details of failure models and assumptions
Too little is known; too much is uncertain…
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
3
WAAS VPE vs. VPL from FAA PAN Data
(3rd Qtr 2010: July – Sept.)
Source: WAAS PAN
Report #34, Oct. 2010.
VPL (m)
http://www.nstb.tc.faa.gov/
DisplayArchive.htm
Max. VPE  7 m
(at Barrow, AK)
25 January 2011
95% VPE
 1.2 m
99% VPE
Integrity
1.6 m
for Non-AviationVPE
Users:(m)
Moving Away from "Specific Risk"
4
WAAS Reference Station Classifications
(for this study only)
Figure source: FAA GNSS Press Kit
http://preview.tinyurl.com/4ofdzz4
18 Remote Stations
13 Outer Stations
7 Inner Stations
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
5
Max. VPE and VPL from FAA PAN Data
(1 Jan. 2004 – 30 Sept. 2010)
Worst Case Between “Inner” and “Outer” WAAS Stations -> “InOut” Set
50
VPE or VPL (meters)
45
40
Max. VPL
35
30
25
20
15
10
5
0
Max. VPE
95% VPE
10
15
20
25
30
35
Quarterly PAN Report Number (8 – 34)
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
6
Max. HPE and HPL from FAA PAN Data
(1 Jan. 2004 – 30 Sept. 2010)
Worst Case Between “Inner” and “Outer” WAAS Stations -> “InOut” Set
45
As expected, both
HPE and HPL are
significantly lower
than VPE and VPL.
40
HPE or HPL (meters)
35
One unusual
result: 12 m
error at
Cleveland in
Spring 2005
(correct
number?)
25 January 2011
30
Max. HPL
25
20
15
10
Max. HPE
5
95% HPE
0
10
15
20
25
30
Quarterly
PAN Users:
Report
Number
– 34)Risk"
Integrity
for Non-Aviation
Moving
Away from(8
"Specific
35
7
Ratio of Max. VPL and Max. VPE from
FAA PAN Data (“InOut” Station Set)
9
Less error reduction after
PAN #20 (March 2007).
VPL / VPE Ratio for Max. Cases
8
7
6
Mean Ratio
= 5.38
5
4
Noticeable improving trend  likely
due to error reduction at individual
WAAS reference stations.
3
2
1
0
10
15
20
25
30
35
Quarterly PAN Report Number (8 – 34)
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
8
Ratio of Max. HPL and Max. HPE from
FAA PAN Data (“InOut” Station Set)
10
HPL / HPE Ratio for Max. Cases
9
Unusual error at
Cleveland (if
correct) just
barely exceeded
by HPL.
25 January 2011
8
7
Mean Ratio
= 5.21
6
5
4
3
Weaker but visible
improving trend –
more variability.
2
1
0
10
15
20
25
30
Quarterly PAN Report Number (8 – 34)
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
35
9
How Many Samples Were Collected?
All validated PAN data from 1 Jan. 2004 to 30 Sept. 2010
Assume data
correlated over
600 sec (10 min)
7.1  10-6
independent
samples
25 January 2011
Assume data
correlated over
150 sec (~ one
CAT I approach)
2.8  10-7
independent
samples
Assume data
correlated
over 30 sec
1.4  10-8
independent
samples
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
4.25  10-9 sec
(49,324.6 days)
(105.04 years)
10
Average vs. Specific Risk Assessment
• Average Risk (my definition): the probability of unsafe
conditions based upon the convolved (“averaged”)
estimated probabilities of all unknown events.
– Probabilistic Risk Analysis (PRA) is based on this procedure
– Risk aversion and value of information (VOI) are applied to the
outputs of PRA  integrity risk requirements, alert limits
• Specific Risk (my definition): the probability of unsafe
conditions subject to the assumption that all (negative
but credible) unknown events that could be known
occur with a probability of one.
– Evolved from pre-existing FAA and ICAO safety standards
– Risk aversion and VOI and buried inside specific risk analysis
– Results (risk and protection levels) are inconsistent with PRA
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
11
Simplified Example:
Ionospheric Spatial Decorrelation (1)
Severe Ionospheric Storm Observed
over CONUS on 20 November 2003
20:15 UT
25 January 2011
21:00 UT
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
12
Simplified Example:
Ionospheric Spatial Decorrelation (2)
• Using PRA, estimated “prior” probabilities of severe
decorrelation are combined with the likelihood of
SBAS or GBAS mitigation to derive resulting user risk.
– Prior probabilities need not be known precisely
– Benefits of improved mitigation (“better information”) appear
naturally as lower integrity risk.
• Under FAA interpretation of Specific Risk, worst-case
iono. delay gradient is “credible” and thus is assigned
a probability of one.
– Worst-case for GBAS (CAT I): an extremely large gradient that
escapes detection by “matching speed” with ground station
» This differs in real time for each site and GNSS geometry
– Worst-case for SBAS (LPV): a very large gradient that is just
small enough to avoid detection by master station
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
13
Simplified Example:
Ionospheric Spatial Decorrelation (3)
Simulated results for Memphis GBAS impacted by severe ionospheric gradient
(RTCA 24-SV GPS, 6-km, User-to-ground separation, 1 and 2-SV impacts)
0.14
Most errors are exactly zero due to ground
detection and exclusion, but all zero errors
have been removed from the histogram.
0.12
0.1
Most plotted (non-zero)
errors are below 10 m even
under severe conditions.
PDF
0.08
0.06
Worst-case
error, or “MIEV”,
is  41 m
0.04
0.02
0
25 January 2011
0
5
10
15
20
25
30
35
User Vertical Position Error (meters)
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
40
45
14
Benefits of an “Average Risk” Approach
(Potential SBAS PL Reduction)
40
35
95% VPL
VPL or HPL (meters)
30
Max. 95% PLs
among stations
in CONUS
(“InOut” set)
25
95% HPL
20
15
Conservative
reduction factors
from PAN data:
Adjusted VPL
10
5
VPL / 4.0
Adjusted HPL
HPL / 2.5
From reports
since Jan. 2008
0
24
25
26
27
28
29
30
31
32
33
34
PAN Report Number
•
“Average risk” approach supports large reductions in HPL and VPL
implied by WAAS PAN data, pending more complete database analysis.
•
Use “full-scale” PRA to re-assess “rare-normal” and faulted errors.
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
15
A Combined “Average/Specific” Risk
Approach
• Depending on user and decision maker risk aversion,
separate “average risk” and “specific risk” integrity
requirements could be issued.
– Both apply at all times  one or the other will tend to
dominate for a particular application.
• For example: 10-7 integrity risk per operation
(“average”) plus requirement that a worst-case
undetected condition cannot increase the total vehicle
loss risk by more than a factor of 10.
– For aircraft case, factor of 10 increase in total risk equates to
specific risk requirement of 10-5 per operation for nav. system
(more strictly, 9  10-6)
– Specific factors for each vehicle and application would vary.
– There is no “correct” degree of risk aversion.
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
16
Summary
• Existing integrity assurance procedures for SBAS and
GBAS are unique to aviation and its history and may
not be suitable for other users.
• SBAS (and GBAS) data analysis suggests that 10-7
HPL and VPL can be greatly reduced if “average risk”
approach is taken.
– Examination of past data is useful, but more thorough PRA
analysis should be conducted.
• If worst-case elements of risk assessment are still
desired, an average/specific risk mixture can be used.
– This flexible “mixture” capability should satisfy almost any
level of user and decision maker risk aversion.
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
17
Backup Slides follow…
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
18
WAAS VPE from FAA PAN Data
(3rd Qtr 2010: July – Sept.)
No. of Samples
Source: WAAS PAN
Report #34, Oct. 2010.
http://www.nstb.tc.faa.gov/
DisplayArchive.htm
Max. VPE 
7 m at
Barrow, AK
Meas. from 37
WAAS stations
25 January 2011
Integrity for Non-AviationVPE
Users:(m)
Moving Away from "Specific Risk"
19
(from PAN #34)
Example Error Table from PAN #34
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
20
Max. VPE and VPL from WAAS PAN Data
(1 Jan. 2004 – 30 Sept. 2010)
(all numbers are in meters)
PAN
Report
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Inner
Inner
Inner WRS VPE_95% VPE_Max
Chicago
Dallas
Dallas
Dallas
Dallas
Dallas
Dallas
Dallas
Albuquerque
Dallas
Dallas
Dallas
Dallas
Dallas
Denver
Kansas City
Memphis
Denver
Denver
Denver
Chicago
Cleveland
Dallas
Denver
Cleveland
Denver
Memphis
Inner
VPL
Outer WRS
Minneapolis
Minneapolis
Minneapolis
Minneapolis
Salt Lake City
Minneapolis
Oakland
Minneapolis
Minneapolis
Oakland
Oakland
Miami
Seattle
Miami
Oakland
Minneapolis
Seattle
Seattle
Oakland
Seattle
Miami
Miami
Wash DC
Miami
Miami
Miami
Seattle
1.086
1.442
1.388
1.371
1.298
1.504
1.141
1.469
0.934
1.202
1.210
1.281
1.184
1.028
1.281
0.945
0.889
0.800
1.100
1.022
0.852
1.041
1.001
1.108
1.001
0.938
1.048
7.541
8.191
8.722
8.280
9.301
9.457
6.426
6.719
8.195
7.893
6.888
6.879
6.040
5.064
3.975
5.016
4.800
3.401
5.025
4.571
4.046
4.664
4.459
5.045
4.143
4.754
4.070
49.612
39.956
43.829
31.969
33.699
28.399
26.887
24.612
24.246
34.771
37.435
35.097
30.050
26.238
34.868
24.232
24.742
27.877
28.390
25.254
21.989
24.292
50.101
25.872
28.377
36.569
13.567
Ave
1.132
6.058
Max
1.504
9.457
25 January 2011
Outer
Outer
VPE_95% VPE_Max
Outer
VPL
Remote
Remote Remote
Remote WRS VPE_95% VPE_Max
VPL
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Fairbanks
Fairbanks
Kotzebue
Fairbanks
Puerto Vallarta
Tapachula
San Juan
S.J. Del Cabo
Iqaluit
Fairbanks
Barrow
Iqaluit
Iqaluit
Iqaluit
Barrow
Barrow
1.710
1.695
1.790
1.501
1.155
1.765
1.706
1.956
1.157
1.273
1.228
1.657
0.886
1.231
1.043
1.067
0.801
0.766
1.061
0.915
2.041
1.537
1.124
1.612
2.005
1.298
0.849
9.133
7.794
7.376
8.034
8.581
12.756
7.931
7.439
8.002
6.385
7.296
6.913
5.858
5.160
4.119
5.029
4.273
4.553
4.808
4.972
4.462
4.384
4.589
4.240
4.738
4.516
4.920
37.430
40.806
32.210
37.367
47.939
44.758
37.235
28.722
31.380
47.296
46.769
46.396
22.705
37.664
30.970
32.445
20.643
23.230
23.802
20.294
28.787
29.033
33.014
24.229
26.618
30.514
37.557
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
1.080
1.062
1.183
1.118
1.466
1.917
1.300
1.138
2.087
0.997
1.128
1.731
1.766
1.869
1.245
1.165
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
7.395
22.492
37.308
9.255
5.854
7.347
5.859
5.566
6.977
8.018
6.733
9.768
7.556
8.106
7.700
6.975
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
40.632
33.620
39.902
34.793
40.937
44.259
31.842
31.806
28.362
35.478
26.198
42.103
27.882
45.033
38.500
44.427
30.849
1.364
6.232
33.326
1.391
10.182
36.611
50.101
2.041
12.756
47.939
2.087
37.308
45.033
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
21
95% and Max. VPE from FAA PAN Data
(1 Jan. 2004 – 30 Sept. 2010)
50
Note: VPL always bounds VPE.
Vertical Position Error (meters)
45
Severe iono. scintillation in
Alaska in March and May 2007
(user receiver should prevent)
40
35
VAL for LPV
Remote Stations
30
25
Max. VPE
20
15
95% VPE
Outer Stations
10
Inner Stations
5
0
10
15
20
25
30
35
Quarterly PAN Report Number (8 – 34)
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
22
WAAS VPE vs. VPL in CONUS (2003 – 2006)
(from Wanner, et al, 2006)
Ratios:
6.9
6.4
6.8
6.5
6.0
6.8
6.8
Vertical Position Error (meters)
VPL
99.99% VPE
99.99%
VPE
99.9%
VPE
99% VPE
95% VPE
Mean VPE
1s VPE
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
23
WAAS Max. VPE in CONUS (2003 – 2006)
(from Wanner, et al, 2008)
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
24
An “Average Risk” Approach to SBAS
(and GBAS) – word version
• Data imply an “average risk” equivalent VPL for WAAS
~ 4 – 5 times lower than current value.
• Re-assess “rare-normal” and faulted error models and
data to build a “certifiable” safety case.
– Multiple rare-normal (“fault-free”) models built from existing
data to incorporate remaining uncertainty
– All fault-mode analyses follow the same approach:
• Estimate prior fault probabilities and probability uncertainties.
• Simulate all significant variations of each fault type rather than
“worst case” focus  convolve with prior dist. to estimate risk.
– Faults whose impact is driven by worst-case scenarios
(ionosphere, signal deformation) will become less important.
– Multiple-fault scenarios neglected as too improbable may
become more important, as probabilistic weighting of risk
may show that fault-combination cases are non-negligible.
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
25
A Combined “Average/Specific” Risk
Approach (1)
Derived from FAA “Hazard Risk Model” (1) and
Simplified Aircraft Accident Risk Breakdown (2)
Major
Hazardous
Catastrophic
(Slight risk of aircraft
loss/pilot challenged)
(Risk of a/c loss; Severe
loss of safety margin)
(Likely a/c hull
loss)
10-5
~ 1%
10-7
10-6
10-9
10-7
~ 10%
10-9
~ 1%
(~ 100 systems)
Overall a/c
loss prob.
Loss prob. due to
equipment failure
Loss prob. due to
GNSS nav. failure
(1)
FAA System Safety Handbook, 2008. http://www.faa.gov/library/manuals/aviation/risk_management/ss_handbook/
(2)
R. Kelly and J. Davis, “Required Navigation Performance (RNP),” Navigation, Spring 1994.
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
26