W H I T E
P A P E R
R
Safer Skies
Baggage Screening and Beyond
With Supporting Analyses
Gary Kauvar, Bernard Rostker, Russell Shaver
National Security Research Division
This research was conducted by RAND as part of its continuing program of selfsponsored research. RAND acknowledges the support for such research provided by
the independent research and development (IR&D) provisions of RAND’s contracts
for the operation of its Department of Defense federally funded research and
development centers.
ISBN: 0-8330-3200-3
RAND white papers are authoritative publications that draw on a strong body of
prior research to summarize key findings relevant to pending decisions or policy
problems. White papers are reviewed by RAND’s corporate management to assure
that they adequately represent both RAND’s best work in the subject as well as
significant differences of opinion.
RAND is a nonprofit institution that helps improve policy and decisionmaking
through research and analysis. RAND ® is a registered trademark. RAND’s publications do not necessarily reflect the opinions or policies of its research sponsors.
© Copyright 2002 RAND
All rights reserved. No part of this book may be reproduced in any form by any
electronic or mechanical means (including photocopying, recording, or information
storage and retrieval) without permission in writing from RAND.
Published 2002 by RAND
1700 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138
1200 South Hayes Street, Arlington, VA 22202-5050
201 North Craig Street, Suite 202, Pittsburgh, PA 15213
RAND URL: http://www.rand.org/
To order RAND documents or to obtain additional information, contact Distribution
Services: Telephone: (310) 451-7002; Fax: (310) 451-6915; Email: order@rand.org
iii
Preface
This paper is the result of an examination of plans for implementing explosive
detection systems for checked baggage at all U.S. airports, and it contains
suggested changes to the existing plans for implementation of the systems.
The work was funded by RAND’s independent research and development
funds. It should be of interest to those concerned about whether the Federal
Aviation Administration’s (FAA’s) plans (now part of the Aviation and
Transportation Security Act) appropriately respond to the security threats
against our air transportation system. It should also interest those who wish
to better understand how to address the natural tension between unfettered
access to our commercial aircraft and enhanced transportation security. The
research and the writing of this paper were completed in March 2002, and
hence it does not reflect subsequent developments in this field. Its publication
serves to document RAND’s approach to this problem as well as its analytical
findings.
This study was conducted by RAND as part of its continuing program of selfsponsored research. We acknowledge the support for such research provided
by the independent research and development provisions of RAND's
contracts for the operation of its Department of Defense federally funded
research and development centers: Project AIR FORCE (sponsored by the
U.S. Air Force), the Arroyo Center (sponsored by the U.S. Army), and the
National Defense Research Institute (sponsored by the Office of the Secretary
of Defense, the Joint Staff, the unified commands, and the defense agencies).
This research was overseen by RAND's National Security Research Division
(NSRD). NSRD conducts research and analysis for the Office of the Secretary
of Defense, the Joint Staff, the unified commands, the defense agencies, the
Department of the Navy, the U.S. intelligence community, allied foreign
governments, and foundations.
v
Table of Contents
Preface ..................................................
iii
Figures ..................................................
vii
SAFER SKIES: BAGGAGE SCREENING AND BEYOND ............
The Importance of Getting It Right .........................
RAND’s Approach .....................................
Problems with the Federal Legislation .......................
An Alternative Approach.................................
Profiling: A Strategy for Reducing the Problem to a
Manageable Level ...................................
Conclusion ...........................................
1
1
2
3
5
6
7
Appendix
A.
B.
C.
ANALYSIS OF DEPLOYMENT REQUIREMENTS FOR
BAGGAGE SCANNING EQUIPMENT ......................
Introduction ..........................................
Assessing Plans for Explosive Detection System Machines ........
Assumptions and Assessment Process........................
General Results for EDS Machines ..........................
General Comments on Criteria for Sizing the Buy...............
Case 1: All Flights at DFW, Circa 1998.......................
Case 2: All Flights at DFW, Circa 2010.......................
Case 3: All American Airlines Flights at DFW .................
Case 3A: American Airlines Flights at DFW Partitioned Into Six
Individual Scanning Locations..........................
Comparisons Across the Three Cases ........................
Estimating a Total Buy Size for All Airports ...................
Would the Results Have Been Worse If the Input Variables Were
Treated as Stochastic?.................................
The Potential Role of Passenger Profiling in Reducing the Buy.....
Sizing the Buy .........................................
Some Additional Considerations ...........................
Trace Scanning Options ..................................
Option 1: Single-Stage ETD Scanning Options .................
Option 2: A Combined EDS/ETD Scanning System. ............
Summary.............................................
9
9
10
13
21
21
23
29
37
40
41
43
44
44
48
49
51
53
57
58
SIMULATION MODELING OF AIRPORT SECURITY
OPERATIONS .........................................
Modeling Approach ....................................
Benefits ..............................................
Simulation Output ......................................
60
60
61
62
TRUSTED TRAVELER PROGRAM .........................
63
vii
Figures
A.1.
A.2.
A.3.
A.4.
A.5.
A.6.
A.7.
A.8.
A.9.
A.10.
A.11.
A.12.
A.13.
A.14.
A.15.
A.16.
A.17.
A.18.
A.19.
A.20.
A.21.
A.22.
A.23.
A.24.
Concept of Operations Used for Baggage Scanning ...........
Check-In Baggage Demand at DFW, 1998 ..................
Check-In Baggage Demand at DFW, 2010 ..................
Actual and Spread Demand for 1998 DFW Case..............
Typical Maximum Delay Matrix .........................
EDS Deployment Options Considered, All Flights at DFW,
1998 ..............................................
Bag Screening Delays as a Function of Time of Day, All
Flights at DFW, 1998 ..................................
Maximum Delays per Number of EDS Machines in Operation,
All Flights at DFW, 1998 ...............................
Passenger Arrival Time for All Bags to Get Onto Aircraft, All
Flights at DFW, 1998 ..................................
Planned Time of Arrival According to Passenger Propensity to
Accept Risk, All Flights at DFW, 1998 .....................
EDS Deployment Options Considered, All Flights at DFW,
2010 ..............................................
Bag Screening Delays as a Function of Time of Day, All
Flights at DFW, 2010 ..................................
Maximum Delays per Number of EDS Machines in Operation,
All Flights at DFW, 2010 ...............................
Passenger Arrival Time for All Bags to Get Onto Aircraft, All
Flights at DFW, 2010 ..................................
Planned Time of Arrival According to Passenger Propensity to
Accept Risk, All Flights at DFW, 2010 .....................
Sensitivity of Results to Reductions in Equipment Reliability,
All Flights at DFW, 2010 ...............................
Sensitivity of Results to Increases in Load Factor During Peak
Afternoon Demand, All Flights at DFW, 2010 ...............
Sensitivity of Results to Changes in the False Positive
Detection Rate, All Flights at DFW, 2010 ...................
Benefits of Flexible Tier 1/2 Deployment for Countering
Uncertainty in False Positive Detection Rate, All Flights at
DFW, 2010 .........................................
Check-In Baggage Demand for Single Airline at DFW (41
Percent of Total Bags), 2010 .............................
EDS Deployment Options Considered, Single Airline at DFW,
2010 ..............................................
Maximum Delays per Number of EDS Machines in Operation,
Single Airline at DFW, 2010.............................
Passenger Arrival Time for All Bags to Get Onto Aircraft,
Single Airline at DFW .................................
EDS Deployment Options Considered, Single Airline
Partitioned Into Six Separate Scanning Locations .............
16
18
18
19
20
24
26
27
28
29
30
32
33
34
34
35
35
36
37
38
39
39
40
41
viii
A.25. Comparison of 2010, 1998, and Single-Airline Deployment
Needs..............................................
A.26. Stochastic Results Versus Expected Value Results, All Flights at
DFW, 2010 ..........................................
A.27. Effect of Reduced Number of Bags Needing to Be Scanned on
Maximum Delays .....................................
A.28. EDS Deployment Options When Passenger Profiling Reduces
Total Bags in Queue, 2010 ...............................
A.29. Comparison of Profiling Results Across Criteria Categories and
Total Demand........................................
A.30. Maximum Queuing Delays Versus System Scan Rate...........
A.31. Point and Expected Values for Maximum Delays as a Function
of Total EDS Machine Buy Size ...........................
A.32. Sensitivity of Performance of Different Concepts of Operation
to System False Positive Detection Rate.....................
A.33. Experimental Experience on Trace Detection Performance in
the Field ............................................
A.34. Maximum Delays Versus Minimum Required Number of ETD
Machines in Tier 2 ....................................
A.35. Required Second-Stage ETD Stations Versus Maximum
Delays .............................................
A.36. Maximum Delays Versus Minimum Required Number of
Trace Detection Machines in Tier 2 ........................
B.1. Sample Output from Discrete-Event Simulation ...............
B.2. Samples of Computer Simulation Animations ................
42
45
46
47
48
50
51
52
55
55
56
57
62
62
1
Safer Skies: Baggage Screening and Beyond
On November 19, 2001, Congress passed the Aviation and Transportation
Security Act.1 The legislation highlights the importance of baggage screening
as part of a comprehensive program to increase airport security. Specifically,
it mandates that by December 31, 2002, 100 percent of checked baggage at all
the nation’s airports must be screened for explosives. RAND examined
existing plans for implementing explosive detection systems (EDSs) for
checked baggage at all airports in the United States and suggested changes
that might be advisable.
An EDS is a very large scanning machine that uses magnetic resonance
imaging (MRI) technology to generate three-dimensional images of the
contents of individual bags.2 EDS machines are expensive and take up a great
deal of room. While our study was in progress, the newly created
Transportation Security Administration (TSA) became interested in the
possible use of explosive trace detection (ETD) machines, which “sniff”
molecules that adhere to a swab run over the surface of a piece of baggage.
ETD machines are less expensive and take up less room than EDS machines
do, but they are less sensitive and less accurate in detecting explosive
materials. To achieve detection rates comparable to those for EDS machines,
samples must be taken from inside each bag. We also evaluated the use of
ETD machines, applying the same methodology we applied to the EDS
machines. Recently, it has been suggested that, while all bags will be
scanned, only some of them will be (randomly) selected to be subject to the
full ETD screening. While this selectivity may help increase safety while
easing delays, it does not change our basic conclusions or recommendations.
The Importance of Getting It Right
In 2001, 710 million passengers traveled on civil airliners in the United States.
For these passengers, baggage screening is a necessary part of an overall plan
for airport and airline security. History has shown, however, that passengers
want security without having to undergo substantial inconvenience. Since
________________
1 Public Law No. 107-71.
2 MRI is the technology that generates CAT (computerized axial tomography) scanner
images of the human body.
2
September 11, TV news programs have shown airline passengers who profess
to be willing to endure any inconvenience in the name of security. But these
programs have not shown the many people who have decided not to fly
because of long and intrusive security procedures. Congress passed the
Aviation and Transportation Security Act partly out of fear that the public
would lose confidence in the safety of flight and thus would cease flying
altogether. The remedy for that fear—baggage screening, together with
enhanced passenger screening—should not make flying so onerous as to
produce the same result.
An analysis of the cost of providing security without massively
inconveniencing the flying public suggests that this cost is small compared
with the losses commercial aviation will suffer if potential passengers elect not
to fly. Some people, especially those who fly infrequently, will tolerate
substantial delays at airports if they think that real security is the result. But
those for whom time is valuable and who may have a reasonable alternative
to commercial air travel are considerably less likely to fly in such
circumstances. Substantial loss of these travelers would seriously harm the
aviation industry—and the nation’s economy. Thus the cost of “getting it
right” in terms of balancing security needs against inconvenience would be
more than compensated by the financial returns resulting from a stronger
national economy.
RAND’s Approach
Before September 11, the Federal Aviation Administration (FAA) planned to
deploy EDS machines nationwide using a government-imposed, “top-down”
approach whereby the federal government would provide EDS machines to
individual airports. This initial plan, which was not coordinated with
individual carriers or the airports where the equipment would be installed,
called for deployment by 2013. After September 11, the FAA advanced that
date to 2005 and prepared a deployment plan showing how the new timeline
might be met. Federal legislation spurred by the terrorist attacks on
September 11 then called for a further compression of the FAA’s schedule, and
machines for screening 100 percent of checked baggage were to be deployed
by December 2002. Again, the approach was top-down and did not
incorporate input from air carriers or local airports.
To determine the feasibility of this deployment schedule, we examined the
FAA’s analysis to better understand the calculations and assumptions that
supported both the 2005 and the new deployment schedule. We also visited
3
two major airports: Dulles International (Washington, D.C.) and Dallas–Fort
Worth. At Dulles, we examined the process that airports and airlines would
have to go through to meet the 2002 deployment deadline. At Dallas–Fort
Worth, we discussed the possible EDS deployments with officials of a major
national airline and a team of independent experts in the analysis and design
of airport facilities. We then built a simple queuing model to test the
assumptions and robustness of the FAA projections of the number of machines
that would be needed to screen 100 percent of checked baggage at all
airports.3
Problems with the Federal Legislation
Our analysis revealed that the FAA’s 2005 deployment plan was not based on
an adequate assessment of the consequences of fielding an effective baggage
screening system of the sort envisioned by Congress. For example, the plan
did not adequately account for the demands that peak loads place on the
baggage handling system, nor did it account for the actual performance and
reliability of machines already deployed. Moreover, it was not clear how the
available EDS equipment would be allocated among the various airports or
what metric was to be used to ensure that the national aviation transportation
system was running effectively. Most important, however, the deployment
plan did not consider the severe space constraints that airports would have to
overcome if they were to field all the new EDS equipment needed to screen
every bag.
We identified six main problems with the deployment schedule set out in the
Aviation and Transportation Security Act:
1.
A rudimentary queuing model showed that the FAA estimate of the
number of EDS machines needed to screen 100 percent of checked
baggage for the major airports was too low by a factor of about 2.5.
2. The allocation of EDS equipment to the nation’s airports has not been
tested against a model of the national transportation system to ensure that
bottlenecks will not develop during the implementation period or that the
requirements of the hub system can be met.
3.
Most disturbing, a top-down analytic approach is inappropriate for
conducting a thorough analysis of the requirements for solving the
________________
3 See Appendix A for a quantitative assessment of potential delays in baggage handling
and their sensitivity to the number of machines deployed, level of demand, machine
reliability, and other factors.
4
problems of airport baggage screening. A more appropriate approach—
one that has been used by a number of airports in planning the design of
new terminals—is a stochastic simulation incorporating a realistic and
detailed representation of the movement of passengers and baggage
through the airport.4 Indeed, simulations already exist for 24 of the 25
largest airports. The stochastic simulations we examined, which were
provided by the TransSolutions Corporation, showed how critical each
airport’s design is to the number and placement of EDS machines at that
airport. Each of the 453 commercial airports in the United States presents
a unique challenge to baggage system designers.
4. Although the FAA did initiate a “go-team” to determine what could be
done to increase the number of approved EDS machines, the deployment
plan does not take into account the ability of industry to produce FAAcertified machines.
5. The FAA’s “top-down” approach does not adequately consider local
constraints, such as the size of the airport terminal. It is space constraints,
not machine availability, that is the proverbial long pole in the tent. Until
suitable airport facilities are constructed, many of the EDS machines now
being acquired at a highly accelerated rate cannot be installed.
6. The shortcomings of deploying EDS machines also hold for ETD
machines. While the latter are much smaller than EDS machines, they are
also less reliable in detecting bombs. Tests show that for ETD machines to
achieve results that even roughly approximate those provided by EDS
machines, trace samples must be taken from inside each bag. The process
of opening every bag and taking multiple samples from each also requires
additional airport floor space—space that might not be available at some
airports without additional construction. Use of the ETD machine would
likely also increase processing times. In sum, ETD machines will not
likely provide an easy answer.
We concluded that the deployment plan proposed in the Aviation and
Transportation Security Act for EDS machines, particularly because of its topdown orientation, was not workable and that replacing EDS with ETD
machines will not eliminate the problem. However, a number of steps can be
taken to improve system performance. These improvements will not achieve
the congressional mandate for 2002, but they will go a long way toward
_________________
4 See Appendix B for a general description of the properties and capabilities of a detailed
simulation model.
5
providing increased levels of airport security through a more effective
program of baggage screening.
An Alternative Approach
As an alternative to the top-down approach of the Department of
Transportation’s (DOT’s) deployment plan, we propose a bottom-up approach
that will empower airports and airlines to work together to solve what is
essentially a local problem. Moreover, we recommend that the problemsolving teams follow the example of the Dallas–Fort Worth Airport in using
the most appropriate stochastic simulation models to provide the best possible
forecasts and projections. With such an approach, the federal government
will play a different but no less important role. That role will be to organize,
coordinate, and ensure the quality of a bottom-up system that fully
enfranchises local airports and airlines in developing baggage-screening
solutions that can work in the field. The government should
•
Establish the standards for machine and system performance.
•
Participate in local partnerships that are designing local solutions.
•
Integrate the local plans into a national architecture.
•
Evaluate the effectiveness of each airport’s security systems.
•
Set parameters now for the longer term so that airports will be able to
incorporate the new requirements into their modernization/expansion
programs, many of which are now on hold.
•
Test proposed options against a model of the national transportation
system to ensure that bottlenecks do not develop during the
implementation period and that the requirements of a hub system can be
met.
A bottom-up approach should be implemented immediately, giving the local
partnerships a period of perhaps 60 days to report to the DOT concerning
their requirements for government-funded EDS machines and to provide an
estimate of the facility modifications needed to incorporate EDS into the
airport infrastructure.
With this approach, the DOT will still have the responsibility for determining
the timeline for airports to receive EDS equipment consistent with the plans
submitted by the partnerships. To do so, the DOT will need to model and
develop metrics to address the overall performance of the civil aviation
system, particularly the impact that new security procedures have on the
6
functionality of hub airports and, ultimately, on the performance of the entire
air transportation system as a vital element in the nation’s economy.
Profiling: A Strategy for Reducing the Problem to a
Manageable Level5
Even if the best planning tools are available, a bottom-up approach is used, all
the airports and airlines work productively with the DOT, and maximum
production of new EDS equipment is achieved—even then, the 2002
congressional deadline for the fielding of EDS machines almost certainly
cannot be met. What is needed is an interim measure, a way of ensuring that
the existing baggage scanning capacity focuses on those bags most likely to
pose a threat. This can be done by adopting baggage handling procedures
that have been proven around the world to increase aviation security without
overburdening the traveling public.
One of the most effective of these procedures would be expanded use of the
Computer Assisted Passenger Profiling System (CAPPS). For example, airport
security could use a so-called “trusted traveler” program to focus on
identifying the bags least likely to pose a threat. This approach is consistent
with generally accepted standards of nondiscriminatory profiling used by
civil aviation authorities throughout the world. The procedure would be
based not on gender, race, or national origin, but rather on “selecting”
passengers about whom a great deal is known and who exhibit behaviors that
keep them off any likely-threat list. U.S. citizens who have detailed
background investigations on record with the government would be obvious
trusted traveler candidates. Numerous other indicators exist as well, many of
which could be used successfully in a nondiscriminatory manner.
Such expanded use of CAPPS in no way alters the desirability or utility of
using CAPPS to positively identify likely threats. Civil aviation authorities
should have up-to-date access to the entire range of information that can be
provided by law enforcement and intelligence organizations about people
who are on “watch lists,” have overstayed their visas, or have drawn
attention to themselves for other reasons. Similar systems are used in Israel,
which is generally believed to have the world’s most secure civil aviation
system. While it would be impractical to try to import Israel’s successful
system on a wholesale basis—the scale and logistics of Israeli and U.S.
_________________
5See Appendix C for further discussion of passenger profiling.
7
operations are vastly different—the concept is sound: focus security efforts on
those who arouse suspicion.
Conclusion
We propose an approach that is based on developing security solutions that
are appropriate to individual locations while ensuring that national interests
are safeguarded. Such a bottom-up approach not only empowers airports and
airlines by involving them in new partnerships with one another and with the
federal government, but also constitutes a systemwide effort that uses the best
analytic tools available to provide the most reliable forecasts and projections.
These are the keystones to an effective program. Baggage screening is an
important component of a full spectrum of security measures that can be
brought to bear immediately. The common goal should be a fully
functioning air transportation system that provides passengers with safe,
efficient, and convenient means of carrying out the nation’s business.
9
Appendix
A. Analysis of Deployment Requirements
for Baggage Scanning Equipment
Introduction
Even prior to September 11, 2001, the Federal Aviation Administration (FAA)
had a plan to field explosive detection system (EDS) equipment to every
commercial airport in the United States. At that time, primarily because of
funding limitations, deployment completion was scheduled for 2013. After
September 11, it was clear to the FAA that the deployment schedule had to be
shortened. Assuming adequate funding and trying to account for limitations on
manufacturing capacity for certified EDS machines, the FAA then tentatively set
2005 as a target date. However, Congress mandated an earlier date, December
31, 2002. RAND undertook a review of whether the FAA’s plan for the 2005
target date could be implemented within the new time frame.
This appendix takes the need for the highest level of security at U.S. airports as a
given. Since this need translates to having to scan all or most of the baggage that
passengers bring to the airport, we have taken two views in seeking an answer to
Admiral Busick’s query. One view, the one we stress in this paper, is that of the
organization that has to buy, deploy, and operate the equipment. The other is
that of the flying public. The buyer of the equipment is concerned with its total
scan rates, its size and cost, and the facility space needed for its deployment and
operation. The airline passenger is worried about security, but is also worried
about whether he and his baggage are going to get onto the aircraft before it
departs.1 If the lines in which he and his baggage must wait are too long, he may
judge that his concerns about the safety of flying and about the loss of time
associated with waiting at the airport outweigh the benefit of flying. He may
then opt to use other forms of transportation or not to travel at all.
The choices of the potential passenger will directly affect the economic vitality of
the airlines and will indirectly affect the growth of the U.S. economy. His
________________
1For convenience, we have labeled all passengers as men. Of course, approximately half are
actually women.
10
viewpoint is thus important. However, it is also the more challenging of the two
viewpoints and the one given less attention is in this paper.
Our work originally focused on EDS machines, as called for in the FAA plan and
implied in the congressional bill. However, subsequent interest in the use of
explosive trace detection (ETD) machines, which can detect very small amounts
of explosive material by chemical means, led us to include their performance as
well.
Assessing Plans for Explosive Detection System
Machines
We were asked to independently assess how many EDS machines would have to
be deployed to achieve two objectives: 100 percent scanning of all checked
baggage, and performance of that scanning within timelines that would not bring
airport throughput to a halt. An EDS machine uses the same magnetic resonance
imagery (MRI) technology commonly found in the medical profession. It can
rapidly image a bag and its contents, measure selected properties of those
contents (e.g., density and shape), and advise a human operator as to whether
the contents are consistent with a bomb. EDS machines are in operation at many
of the large airports around the world; approximately 150 were in operation at
U.S. airports at the end of 2001. The FAA estimated that approximately 2,000
EDS machines would be required to meet the objectives just stated. Our primary
research objective was to test whether this estimate was reasonable for planning
purposes.2
After review of the FAA’s estimates, we identified four factors that could be
cause for concern:
•
The estimated buy size of 2,000 EDS machines was derived using current
traffic levels. Based on our understanding of the need at many airports for
additional space for baggage scanning operations, and considering the likely
time that will be needed for new airport construction, we believe that it is
sensible to plan against a future demand. Our assessment considers two
demand levels, one consistent with recent levels of airport operations and
one consistent with traffic levels forecast for calendar year 2010. 3
________________
2During our first conversation with Admiral Busick, he provided us with these estimates and
helped arrange for us to visit the contractor that had generated them.
3The combined impact of the September 11 attacks and the economic slowdown of 2001 has
altered the FAA forecast of future passenger demand. This paper’s analyses use the old date. New
FAA forecasts predict that the 2010 demand we used will not occur until 2012.
11
•
The criterion used in the FAA sizing of the EDS buy was “no delay greater
than one hour” over the course of the day. There is consensus among airport
managers that 1 hour is too long. Delays of this magnitude, if they occur
over extended portions of the day or apply to a substantial fraction of the
bags, will seriously impede passenger and baggage throughput at the airport
and will eventually reduce passenger willingness to fly.
•
The FAA’s criterion offered no margin for dealing with the numerous
uncertainties associated with achievable scan rates, false positive detections,
variable passenger demand, equipment reliabilities, and so forth. These
uncertainties are likely to exacerbate delays, causing substantially higher
peak-hour delays as well as an overall rise in non-peak delays.
•
Airport facilities, airline incentives, and other factors limit an airport’s ability
to concentrate the machines for maximum utility.
Because of the project’s scope, we were only able to examine operations at a
single large hub airport, Dallas–Fort Worth (DFW). From that analysis we
reached general observations about what might happen at all large airports.
Early in the study we visited the contractor that had provided the FAA with
initial estimates of how many EDS machines would be needed at U.S. airports to
complete the FAA’s program. The contractor shared its methodology and
resulting calculations with us.4 The contractor also concurred with our view that
if additional time and resources became available, a stochastic simulation of
airport-passenger-bag management should be undertaken to properly size EDS
deployments at major U.S. commercial airports. However, neither the contractor
nor we have had the time or resources to undertake such a simulation. The
contractor’s calculations and the analysis presented in this appendix should be
viewed as a rough first approximation. We replicated the contractor’s single
point estimate and extended it by performing a series of sensitivity analyses
designed to test the robustness of our estimates. 5 We show outcomes as a
________________
4We believe that the estimates provided to the FAA were competently done. The contractor
deserves high marks for what it was able to accomplish in a relatively short time.
5The FAA’s estimates were based on the following:
(a) The demand for baggage scanning was derived from the timing of aircraft departures, as
taken from a recent Official Airline Guide (OAG). A common load factor was assumed
(70 percent), and the check-in baggage total for each aircraft was taken from estimates of
average check-in bags per passenger.
(b) Demand was derived for individual airlines unless an airline did not have enough
demand to warrant its own machine. It was judged reasonable for airlines with only a
few departure flights a day from a particular airport to “share” a machine with another
airline that also had a small number of flights from that airport.
(c) A concept of operations for the machines was assumed for each airport. For large
airports, a two-tiered approach was assumed: the first tier reached judgments about
whether the bag might have something dangerous within it, and the second tier looked
12
function of equipment reliability, passenger arrival characteristics (what we call
the “spread”), false positive detection rates, and aircraft load factors. We show
how increasing the number of deployed machines can compensate for the
negative effects of these factors.
Note, however, that our sensitivity analysis cannot replace a realistic simulation
of total airport throughput. The design of airport security needs to ensure that
passengers and their bags reach their aircraft in a manner that is not so
inconvenient that the flying public chooses to forgo flying. The Congress passed
the Aviation and Transportation Security Act partly in response to its fear that
the flying public would lose confidence in the safety of flight and thus cease
flying. The cure for that fear should not unintentionally yield the same result. If
buying more machines is the way to provide efficient airport flow, then more
machines should be bought.6
Before turning to the results of our analysis, it is also worth noting that most
busy airports have multiple queues, some that can reinforce and extend the
delays associated with getting bags scanned, and others that lessen the perceived
impact of scanning delays.7 The lines at check-in counters and at the screening
stations for passenger carry-on baggage are often substantial, the latter more so
now, since September 11. The perceived requirement that passengers must
arrive at the airport at least 2 hours before departure time (common at some
airports at the time of this writing) is, we believe, a deterrent to travelers. If
baggage screening by itself produces a queue, it will be an entirely new queue.
How these queues interrelate is part of the overall problem, beyond our analysis,
_______________________________________________________________________
more closely to resolve whether the bag needed to be pulled and openly searched. We
used this concept as our base case.
(d) The EDS scanning rates used were based on real experience gained by fielded
equipment. We used similar rates.
(e) The prime criterion for sizing the total buy was keeping the longest delay experienced by
any airline under 1 hour.
Our primary concerns with the current estimate of 2,000 are (1) we think the 1-hour sizing
criterion is too long; (2) there was a general lack of sensitivity analysis related to equipment
malfunctioning, etc.; (3) the average load factors probably understate peak-hour traffic experience
and obviously underestimate conditions during high-volume traffic days (e.g., the day before
Thanksgiving); (4) no future traffic levels were used.
6Other analysis suggests that limiting the growth of aviation would cost the nation tens of
billions of dollars annually in the near term, and hundreds of billions in subsequent decades. If
buying more EDS machines can remove hindrances to flying, the cost would be more than
compensated by the return to the nation’s economy.
7One of the pivotal reasons for using a realistic large-scale simulation of airport throughout is
the need to better understand how these various queues interact. For example, a long queue at the
baggage check-in counter may be caused by having only enough clerks to accept bags at a rate
consistent with the scanning capabilities of the machines. Thus, queues that would normally be
attributed to the limitation of EDS machines might be attributed to there being too few clerks
handling the check-in process. Obviously, these check-in queues would add directly to queues
associated with passenger screening, while bag scanning delays would not.
13
and another reason for a stochastic simulation of an airport-passenger-bag
management system.
Assumptions and Assessment Process
Generalizations drawn from an analysis of the equipment needs of a single large
airport may not be appropriate for all commercial airports. Small airports differ
from large airports in terms of requirements and geometries. At many small
airports, one EDS machine may suffice. However, to provide a backup capability
to handle machine outages, a small airport might need two machines rather than
one, or access to personnel that can handle the job by hand-searching the bags.
However the capability is supplied, these airports accommodate only a small
fraction of the total passengers, lessening their impact on total equipment needs
for all airports.8 Overall traffic behavior and baggage demand levels suitably
scaled do not differ much among large airports having substantial hubbing
activity, with the exception of origin-destination (O/D) traffic levels. The
fraction of O/D passengers, compared to the fraction of connecting passengers,
varies at the major hubs, being low at some—Atlanta Hartsfield International—
and higher at others—Chicago O’Hare International. Over the course of the day,
Dallas–Fort Worth (DFW), our selected hub airport, is somewhat lower (~36
percent O/D traffic, according to 1996 data). However, O/D traffic is generally
higher during peak traffic hours, when business travelers tend to dominate the
manifest. We assumed an O/D traffic level of 50 percent as a norm in our
calculations, approximately matching the industry average at large hubs. This
norm was also used in the FAA estimates.
We also assumed that only check-in baggage from O/D passengers is scanned.
This implies that bags being transferred from one aircraft to another have already
been scanned at the airport where the baggage was first loaded onto an aircraft.
As it is uncertain when small feeder airports will get their share of the EDS
machines, this assumption is optimistic for hub operations.9
We used a computerized 1998 OAG schedule for flight departures at DFW to
estimate the number of bags that will need to be scanned in the “near term.” This
computerized OAG also provided a schedule for aircraft traffic projected at DFW
________________
8Small airports need baggage scanning equipment regardless of their size. The number of
machines needed will be disproportionately large because of the sensitivity of overall performance to
uncertainties. However, the total number will still be small compared with the buy requirements at
the 30 largest U.S. airports.
9Having to scan all transiting bags would be a disaster for airport hubbing operations. It would
force significantly longer ground stops, lowering airline fleet utilization and aircraft bank integrity at
the affected airports.
14
in the year 2010, which we used for our longer-term assessment.10 Whether this
growth is realistic after the September 11 events is obviously subject to question.
Nevertheless, we believe it is prudent to assume that, security issues
notwithstanding, the U.S. economy will resume a reasonable growth rate. Given
that the transportation sector of the economy is judged to be a strong factor in
that growth, we believe the government should recognize that designing the
baggage scanning system so as not to impede that growth is an important public
policy goal.11
Another reason for using the futuristic airline demand is the long lead time for
airport facility planning and construction. Limited airport space is a critical
issue. Most large airports will require either new facility construction or the
invention of innovative ways to dramatically modify the space that already
exists. The identification of what needs to be done at each airport must consider
demand growth well beyond the date of initial machine deployments if the risk
of being short of space immediately thereafter is to be avoided. By focusing on
2010, we can estimate the amount of equipment space that will have to be
provided at the large hub airports within the current decade.12
Following the approach used in the FAA’s analysis, we assumed a three-step
detection process: a high- and a low-speed EDS as the first two steps and, when
indicated, a manual open-bag inspection as the third step.13 These three steps
proceed as follows:
•
Initial inspection by high-speed EDS machines. These machines rapidly scan the
bag, searching for objects whose density matches that of a known explosive
material. If no such objects are found, the bag is judged “safe.” Baggage
scan rates of 350 bags per hour (or approximately one bag every 10 seconds)
have been achieved in these conditions.14 Test data suggest that
approximately 25 percent of the bags will have objects (not necessarily
bombs) that match the sought density and thus cannot be judged “safe”
________________
10This computerized OAG was constructed a few years ago to generate nationwide traffic
profiles that could be used in large-scale simulations of the national airspace system (NAS). The 2010
traffic schedule is based on a computer program that looks at demand at all U.S. airports and
apportions flights on a demographic basis to airport-pairs. In general, the computer seeks “open”
slots for the scheduling of arrival and departure times, avoiding congestion as best it can. Whether
this is a sensible assumption for future airline schedules is a topic for a later day.
11Comments by Secretary Mineta on setting a goal of no security-related delays greater than 10
minutes clearly relate to this point.
12It is worth noting that any major new construction at an airport will take nearly a decade to
complete if current planning and construction timelines are assumed. If this is the case, planning
needs to begin today if a 2010 date for full operation is to be met.
13No analysis on delays has been provided for the manual inspection part of this process.
14We have been told that this is consistent with the operational performance of the CTX9000,
one of the EDS machine types currently in production.
15
without a more in-depth examination of their contents.15 This is the
phenomenon usually referred to as a false positive.16 Any “unresolved” bag is
passed on to step two; all other bags go directly to the airline’s baggage
assembly area.17 We use the term tier 1 as a descriptor of this step.
•
Follow-up inspection of unresolved bags by slower-speed EDS machines. To further
identify an object having the threatening density, another EDS machine
provides a set of images to the machine operator who can manipulate the
images and get a feeling for the object’s size, shape, and density.18 Based on
this more thorough inspection of the bag, the operator reaches a judgment
about whether the bag has a bomb in it. The time required for this inspection
is dictated by operator skill and experience. Based on field data, we have set
this rate at one bag per minute. 19 There is, naturally, a wide dispersion
about this number. We have assumed that all but a small fraction of the bags
(less than 1 percent) are resolved as safe and sent to the airline’s baggage
assembly area. Those still not resolved must be hand searched to determine
whether they are “safe.”20 We use the term tier 2 as a descriptor of this step.
Manual open-bag inspection. This third step does not impact our analysis. No
time was added to the delays to represent it. 21 So the assumption of a less
than 1 percent rejection factor in tier 2 has no bearing on our results.
________________
15This assumption also matches current operating experience. There is a close coupling
between the machine’s average scan time and its ability to discern whether a bag is “safe.” For this
analysis, we made no attempt to “optimize” this parameter and lack the data to do so.
16We do not know the actual probability of failing to detect a real bomb. But recognizing that
the attempt to put a bomb onto a U.S. aircraft is a very rare event, a false positive detection rate of 25
8
percent will generate at least 10 more false detection than real detections.
17If the baggage scanning function is done in a centralized manner, with multiple airlines
involved, then an additional process is required to sort the baggage. The airlines are understandably
very reluctant to entrust the airport or the government with the responsibility of ensuring that bags
are suitably sorted. Missing bags will be blamed on the airlines, and it is the airlines that will be
damaged by lost passenger loyalty. Mainly for this reason, the major airlines remain fairly adamant
that they keep control of check-in luggage, even if it means they need to have EDS equipment
dedicated to their needs.
18One EDS machine manufacturer offers a capability in which images obtained while the bag is
being scanned in tier 1 are stored for later replay if the tier 1 scan does not judge the bag safe and
send it to tier 2. In this case, tier 2 becomes a set of workstations (rather than EDS machines) where a
human operator inspects the stored images in a manner similar to the tier 2 operation we assumed.
This approach probably requires fewer EDS machines, although total throughput may not be
significantly improved.
19We have been told that this is also consistent with operational experience, assuming a wellqualified operator.
20This percentage is at best a rough estimate. It is sensitive to the competence of the tier 2
inspector, the decision tools provided for explosive detection, and the amount of time used in the
inspection. There is also controversy as to whether the bag’s owner needs to be present when the bag
is hand inspected. If the owner’s presence is required, the inconvenience to the impacted travelers
might be significant. Recently (2/1/02) the TSA announced during a teleconference with many large
airports that it intends to do away with this requirement.
21Hand searching of suspicious bags can take up to 5 minutes and perhaps even more per bag.
16
Figure A.1 depicts the assumed operational concept of how passengers and
check-in baggage flow through the airport. Tier 1 and 2 are shaded.22
The OAG provides scheduled arrival and departure times for individual aircraft.
Each aircraft is designated by type (e.g., Boeing 737). Knowing the approximate
number of seats per aircraft type, we derived an estimate of the number of
available seats over any time increment.23 Assuming a load factor of 70 percent,
RAND WP131/1-A.1
Airport Entrance
Ticketing
Passenger Check-in
Baggage Check-in
Passenger Screening
Carry-on Screening
Tier 1: Initial
Baggage Screening
Tier 2: In-depth
Luggage Screening
Baggage Assembly
Airport secure area
Ticketed passengers only
Passenger without
baggage, with ticket
Passenger with baggage,
or without ticket
Check-in baggage
Gate
Passenger Check-in
Aircraft
Figure A.1—Concept of Operations Used for Baggage Scanning
________________
22Although two-tier scanning concepts have some operational advantages, other concepts—e.g.,
a single stage in which the scanning machine performs both functions—can provide comparable
capabilities. Of the two-stage and the single-stage concepts, the latter are generally more flexible in
handling unexpected performance variations.
23By 2010, many of the aircraft found in the 1998 OAG will have been upgraded or replaced by
newer models. As such, it is expected that the number of seats per aircraft will grow over time. No
consideration of this growth is included here. Such seat growth has been predicted in the recent past,
but the real-world experience—exacerbated since September 11—has been exactly the opposite. The
average number of seats per departure has declined, in fact, in recent years. Thus we have taken a
neutral stance.
17
we translated the OAG into the number of passenger enplanements. The number
of originating passengers was estimated by multiplying the number of
enplanement passengers by the assumed O/D fraction of 50 percent. This
assumes that originating passengers were only half of the total enplanements, the
other 50 percent being transfers from arriving aircraft.
The number of passengers—passenger demand—using the flight departure time
from the OAG was collected into 3-minute segments. We translated this into the
number of bags that must be scanned before being placed in the aircraft. Based
on discussions with a major airline, we assumed that
•
30 percent of all originating passengers have no check-in bags and go straight
to the gate for check in.
•
Of the remaining 70 percent, each passenger averages 1.67 check-in bags.
The overall average we used was 1.17 bags per originating passenger. 24
Figures A.2 and A.3 show the number of baggage check-ins on a typical day
using the 1998 and 2010 demand, respectively. The time shown is based on the
aircraft’s scheduled departure time, not passenger’s arrival time at the airport.
The peaks in baggage demand over the day stress the system and dictate how
many EDS machines need to be bought. The actual time stamp is not important
to our analysis. All originating passengers and their bags are included in these
data. We show a similar demand curve for a single airline (American Airlines) at
DFW later, when we evaluate decentralized baggage scanning operations.
Passengers do not arrive at airports at a precisely known time before departure.
We would be overstating the load on the bag scanning system if we modeled
their arrival as if they did. Actual arrival times vary, based on passenger risk
aversion and anticipated airport congestion.25 To reflect this we spread their
arrival times uniformly over a 30-minute increment.26 Figure A.4 shows how
________________
24This number is similar to that used in the FAA’s estimate of machine buy-size requirements.
25Most experienced travelers, having bags to check in, know to arrive 45–75 minutes prior to
aircraft departure time, assuming no airport congestion for bag or personnel screening (where
curbside check-in is available, times may be somewhat shorter). A risk-averse person might come
even earlier, while some passengers choose to arrive later. Currently, experienced travelers add as
much as 1 additional hour to account for delays associated with more-extensive security screening.
26The “spread” can be distributed across the specified time in a variety of ways. For simplicity
(and because it favors baggage handling), we chose to use a flat distribution. Thus, for a 30-minute
spread, 1/30th of the passengers would arrive during each minute of the time span. We examined
alternative distributions (e.g., triangular, binomial) and different lengths of time (from 15 to 60
minutes). The differences in outcomes were small except at the largest deployment sizes, where the
maximum delays were calculated to be very small.
18
Check-in baggage demand (3 min increments)
RAND WP131/1-A.2
1200
1000
800
600
400
200
0
6
9
12
Time of day
15
18
Figure A.2—Check-In Baggage Demand at DFW, 1998
Check-in baggage demand (3 min increments)
RAND WP131/1-A.3
1400
1200
1000
800
600
400
200
0
6
9
12
Time of day
15
Figure A.3—Check-In Baggage Demand at DFW, 2010
18
19
Check-in baggage demand (3 min increments)
RAND WP131/1-A.4
Scheduled demand; no spread
Arrival demand; spread = 30 min
1200
1000
800
600
400
200
0
6
9
12
15
18
Time of day
Figure A.4—Actual and Spread Demand for 1998 DFW Case
spreading the passengers and thus their baggage alters the baggage check-in
demand arrival times for the case of all flights at DFW, 1998. Spreading the
demand over time lowers the relative peaks, producing more-uniform baggage
flows into the airport; it also makes the analysis more tractable. Accordingly,
•
We calculated the baggage demand, based on input assumptions regarding
the number of flights (all, or limited to a single airline), using standard input
assumptions on O/D percentage of total passengers and the average number
of check-in bags they bring. Demand was broken into 3-minute increments,
a total of 240 over the entire day.
•
Using this demand, we determined the length of baggage scanning queues at
each time increment (in both tier 1 and tier 2) for the specified EDS
deployment in tiers 1 and 2. We calculated the longest time required for
scanning a bag at each increment and noted the maximum length calculated
across all time steps.27
________________
27The maximum and minimum delays are calculated for bags entering a specific 3-minute
window. Those arriving first in the window experience the minimum delay; those arriving last, the
maximum. Using a strict first-in, first-out queuing strategy for each tier, it can be shown that the
largest delay that any bag experiences is the larger of the implied delay in tier 2 (assuming that 25
percent of all bags were directly sent to tier 2 and added to the existing queue in that tier) or the delay
associated with simply passing through tier 1.
20
•
Next, we varied the number of machines deployed in the tier 1 and tier 2,
recording the maximum delay time for each deployment combination.
Figure A.5 is a portion of a larger matrix that contains the full results from
this calculation. The columns are for tier 2 EDS machine deployments, and
the rows are for tier 1 EDS machine deployments.28
•
We then selected a standard against which to judge the results. For example,
using the data in Figure A.5, we could specify that the maximum delay can
be no more than 10 minutes. This standard is met for a number of machine
buys, but the one requiring the fewest machines involves 20 tier 1 machines
and 29 tier 2 machines, for a total buy of 49.
•
We also noted that machine reliability can differ between tiers 1 and 2,
reflecting the likelihood that different machines will be used in the different
tiers. We calculated the probabilities that an individual pair of tier 1 and tier
2 machines will be operating on a given day. Using these probabilities, we
calculated the expected value outcomes and their distributions for delays
averaged over a large number of days.
•
We performed a sensitivity analyses on selected variables, and compare
results.
RANDWP131-A.5
Number
of Tier 1
machines
Number of Tier 2 machines
20
21
22
23
24
25
26
27
28
29
30
31
14
93.51 82.06 82.06 82.06 82.06 82.06 82.06 82.06 82.06 82.06 82.06 82.06
15
93.51 66.87 49.58 49.58 49.58 49.58 49.58 49.58 49.58 49.58 49.58 49.58
16
93.51 66.87 47.29 33.63 31.86 31.86 31.86 31.86 31.86 31.86 31.86 31.86
17
93.51 66.87 47.29 33.63 28.55 24.86 24.86 24.86 24.86 24.86 24.86 24.86
18
93.51 66.87 47.29 33.63 28.55 23.93 19.72 18.76 18.76 18.76 18.76 18.76
19
93.51 66.87 47.29 33.63 28.55 23.93 19.72 15.99 13.51 13.51 13.51 13.51
20
93.51 66.87 47.29 33.63 28.55 23.93 19.72 15.99 12.53
9.41
9.23
9.23
21
93.51 66.87 47.29 33.63 28.55 23.93 19.72 15.99 12.53
9.41
8.39
7.79
22
93.51 66.87 47.29 33.63 28.55 23.93 19.72 15.99 12.53
9.41
8.39
7.46
Shaded rectangles house “balanced deployment” options.
Figure A.5—Typical Maximum Delay Matrix
________________
28The lightly shaded entries in the matrix reflect where tier 1 and tier 2 buy sizes are “balanced.”
In all other parts of the matrix, either tier 1 or tier 2 is the bottleneck (those above the lightly shaded
entries are limited by tier 1 capabilities; those below, by tier 2.
21
To understand how our centralized scanning assumption affected our
conclusion, we examined how a single airline might operate if scanning were
done at a number of independent locations. We generated a demand profile for
each of these locations by dividing the OAG schedule uniformly into mini-OAG
schedules (the results of this decentralized scanning example are shown later in
this appendix). Centralization requires fewer machines for a given capability,
and it is part of our base case assumptions. However, the more realistic
assumption is decentralization. Current airline incentives and practices all point
toward it.
General Results for EDS Machines
We examined three cases:
•
Case 1: All flights at DFW, circa 1998 (based on an “old” OAG schedule).
•
Case 2: All flights at DFW, circa 2010 (based on a “futuristic” OAG schedule).
•
Case 3: All American Airlines (both American and Eagle—AAL/EGL) flights
at DFW (based on the “old” OAG schedule). This case includes the
decentralized scanning example.
The first case is the most comparable to the analysis done for the FAA by its
contractor, as we discussed earlier. The second case extends the analysis to 2010.
The third case explores the issue of centralization, giving us an assessment of the
inefficiency associated with fractionating the baggage scanning equipment
among individual airlines and the number of additional machines that would be
required.
General Comments on Criteria for Sizing the Buy
We include in this appendix several “measures of merit” to help determine the
performance of different cases and to understand the value of increasing or
decreasing the number of machines installed. No single measure of merit
captures all the different interests of the various stakeholders. One set of
stakeholders, the airlines, is interested in keeping its passengers happy, which
translates into an incentive to get all checked-in bags onto the proper aircraft
before departure time. Security is also important for the airlines, as is meeting
the demands of their passengers for convenient, on-time performance. If these
objectives come into conflict, the airlines face a dilemma. Other things being
equal, the airlines will want the government to install sufficient equipment to
22
assure that, under almost all circumstances, baggage scanning will not prevent
on-time departures.
Another stakeholder, the new Transportation Security Administration, was
created to make certain that bombs do not get onto aircraft. If this objective
forces passengers to come to the airport earlier so that they and their baggage get
onto the aircraft in time, that is the “price” of security. Rapid deployment of
scanning machines and 100 percent baggage scanning appear to be the
government’s primary concern.
The flying public is obviously interested in flight safety. For business travelers,
however, timeliness and convenience are also important. They want rapid
transit through the airport so as not to waste more time than necessary boarding
the aircraft. Time is money to them. They may well seek alternative forms of
travel if the time spent traveling by commercial airline gets to be too long or too
inconvenient.29
Business travelers often choose not to check their bags, motivated mainly by the
inconvenience and delays associated with baggage check-in and retrieval. Those
that do check bags want a high probability that those bags will be waiting for
them at their destination. Because they value their time, they are interested in
the risk associated with arriving at the airport too late for their bags to get onto
the aircraft. This suggests a measure of merit that looks at the fraction of bags
that make it onto the aircraft as a function of passenger arrival time at the airport.
The results shown below attempt to capture these various viewpoints. The
principal measures of merit examined are as follows:
•
The maximum check-in bag delay encountered over the entire day, assuming
all deployed machines are operating, for each EDS deployment. We call
these point delays.
•
The total number of EDS machines deployed (the sum of the machines in
tiers 1 and 2).
•
The number of bags suffering delay as a function of the magnitude of that
delay, assuming all deployed machines are operating, for each EDS
deployment.
________________
29The well-documented increase in the use of private jets (owned or leased by businesses)
subsequent to September 11 may be a reflection of this. While some of the increase may be safety
related, the data suggest that a major portion of it is convenience related.
23
•
The expected value of the maximum bag delay, where the number of
operating EDS machines varies randomly because of reliability losses, for
each EDS deployment.
•
The expected number of bags experiencing delays as a function of the time
between bag arrival at the queue and flight departure. The number of
operating EDS machines is a random variable, based on reliability losses, for
each EDS deployment.
In addition, we show a number of results not related to these measures of merit
that provide additional insight into what various sizes of machine deployments
mean to the airlines and the flying public.
Case 1: All Flights at DFW, Circa 1998
Focusing on the size selection problem from the perspective of the airlines
(maximum queues short enough that all bags can get on the aircraft at any time
of the day), we constructed a list of machine deployment options using the
maximum delay measures of merit. Figure A.6 shows a set of plausible EDS
machine deployments for demand generated by the 1998 OAG. The short
descriptive phrase associated with each entry relates to the measures of merit
used to derive the deployment. The tier 1 and 2 numbers relate to the equipment
deployed and not to the actual number of machines in operation at any specific
time. We also assumed that machines cannot be reallocated from one tier to the
other.30
Option 1 entails installing a sufficient number of machines to scan all the bags
within a 16-hour workday at the airport. If demand were “flat” over time, this
deployment would be sufficient to handle the load without delays. However,
demand is not flat, so substantial queues occur. As the results show, the
maximum delay would be over 5 hours. This is obviously not a good option.
Option 2 shows the “not in excess of one hour” standard for maximum delay
used by the FAA’s contractor. A total of 37 machines satisfies this standard. The
difference between our number and the FAA’s estimate of 30 machines is most
likely due to differences in our input demand numbers.
The resulting maximum delay in option 2 was just under 50 minutes. We
suggested earlier that maximum delays of this magnitude are unacceptably large.
________________
30Our primary rationale for this assumption is the size of the machines, the virtual impossibility
of moving them from one location to another on short notice, and the inherent fixed structure of the
conveyor belts that deny flexible routing options between tiers.
24
RANDWP131-A.6
All flights at DFW
(CY1998 traffic)
Number of
machines
Tier Tier
Total
1
2
Results
Max delay (min) Criteria (% less than)
Point
Exp
value
10
min
30
min
60
min
Days over criteria
10
min
30
min
60
min
1 Estimate based on
daily averages
10
15
25
303.2
2 No queue greater
than 60 min
15
22
37
49.58
131.1
0.0
0.0
2.0
365.0 365.0 357.6
3 No queue greater
than 30 min
17
24
41
28.55
73.61
0.0
1.3
43.0
365.0 360.1 208.1
4 No queue greater
than 15 min
19
28
47
13.51
31.82
0.0
60.5
94.8
365.0 144.1
5 No queue greater
than 10 min
20
29
49
9.41
24.87
0.6
81.2
98.3
362.9
68.7
6.4
6 Reliability hedge
22
32
54
6.66
14.17
37.2
97.9
99.9
229.1
7.8
0.4
7 Exp value less
than 10 min
24
34
58
5.21
9.39
80.6
99.8 100.0
70.6
0.7
0.0
8 Impact of
variations in input
Reliability = 85%
22
32
54
6.66
21.64
9.2
86.3
98.4
331.4
50.0
6.0
Reliability = 80%
22
32
54
6.66
33.03
1.4
60.7
90.5
359.7 143.3
34.6
a Hedge vs.
80% reliability
25
37
62
3.97
14.07
49.9
95.0
99.4
182.9
Total bags = 55,452
18.3
19.2
2.2
Figure A.6—EDS Deployment Options Considered, All Flights at DFW, 1998
Less than perfect reliability is one reason. For the purposes of this analysis, we
assumed the operating reliability of the EDS machines was 90 percent.31 Given
this reliability, the expected value of the maximum delay exceeds 2 hours. A
similar measure of merit—the median delay, where 50 percent of passengers
would experience greater delays and 50 percent would experience lesser delays
(not shown in the figure)—also results in delays slightly over 2 hours. The farright column shows that the 60-minute measure of merit would only be satisfied
on fewer than 8 days of the year. The calculation of number of days on which the
standard is not satisfied is, we believe, a useful measure of the deployment’s
adequacy.
Before proceeding, more must be said about how we selected our “preferred”
options. Maximum delays obviously decrease as we add machines to tiers 1
and 2. If properly balanced between the two tiers, the added machines are
________________
31For analytic simplicity, we assumed that if a machine were out of service, it remained so for
the entire day. In the real world, equipment would be maintained constantly, with outages for a
particular machine probably lasting a few hours at most. Having machines going off line and then
coming back at various times of the day, with average availability being 90 percent, would increase
the “noise” of the outcomes, reducing performance somewhat.
25
always going to yield better results. But is it worth the added cost? Not having
an economic measure of merit related to delay reduction, nor knowing the cost of
additional deployments, we cannot conclude that adding more machines is the
cost-effective answer. What we can note is the degree to which the flying public
is inconvenienced. Below we show curves depicting the probability of bags not
reaching the aircraft as a function of number of machines deployed and
passenger arrival time at the airport. These curves are strongly correlated with
maximum delay estimates, showing decreasing benefit as the number of
machines grows. Thus, our judgment about “preferred” outcomes has led us to
select buy sizes consistent with point delays somewhat below the 10-minute
mark. Selecting a buy option with higher point delays would likely result in
many bags missing their intended aircraft even if passengers arrived 90 or 120
minutes early.
Returning to Figure A.6, it can be seen that option 3 reduces the maximum delay
standard to 30 minutes. Options 4 and 5 further reduce it to 15 and 10 minutes,
respectively. The associated delay numbers are reduced substantially. The
median delay for the deployment of 20 machines in tier 1 and 29 in tier 2 (the
case of maximum delay less than 10 minutes) is about 25 minutes, matching the
expected delay. The expected and median delays of 25 minutes and higher
suggest that the average traveler will be forced to arrive at the airport
significantly before the plane is scheduled to leave if he expects his bag to reach
the aircraft in time for departure (peak hours only).
Figure A.7 is a plot of the calculated delays over the day. Two deployment
options are shown—option 2 (the FAA-equivalent case) and option 5 (no queue
greater than 10 minutes). Option 2 yields relatively small delays until midafternoon, when delays rise to their peak of just under 50 minutes before falling
back to zero. A person who flies frequently would quickly learn that afternoons
are a bad time in terms of check-in baggage delays and would adjust his airport
arrival timing (or check-in baggage propensity) accordingly. Nevertheless,
arrival times of approximately 90 minutes before flight would be needed, adding
up to 45 minutes to what would be normal earlier in the day. Option 5 shows
almost no delays across the entire day. This option would not affect passenger
behavior, assuming that the screening equipment is in full operation.
We selected option 6 (a deployment of 22 machines in tier 1 and 32 in tier 2) as
our “preferred” option. It yields a maximum point delay of 6.66 minutes, and an
expected and median delay of around 14 minutes. Maximum delays of 60
minutes or greater would only occur one day every two years, a totally
acceptable outcome. However, option 6 is still sensitive to the assumed
reliability of the equipment, as shown toward the bottom of Figure A.6.
Length of queue for baggage screening (minutes)
26
RAND WP131/1-A.7
60
EDS deployed = [20:29]
50
EDS deployed = [15:22]
40
30
20
10
0
6
7
8
9
10
11
12
13
Time of day
14
15
16
17
18
Figure A.7—Bag Screening Delays as a Function of Time of Day, All Flights
at DFW, 1998
Maximum delays increase sharply when the reliability degrades to 0.85 or
(worse) 0.80.
Our preferred option requires the FAA to install at DFW approximately 80
percent more EDS machines than originally planned. Our estimate for
equipment requirements; even without considering future demand needs, is thus
almost double that of the FAA.
Figure A.8 shows estimates of point and expected-value maximum delays as a
function of total number of deployed EDS machines. The deployments between
tiers 1 and 2 were “balanced” to yield the lowest maximum delays for the
number of machines plotted on the ordinate.
Figure A.9 extends these results. It assumes that an airline will alert passengers
to the need to come to the airport earlier. The calculation starts with the
assumption that the airline would want a window of at least 35 minutes before
flight departure to assure that the passenger and his bag can make it onto the
27
RAND WP131/1-A.8
70
Point estimate, rel = 100%
Expected value, rel = 90%
Maximum baggage delays (minutes)
60
50
40
30
20
10
0
40
45
50
55
60
65
70
Number of operating machines
75
80
Figure A.8—Maximum Delays per Number of EDS Machines in Operation, All Flights
at DFW, 1998
aircraft.32 Anticipated bag queuing delays would simply add to this window.
The figure plots the probability that all bags get onto the aircraft as a function of
the “recommended” early arrival time of the passenger. The preferred buy of 22
tier 1 and 32 tier 2 EDS machines depicted as [22:32] allows the airlines to be
confident that they can handle virtually all check-in bags, assuming passengers
arrive 1 hour before departure. Only under extraordinary circumstances—say,
when a large number of the EDS machines are out of operation simultaneously—
would the airlines have a problem. The nonpreferred deployment curves in the
figure tell a different story, requiring substantially earlier arrival times for highconfidence baggage handling.
Passengers also want their bags to be carried on their aircraft. When planning
their trip to the airport, they instinctively balance their desire to be certain that
________________
32Airlines are probably capable of processing and loading onto the aircraft most check-in
luggage within a window of 15–20 minutes. However, airlines will want to allow additional
passenger time for check-in, and check-in queues often reach 15 minutes or more. Thirty-five
minutes seems to be a reasonable time to handle both of these considerations.
28
RAND WP131/1-A.9
Probability that all bags get onto aircraft (%)
100
90
80
Machines deployed = [24:34]
Machines deployed = [22:32]
Machines deployed = [20:29]
Machines deployed = [19:28]
Machines deployed = [17:24]
70
60
50
40
30
20
10
0
30
35
50
40
45
55
60
65
70
75
80
Arrival time prior to aircraft departure (minutes)
85
90
Figure A.9—Passenger Arrival Time for All Bags to Get Onto Aircraft, All Flights
at DFW, 1998
their baggage will be successfully placed on the aircraft against the
inconvenience of having to spend a lot of time at the airport. Figure A.10
attempts to capture passenger concerns by plotting the planned arrival time of a
passenger against his estimate of risk in not having the bag loaded on the aircraft.
The risk is a function of the amount of screening equipment deployed at the
airport. Of course, in reality, the passenger is likely to discover the risk only
through experience.
The results presented above are based on the passenger demand profile
calculated from the 1998 OAG. Predicting the magnitude of future demand is a
risky business, especially after September 11. It is, however, reasonable to
predict that traffic demand will grow over time, and that with that growth will
come the need for larger deployments of bag scanning equipment. The forecasts
have never been wrong in predicting the magnitude of future growth—only in
predicting the exact year in which it would occur.
29
RAND WP131/1-A.10
Planned time of arrival before scheduled departure,
including a 15-minute unplanned delay before baggage
check-in (minutes)
120
105
90
75
60
45
30
15
0
100.0
Machines deployed = [15:22]
Machines deployed = [17:24]
Machines deployed = [19:28]
Machines deployed = [20:29]
Machines deployed = [22:32]
Machines deployed = [24:34]
10.0
1.0
Probability that bags do not get on aircraft (%)
0.1
Figure A.10—Planned Time of Arrival According to Passenger Propensity to Accept
Risk, All Flights at DFW, 1998
Case 2: All Flights at DFW, Circa 201033
Except for the magnitude of the demand, the 1998 and 2010 cases are quite
similar, so we do not go into comparable detail here. As explained earlier, we
believe selection of an appropriate EDS buy should be based on future demand.34
We chose the 2010 demand as our preferred base case and used it to perform
most of our sensitivity analyses.
Figure A.11, the options chart for this case, closely parallels Figure A.6.
Following the same logic as before suggests an equipment deployment of 31 tier
1 machines and 44 tier 2 machines [31:44] as a preferred option. This deployment
________________
33As noted earlier, the FAA has altered its forecast of future passenger demand, now
predicting that the 2010 demand we used in our analyses will not occur till 2012.
34This hypothetical OAG was constructed at a time when passenger air travel was expected to
grow at approximately 3.5 percent per year. The recent downturn in the nation’s economy raises
valid concerns about whether the traffic levels shown in the case are realistic for 2010. The results
given here scale more-or-less linearly with the gross level of demand, so this can be suitably scaled
when better demand data are available.
30
RANDWP131-A.11
All flights at DFW
(CY2010 traffic)
Number of
machines
Results
Max delay (min) Criteria (% less than)
Tier Tier
Total
1
2
Point
Exp
value
10
min
30
min
60
min
Days over criteria
10
min
30
min
60
min
1 Estimate based on
daily averages
16
24
40
260.3
2 No queue greater
than 60 min
23
34
57
54.29
126.5
0.0
0.0
0.2
365.0 365.0 364.1
3 No queue greater
than 30 min
25
36
61
22.08
87.68
0.0
0.2
15.5
365.0 364.4 308.6
4 No queue greater
than 15 min
26
38
64
13.73
63.73
0.0
6.4
49.7
365.0 341.7 183.8
5 No queue greater
than 10 min
28
40
68
9.6
36.47
0.1
43.7
85.1
364.7 205.5
54.4
6 Reliability hedge
31
44
75
6.62
13.68
34.1
94.2
99.4
240.4
21.1
2.2
7 Exp value less
than 10 min
32
47
79
5.22
9.73
71.5
98.6
99.9
103.9
4.9
0.3
8 Impact of
variations in input
Reliability = 85%
31
44
75
6.62
27.45
5.6
65.7
91.1
344.4 125.3
32.4
Reliability = 80%
31
44
75
6.62
52.98
0.5
26.9
63.4
363.3 267.0 133.7
a Hedge vs.
80% reliability
35
51
86
3.86
15.15
41.1
89.1
97.8
215.1
39.7
8.0
Load factor = 0.8
31
44
75
11.95
48.79
0.2
64.2
94.8
364.4 130.7
18.8
Hedge option 1
34
50
84
6.99
15.77
14.0
92.8
99.4
313.8
26.1
2.2
Hedge option 2
36
53
89
5.60
10.25
60.0
99.1
99.9
146.0
3.4
0.2
False positives =
30%
31
44
75
17.27
63.61
0.0
5.5
54.6
365.0 344.8 165.8
b Hedge option
32
56
88
5.22
9.77
70.7
98.7
99.9
106.8
4.6
0.3
Passenger
spread = 45 min
31
44
75
4.13
10.60
78.3
94.2
99.6
79.2
21.1
1.3
Passenger
spread = 60 min
31
44
75
3.68
9.07
85.4
98.0
99.6
53.2
7.2
1.3
Total bags = 89,946
Figure A.11—EDS Deployment Options Considered, All Flights at DFW, 2010
provides a reasonable hedge against uncertain machine reliability. At a total
deployment of 75 machines, this option calls for 45 more machines than the
original estimate of 30 that the contractor provided the FAA, and 22 more than
our own estimate of 54 in the 1998 case. Note that 75 divided by 30 equals 2.5,
the EDS machine buy growth estimate given in the main part of this paper.
Figure A.11 also shows the impact of degraded reliability on our preferred buy
and suggests a deployment of 86 machines if reliability turns out to be only 80
percent. The actual operational reliability is often not well known until the
31
machines are fully fielded, and the degree of hedging needed is not easily known
at this point.
Our prior calculations assumed a uniform load factor of 70 percent. Most
airports, in sizing their facility needs for the future, assume an 80 percent load
factor as their stressing case. We have followed their example, testing the impact
of an 80 percent load factor. Increasing the load factor for our nominal [31:44]
deployment caused delays to rise sharply. The desired machine buy increased
into the mid to high 80s.
Actual results are also sensitive to the false positive detection rate achieved in tier
1. Increasing the rate to 30 percent resulted in an increase in delays35 and
required the number of machines between the two tiers to be rebalanced, with a
higher fraction of machines going to tier 2. We say more about sensitivity to
these factors later.
To summarize this case, our analysis shows that
•
A buy size of 75 is a reasonable design point for deployments expected to
handle airline passenger growth up to 2010.36
•
A variety of uncertainties could push this buy size into the mid to high 80s
and, if multiple uncertainties exist, even into the 90s. In light of this, our
preferred buy size of 75 is at the low end of what we believe to be a
reasonable design point.
•
Given the various uncertainties that might determine actual baggage
scanning throughput, these numbers are approximate at best.
•
In light of this, it is essential that the airports and airlines collectively
examine the opportunities and constraints associated with equipment
deployment and baggage system operation.
It is important to note that most major airports (and certainly DFW) probably do
not have sufficient space for installing the equipment needed for the 2010
demand case. This simply underscores the need for serious airport planning
________________
35This is as it should be. Given the balancing of our machine deployments between tier 1 and 2,
where neither tier becomes the dominant bottleneck, any growth in traffic in either tier is going to
cause comparable increases in machine buy requirements. The percent increase in bags to be scanned
in tier 2 was about 14 percent for the load factor increase and 16 percent for the increase in the false
positive detection rate.
36An analysis of single-tier concepts of operations (not discussed here) led to total buy sizes that
were about 10 machines lower than those reported here. Single-tier operations (using higher-speed
scanning machines) yield total buy results that are approximately 15 percent lower than those shown
here. Operational considerations (e.g., starting and stopping of the high-speed machines for detailed
scanning purposes) would lessen this difference somewhat, but it appears that some reduction in
total buy size might be possible with a single-tier operation.
32
involving all parties—airlines, airports, and the government. In addition, it
reinforces the need to examine alternative ways to achieve the desired security
goals while simultaneously providing airline passengers and their baggage with
a relatively easy and swift transit through the airport.
Figures A.12 through A.15 are replications of Figures A.7 through A.10, modified
to reflect the forecast demand in 2010. Figure A.12 shows maximum baggage
screening delays for two EDS deployments as a function of time during the day.
In contrast to the result for the 1998 OAG data, the delays here for the 60-minute
maximum delay criterion rise steeply by mid-morning and persist and grow
throughout the remainder of the day. This suggests that by 2010 a larger fraction
of the flying public will have to face delays associated with baggage scanning,
further reinforcing the point that “getting it right” is important. The preferred
option shows delays that would not hinder passenger or baggage flow through
the airport.
Length of queues for baggage scanning (minutes)
RAND WP131/1-A.12
60
EDS deployed = [31:44]
50
EDS deployed = [23:34]
40
30
20
10
0
6
7
8
9
10
11
12
13
Time of day
14
15
16
17
Figure A.12—Bag Screening Delays as a Function of Time of Day, All Flights
at DFW, 2010
18
33
RAND WP131/1-A.13
Maximum baggage delays (minutes)
60
Point estimate, rel = 100%
Expected value, rel = 90%
50
40
30
20
10
0
50
60
70
80
Number of operating machines
90
100
Figure A.13—Maximum Delays per Number of EDS Machines in Operation, All Flights
at DFW, 2010
The implications of the results shown in Figures A.13, A.14, and A.15 are the
same as those for the 1998 demand case. The overall observations are essentially
unchanged, although the actual number of machines required obviously has
grown.
The next three figures contain the sensitivity analyses for EDS equipment
reliability, aircraft load factor during peak hours, and EDS false positive
detection rate. Figure A.16 shows the impact of reductions in EDS equipment
reliability on the probability of getting bags onto the aircraft. The performance of
our preferred buy size of [31:44] rapidly deteriorates as reliability falls toward
0.8, reinforcing the message that reliability is an important factor in overall EDS
screening performance. Care needs to be taken to ensure that the reliability of
the EDS machines remains high and is well bounded.
Figure A.17 shows how maximum point delays change as a function of the
maximum load factor in the afternoon. To compare results evenhandedly, we
held total demand constant, shifting some demand in the early hours to the
afternoon. The maximum load factor during the afternoon was varied to as high
as 0.875. The increase in maximum point delays leads to the observation that
34
RAND WP131/1-A.14
[31:44]
[29:42]
[28:40]
[26:38]
[25:36]
Probability that all bags get onto aircraft (%)
100
90
80
70
60
50
40
30
20
10
0
30
35
50
40
45
55
60
65
70
75
80
Arrival time prior to aircraft departure (minutes)
85
90
Figure A.14—Passenger Arrival Time for All Bags to Get Onto Aircraft, All Flights
at DFW, 2010
RAND WP131/1-A.15
Planned time of arrival before scheduled departure,
including a 15-minute unplanned delay before baggage
check-in (minutes)
120
105
90
75
60
45
30
15
0
100.0
Machines deployed = [25:36]
Machines deployed = [26:38]
Machines deployed = [28:40]
Machines deployed = [31:44]
10.0
1.0
Probability that bags do not get on aircraft (%)
0.1
Figure A.15—Planned Time of Arrival According to Passenger Propensity to Accept
Risk, All Flights at DFW, 2010
35
RAND WP131/1-A.16
Probability that all bags get onto aircraft (%)
100
90
80
70
60
50
40
30
[31:44], rel = 0.90
[31:44], rel = 0.85
[31:44], rel = 0.80
[35:51], rel = 0.80
20
10
0
30
35
50
40
45
55
60
65
70
75
80
Arrival time prior to aircraft departure (minutes)
85
90
Figure A.16—Sensitivity of Results to Reductions in Equipment Reliability, All Flights
at DFW, 2010
RAND WP131/1-A.17
Total demand held constant over full day;
Load factor prior to 2 PM decreased as needed
70
Maximum delay (minutes)
60
Initial load factor = 0.70
50
[23:34]
[25:36]
[26:38]
[28:40]
[31:44]
[32:47]
40
30
20
10
0
1.00
1.05
1.10
1.15
1.20
1.25
Load factor increases after 2 PM
Figure A.17—Sensitivity of Results to Increases in Load Factor During Peak Afternoon
Demand, All Flights at DFW, 2010
36
even modest increases in the load factor during peak afternoon hours would
substantially lower performance. This suggests that our assumption about
constant load factors over the day is likely to yield estimates that are lower than
what is really needed.
Figure A.18 varies the false positive detection rate from 0.2 to 0.33 (0.25 was the
nominal value in all our previous calculations). Reductions in this rate below
0.25 show no gain because the demand on tier 1 is unchanged and the tier 1
queue becomes dominant. The rapid rise in delays as the false positive detection
rate rises above 0.25 reflects the increased demand on tier 2, with tier 2’s queue
now dominant.
It is generally accepted that the real false positive detection rate for the deployed
EDS equipment will be well known only after the machines are installed and
some operational experience is gained. Thus, the upside sensitivity to this rate is
a matter of concern. Some of that upside can be mitigated if the two tiers are
rebalanced. Figure A.19 shows the benefits of rebalancing, assuming that it is
possible. There are obvious limitations (e.g., redoing the baggage flow conveyer
lines, hiring more trained operators for the more-challenging scanning job in
RAND WP131/1-A.18
180
Maximum point delays (minutes)
150
120
[23:34]
[25:36]
[26:38]
[28:40]
[31:44]
[32:47]
Design Point
90
60
30
0
0.23 0.24 0.25 0.26 0.27 0.28 0.29 0.30 0.31 0.32 0.33
False positive match / all bags
Figure A.18—Sensitivity of Results to Changes in the False Positive Detection Rate, All
Flights at DFW, 2010
37
RAND WP131/1-A.19
180
Maximum point delays (minutes)
[23:34]
Total = 57
150
[25:36]
Total = 61
120
[26:38]
90
Design Point
Total = 64
[28:40]
Total = 68
60
[31:44]
Total = 75
30
[32:47]
Total = 79
0
0.23 0.24 0.25 0.26 0.27 0.28 0.29 0.30 0.31 0.32 0.33
False positive match / all bags
Figure A.19—Benefits of Flexible Tier 1/2 Deployment for Countering Uncertainty in
False Positive Detection Rate, All Flights at DFW, 2010
tier 2), but additional flexibility in sizing the two tiers would be a substantial aid
to dealing with uncertainty.
Case 3: All American Airlines Flights at DFW
Our final case focuses on a single airline at DFW. The purpose of examining a
single-airline scenario is to permit us to calculate the inefficiency associated with
the baggage screening being performed by individual airlines. For our purposes,
looking at a single airline is sufficient to obtain an approximate answer to
whether this inefficiency is large or small.
Figure A.20, which is similar to Figures A.2 and A.3, is the demand profile for
American Airlines at DFW. The total number of bags that get checked at the
counter is 36,857, or about 41 percent of the total for all DFW in the 2010 case. 37
The demand has more individual spikes than are seen for the other cases,
________________
37The single-airline case is the American Airlines flight schedule in 1998. As such, it does not
reflect (nor is it expected to reflect) American’s traffic levels in 2010. However, it is a useful device for
constructing a comprehensive schedule that would be typical of a single airline with 41 percent of the
traffic in 2010.
38
RAND WP131/1-A.20
Check-in baggage demand (3-min Increments)
1000
900
800
700
600
500
400
300
200
100
0
8
9
10
11
12
13
14
Time of day
15
16
17
18
Figure A.20—Check-In Baggage Demand for Single Airline at DFW (41 Percent of
Total Bags), 2010
suggesting that the total number of machines needed to keep delays within
reason will be greater in percentage terms than it will be for either of the other
cases.
Figure A.21 lists the EDS machine deployment options using the same selection
criteria used above. Stepping through the same logic used above, we concluded
that deployment option 6, involving a total machine buy of 37 machines, is our
preferred choice on the list. As before, changes in assumed reliability (shown),
load factor (not shown), and other factors would tend to increase this number to
44 and above.
Figure A.22 shows the sensitivity of point and expected delays to total size of the
EDS deployment. To achieve maximum-delay expected-value outcomes of
under 10 minutes, total buys must be in the 57–58 range.
Figure A.23 is similar to Figures A.9 and A.14. Our preferred [15:22] deployment
size performs relatively well, requiring little on the part of the airline in advising
39
RANDWP131-A.21
AAL/EGL flights
at DFW (1998)
Number of
machines
Tier Tier
Total
1
2
Results
Max delay (min) Criteria (% less than)
Point
Exp
value
10
min
30
min
60
min
Days over criteria
10
min
30
min
60
min
1 Estimate based on
daily averages
8
11
19
243.1
2 No queue greater
than 60 min
11
15
26
56.11
133.1
0.0
0.0
6.5
365.0 365.0 341.4
3 No queue greater
than 30 min
11
16
27
28.87
110.0
0.0
5.8
16.2
365.0 343.8 306.0
4 No queue greater
than 15 min
12
18
30
14.77
58.71
0.0
48.4
59.4
365.0 188.5 148.1
5 No queue greater
than 10 min
14
20
34
9.92
21.89
2.8
91.5
94.5
354.8
31.2
20.0
6 Reliability hedge
15
22
37
7.25
13.69
34.0
98.3
98.6
240.7
6.2
5.0
7 Exp value less
than 10 min
17
24
41
4.95
8.77
83.9
99.9
99.9
58.6
0.4
0.3
8 Impact of
variations in input
Reliability = 85%
15
22
37
7.25
13.69
34.0
98.3
98.6
240.7
6.2
5.0
Reliability = 80%
15
22
37
7.25
13.69
34.0
98.3
98.6
240.7
6.2
5.0
a Hedge vs.
80% reliability
18
26
44
3.65
12.99
53.5
97.6
98.1
169.6
8.8
6.8
Total bags = 36,857
Figure A.21—EDS Deployment Options Considered, Single Airline at DFW, 2010
RAND WP131/1-A.22
Maximum baggage delays (minutes)
60
Point estimate, rel = 100%
50
Expected value, rel = 90%
40
30
Balanced Tier 1 and 2 deployments
20
10
0
40
45
50
55
60
Number of operating machines
65
70
Figure A.22—Maximum Delays per Number of EDS Machines in Operation, Single
Airline at DFW, 2010
40
RAND WP131/1-A.23
Probability that all bags get onto aircraft (%)
100
90
80
[15:22]
[14:20]
[13:19]
[12:18]
[11:16]
[11:15]
70
60
50
40
30
20
10
0
30
45
60
75
90
105
Arrival time prior to aircraft departure (minutes)
120
Figure A.23—Passenger Arrival Time for All Bags to Get Onto Aircraft, Single
Airline at DFW, 2010
passengers to arrive early at the airport. Smaller deployments perform poorly,
pointing out the sensitive nature of these outcomes to buy size.
We compare the machine buy calculation for American Airlines at DFW against
the total DFW requirement after we look at a further partitioning of the demand
within the airline itself.
Case 3A: American Airlines Flights at DFW Partitioned
Into Six Individual Scanning Locations
As a variant of case 3, we examined how total EDS buy size would change if we
partitioned the baggage scanning operation into six equal but separate locations
corresponding to the terminal arrangement at DFW. American’s current
operations essentially fill two terminals. Assuming that DFW chooses not to
totally reconfigure the airport and to instead find a way to “stuff” the machines
into existing space, the result will probably be six individual scanning locations.
Figure A.24 shows the outcome of our analysis for one partition. Following the
logic above, we concluded that option 5 with eight machines was best.
41
RANDWP131-A.24
0.16 of AAL
flights at DFW
Results
Number of
machines
Max delay (min) Criteria (% less than)
Tier Tier
Total
1
2
Point
Exp
value
10
min
30
min
60
min
Days over criteria
10
min
30
min
60
min
1 Estimate based on
daily averages
2
2
4
165.1
2 No queue greater
than 60 min
2
3
5
17.13
136.2
0.0
59.0
59.0
365.0 149.5 149.5
3 No queue greater
than 30 min
3
3
6
15.1
82.59
0.0
70.9
70.9
365.0 106.4 106.4
4 No queue greater
than 15 min
3
4
7
5.84
31.53
47.8
92.1
92.1
190.4
28.8
28.8
5 No queue greater
than 10 min
3
5
8
4.29
20.71
67.0
96.4
96.4
120.6
13.3
13.3
6 Reliability hedge,
exp value < 10 min
4
5
9
2.99
8.73
87.1
98.8
98.8
47.3
4.5
4.5
7 Impact of
variations in input
Reliability = 85%
3
5
8
4.29
81.50
34.6
80.1
80.1
238.8
72.5
72.5
Reliability = 80%
3
5
8
4.29
124.74
25.1
69.0
69.0
273.4 113.1 113.1
a Hedge vs.
80% reliability
3
7
10
2.14
8.07
91.1
98.9
98.9
32.6
4.1
4.1
8 Reliability hedge,
all AAL flights
15
22
37
7.25
13.69
34.0
98.3
98.6
240.7
6.2
5.0
Scaled AAL
partitioned flights
18
30
48
4.29
81.50
34.6
80.1
80.1
238.8
72.5
72.5
Percent increase
Total bags = 36,857
29.7
Figure A.24—EDS Deployment Options Considered, Single Airline Partitioned Into Six
Separate Scanning Locations
Assuming the other five partitions are comparable, this suggests a total
deployment of 48 machines, compared to the nonpartitioned American Airlines
requirement for 37 machines. Thus, the partitioning leads to an increase of 11 in
total machines required for American, or an increase of about 30 percent.
Comparisons Across the Three Cases
What do these three cases mean in terms of anticipated EDS machine buy sizes?
The top part of Figure A.25 compares the 1998 and 2010 EDS buys given
comparable “maximum delay performance standards.” Given the scaling for
42
RANDWP131-A.25
Percent increase in total bags = 62.2%
Comparing
2010:1998
deployments
9 Reliability hedge
(1998)
Results
Number of
machines
Max delay (min) Criteria (% less than)
Tier Tier
Total
1
2
22
32
54
Scaled to full
baggage traffic
36
52
88
Reliability hedge
(2010)
31
44
75
Percent increase
Total bags = 55,452
Point
Exp
value
10
min
30
min
60
min
10
min
30
min
60
min
6.66
14.17
37.2
97.9
99.9
229.1
7.8
0.4
6.62
13.68
34.1
94.2
99.4
240.4 21.1
2.2
–14.8
Ratio of bags (AAL vs. all Flts) = 0.410
AAL flights only
(CY1998 traffic)
Total bags = 36,857
Results
Number of
machines
Max delay (min) Criteria (% less than)
Tier Tier
Total
1
2
10 Reliability hedge
(AAL only)
15
22
37
Scaled to full
demand
37
54
91
3
5
8
Scaled to full
demand
44
74
118
All flights
reliability hedge
31
44
75
Single partition
(< 10 min delay)
Days over criteria
Percent increase
(AAL, no part)
21.3
Percent increase
(AAL, 6 parts)
57.3
Days over criteria
Point
Exp
value
10
min
30
min
60
min
10
min
30
min
60
min
7.25
13.69
34.0
98.3
98.6
240.7
6.2
5.0
4.29
20.71
67.0
96.4
96.4
120.6 13.3
13.3
6.62
13.68
34.1
94.2
99.4
240.4 21.1
2.2
Figure A.25—Comparison of 2010, 1998, and Single-Airline Deployment Needs
additional traffic, the required growth in EDS deployments actually is slower
than the growth in baggage screening demand. The primary reason for this is the
overall reduction in the difference between the traffic peaks and valleys. The
new flights (those added to the 1998 OAG) are spaced between the existing
flights, filling the valleys without comparable increases in the peaks. This
leveling effect still leads to higher demand for EDS equipment, but at a rate
lower than that of the new traffic. Of course, centralization is the principle
reason why such economies of scale were possible.
43
The lower part of Figure A.25 compares American Airlines flights in 2010 with all
flights in 2010, allowing us to judge the inefficiency of having individual airlines
assume the responsibility for scanning baggage that is to be placed on their
aircraft. Assuming that American can centralize its baggage scanning
equipment, a 21 percent increase is implied, which suggests that a reasonable
requirement for equipment deployments at DFW in 2010 might be 88 machines,
assuming that airlines continue the current practice of assuming responsibility
for scanning their own baggage. If American cannot centralize its equipment and
has to partition it into six separate scanning areas, the total climbs to 118
machines, for a total increase in buy size of over 50 percent. The ratios of these
two buy sizes over the original FAA estimates are 3.03 and 3.93, respectively.
Estimating a Total Buy Size for All Airports
To scale our findings to total EDS machine buys for all airports, we took the
straightforward approach of simply multiplying the planned total buy of 2,051
EDS machines by 2.5, the scaled increase derived for DFW. The result is 5,077.
We believe this number to be conservative. As our analysis demonstrated, there
is an economy of scaling when traffic grows at an airport. Because most airports
have fewer scheduled operations than DFW does, we would expect their scaled
requirements to be somewhat larger.
Our estimate is also conservative because it assumes centralized baggage
scanning, which is currently inconsistent with the practices of U.S. domestic
airlines. Many foreign airports practice (at least to some degree) centralized
baggage handling, enabling them to have a central facility for EDS scanning. U.S.
airlines and airports could follow suit, albeit only with major modifications to the
airports. If centralization does not occur, a buy size closer to 6,000 may be
required. Scaling the two machine buy estimates for the American Airlines cases,
we get total buy sizes of 6,221 and 8,067 respectively. Adding in other
uncertainties (e.g., reliability) to the nonpartitioned American case, the real
number could easily grow to 7,000.
Finally, to repeat what we said earlier, the real EDS equipment buy size needs to
be determined with calculation tools far more sophisticated than the simple
spreadsheets used here. We are confident that our work does not overstate the
need. How much the need is understated is uncertain.
44
Would the Results Have Been Worse If the Input
Variables Were Treated as Stochastic?
Our calculations assumed that the stated values of all the variables were fixed for
a particular run. Clearly, this is not the case. For example, the likelihood of
detecting an object in a bag that has the right density to be potentially
threatening is a random event. Our calculations assumed that every fourth bag
in the sequence would have an object with suitable density to send the bag to tier
2. In the real world, sometimes none of the four bags would be sent to tier 2, and
sometimes all four bags would be sent. Our question here is whether random
outcomes of this kind would alter the conclusions we reached.
Using the all flights at DFW, 2010, case, we selected the balanced tier 1 and 2
deployments shown in Figure A.11 for our calculations. For each deployment we
ran 100 trials. Two separate cases were run for each trial—one in which we
randomized only the arrival of bags with suitably dense objects for detection and
assignment to tier 2, and one in which we also randomized the arrival of the bags
themselves. In both cases we balanced the outcomes to ensure that the overall
statistics were identical to the nonrandom results (we did not want to skew the
results by altering the overall demand statistics). The results were averaged over
the 100 trials, and each outcome was compared against the outcome for the case
in which no randomness was permitted.
Figure A.26 shows the results. Overall, the maximum delays increased by a few
percent. 38 Based on this result, we conclude that stochastic outcomes are going
to yield results somewhat higher than those shown in our outcome charts, but by
an amount that will raise the total buy requirements by only a small amount.
While this factor should not be ignored (and would obviously be a prominent
output in any large-scale simulation of airport throughput), it also should not be
cause for serious concern.
The Potential Role of Passenger Profiling in Reducing
the Buy
Passenger profiling has the potential both to increase in the near term the
likelihood that those bags most likely to pose a threat get scanned and to lower in
the long term the total buy size. The goal of 100 percent scanning of all baggage
________________
38The negative change in maximum delays shown for the smaller deployment sizes is not an
outlier. It persists even when the number of trials is greatly increased. The overall delays increase
slightly, but the peaks drop.
45
RAND WP131/1-A.26
7
6
Percent change in max delay
5
Bag scanning demand
stochastic in tiers 1 and 2
Bag scanning demand
stochastic in tier 2 only
4
3
2
1
0
Average change over all trials:
Tier 1 & 2 : 3.28%
Tier 2 only : 3.20%
Adjusted for constant demand
-1
-2
-3
[23:34]
[25:36]
[26:38]
[28:40]
[31:44]
[32:48]
Number of EDS machines deployed [tier 1 : tier 2]
Figure A.26—Stochastic Results Versus Expected Value Results, All Flights
at DFW, 2010
cannot be met for several years if airport throughput is to remain at a reasonable
rate. Thus we are left with the question of how many bags must be removed (i.e.,
judged not to be a potential threat) from the baggage scanning queue so that all
remaining bags can be scanned within a reasonable time.39
Figure A.27 plots the maximum delay times associated with total machine buy
size, assuming that all equipment is fully operational and being used. The
baggage demand at DFW was reduced by excluding from the baggage queue a
fraction of all bags arriving at the check-in counter. The chart shows reductions
of up to 50 percent. Not surprisingly, the drop in demand is almost exactly
matched by a drop in the number of machines needed to achieve a specific value
________________
39As noted in the main part of this paper, profiling can be used in two ways—to identify
potential threats and to identify nonthreats. The latter is probably the best approach to use if we are
talking about reductions of 50 percent in the total baggage flow. Having a list of those passengers
most likely to be terrorists would greatly reduce the number of bags to be scanned but would raise
some serious questions about (a) the list’s completeness and (b) unequal treatment of some people
who fall too close to the class of people deemed threatening. Both approaches can and should be
used simultaneously, but it is likely that something close to 50 percent of all fliers have histories that
would rule out terrorist behavior as even remotely likely.
46
RAND WP131/1-A.27
Passengers passing
profiling criteria
None
10 percent
20 percent
30 percent
40 percent
50 percent
Maximum delay over day (minutes)
100
10
1
0
10
20
30
40
50
60
70
Total machines in operation
80
90
100
Figure A.27—Effect of Reduced Number of Bags Needing to Be Scanned
on Maximum Delays
of the maximum delay point estimate. Thus, if passenger profiling can eliminate
50 percent of the bags as threats, the number of machines required to handle the
remaining bags is about half what we estimated above.
Obviously, the successful use of profiling can have a dramatic impact on how
many EDS machines need to be acquired and deployed in order to scan all bags
likely to be threatening. We do not know how many travelers might confidently
fall into the “safe” category; nor do we know what additional data would have to
be provided to the airlines/airports/FSA to achieve demand reductions that
would enable airports to avoid costly facility modification/replacement. On the
one hand, it seems logical to expect that people who fly frequently will
disproportionately satisfy the positive profiling criteria. And because these
people often fly at peak hours, we might expect profiling to have a greater impact
on reducing peak demand than is indicated here. On the other hand, those who
fly frequently also tend to carry their bags onto the aircraft, which means the
effect could be less than is indicated. Without data, it is impossible to know
which factor is the most important.
47
Figure A.28 parallels earlier figures, showing the performance results for various
equipment deployment options at the 50 percent demand reduction level.
The smaller deployment numbers make the outcomes more sensitive to
reliability, suggesting that reasonable buy sizes might be about 60 percent (rather
than 50 percent) of those recommended at the 100 percent scan level. Figure A.29
shows how the deployment options in Figure A.28 vary as demand is reduced.
Noteworthy is that the indicated buy sizes scale almost linearly with reductions
in demand. This gives us an easy way to assess the value of passenger profiling
in managing both baggage check-in delay and overall EDS buy requirements.
RANDWP131-A.28
All flights at DFW,
50% bag reduction
Number of
machines
Tier Tier
Total
1
2
Results
Max delay (min) Criteria (% less than)
Point
Exp
value
10
min
30
min
60
min
Days over criteria
10
min
30
min
60
min
1 Estimate based
on daily averages
9
12
21
2 No queue greater
than 60 min
12
17
29
47.90 125.00
0.0
0.0
4.7
365.0 365.0 347.8
3 No queue greater
than 30 min
13
18
31
22.08
89.39
0.0
3.8
28.0
365.0 351.1 262.9
4 No queue greater
than 15 min
13
19
32
13.73
73.28
0.0
10.7
43.8
365.0 326.0 205.0
5 No queue greater
than 10 min
14
20
34
9.6
45.93
2.8
39.6
73.0
354.8 220.6
98.6
6 Reliability hedge
15
22
37
6.76
22.30
34.0
76.5
92.7
240.7
85.7
26.6
7 Added reliability
hedge
16
23
39
5.55
13.92
63.7
91.1
97.7
132.4
32.7
8.3
8 Reliability hedge,
17
exp value < 10 min
24
41
4.59
9.64
83.9
97.1
99.4
58.6
10.7
2.3
9 Reliability hedge +,
18
exp value < 10 min
26
44
3.30
6.55
96.0
99.3
99.9
14.5
2.6
0.5
44.4
Total bags = 36,857
274.84
10 Impact of
variations in input
Reliability = 85%
16
23
39
5.55
27.75
30.3
69.6
87.8
254.4 111.0
Reliability = 80%
16
23
39
5.55
51.62
10.4
41.6
67.1
326.9 213.3 120.2
a Hedge vs.
80% reliability
20
29
49
2.81
8.99
86.8
96.1
98.8
48.1
14.2
4.4
11 Reliability hedge,
all bags
31
44
75
6.62
13.68
34.1
94.2
99.4
240.4
21.1
2.2
Reliability hedge,
50% of bags
15
22
37
5.55
13.92
63.7
91.1
97.7
132.4
32.7
8.3
Percent decrease
50.7
Figure A.28—EDS Deployment Options When Passenger Profiling Reduces Total Bags
in Queue, 2010
48
RANDWP131-A.29
Results comparing EDS buys and delays versus reduced demand
Exp value < 10 min
Reliability hedge
Criterion PV < 10 min
Number of
machines
Max delay (min)
Criteria (% less than)
Days over criteria
Tier
1
Tier
2
Total
Point
Exp
value
10
min
30
min
60
min
10
min
30
min
60
min
100%
28
40
68
9.60
36.47
0.1
43.7
85.1
364.7
205.5
54.4
90%
25
36
61
9.60
39.04
0.2
54.3
54.5
364.4
166.9
56.5
80%
22
32
54
9.60
42.04
0.3
37.2
75.0
363.8
229.1
91.3
70%
20
28
48
9.60
38.01
0.6
60.2
82.1
362.7
145.2
65.4
60%
17
24
41
9.60
40.90
1.3
43.0
83.9
360.1
208.1
58.6
50%
14
20
34
9.60
45.93
2.8
39.6
73.0
354.8
220.6
98.6
100%
31
44
75
6.62
13.68
34.1
94.2
99.4
240.4
21.1
2.2
90%
28
40
68
6.38
13.44
43.7
96.7
99.4
205.5
12.1
2.3
80%
25
36
61
6.08
13.06
54.3
94.4
98.8
166.9
20.5
4.3
70%
22
32
54
5.70
12.88
48.9
97.0
99.2
186.6
10.8
2.8
60%
19
28
47
5.46
12.38
60.5
94.8
99.0
144.1
19.2
3.6
50%
16
23
39
5.55
13.92
63.7
91.1
97.7
132.4
32.7
8.3
100%
32
47
79
5.22
9.73
71.5
98.6
99.9
103.9
4.9
0.3
90%
29
42
71
4.24
9.98
73.9
99.2
99.8
95.1
3.1
0.6
80%
26
38
64
4.86
9.55
81.7
98.5
99.7
66.7
5.6
1.2
70%
23
33
56
4.98
9.87
72.2
99.0
99.8
101.6
3.6
0.8
60%
20
29
49
4.46
9.20
81.2
98.3
99.7
68.7
6.4
1.0
50%
17
24
41
4.59
9.64
83.9
97.1
99.4
58.6
10.7
2.3
Figure A.29—Comparison of Profiling Results Across Criteria Categories
and Total Demand
Sizing the Buy
We started this appendix by assessing equipment buy sizes corresponding to the
FAA’s assumed maximum delay standard of no more than 60 minutes. The
results suggest that average delays could be far longer, especially if equipment
reliability is taken into account. In various examples we suggested alternative
standards, including maximum delays of 10 minutes or less.
The actual standard should be determined, however, based on an assessment of
the potential damage that excess delays might cause the air transport industry
and the nation’s economy. Understanding how passengers will react to growing
airport congestion and the increased ticket price associated with security
49
measures is central to such an assessment.40 Unfortunately, basic data on
passenger preferences are not robust and, in some cases, apparently do not exist.
Nevertheless, based on similar analyses, we believe it is almost certain that the
potential loss to the national economy would, as a minimum, be measured in
billions of dollars—and potentially in tens of billions of dollars—per year.
Judgments about the proper size of the buy should be weighed against these
numbers.
Some Additional Considerations
Our analysis was based on current EDS technology. The performance of any
feasible alternative detection technologies would surely alter our calculations.
The Aviation Security Technology Conference held in Atlantic City in November
2001 discussed a broad range of future options. This is not the proper place to
discuss these options and their potential. Nevertheless, it is possible that future
detection technology may alter our assessments. To assess this potential, we
scaled machine buy requirements against hypothetical scanning rates.
For simplicity we assumed a single-stage system. The operating assumption is
that the machine first scans the bag to see whether there is anything suspicious.
If it finds nothing, the bag is sent to the baggage assembly area; if it finds
something suspicious (perhaps via the machine’s automatic threat detection
algorithms), the bag is stopped, backed up, and exposed to a more detailed scan.
If that scan still finds something troubling, the bag is sent to an area where it is
opened and inspected by humans. We assume that the latter is a low probability
occurrence and does not materially alter the general throughput calculation
related to bags making it onto aircraft.
For calculation purposes, we used the demand profile taken from the 2010 OAG
for DFW (see Figure A.3). Figure A.30 shows the resulting maximum queuing
delays as a function of the operational scan rate. We included three passenger
spread assumptions (0, 30, and 60 minutes) to demonstrate the sensitivity to this
parameter. It is clear that total system scan rates above 150 bags per minute are
needed to keep the maximum bag delay below the 10-minute mark.
________________
40There is a growing movement in Congress and at the DOT to require that the flying public
bear most of the cost of making flying safe and secure. This movement has not as yet considered the
consequences of increasing ticket prices by an amount that could raise the overall price 25 percent or
more. Existing estimates of the elasticity of passenger willingness to fly as a function of ticket price
are between –0.7 and –1.0. If these elasticities are close to correct, the drop in passenger demand
would be substantial if the cost of security were fully added to ticket price.
50
RAND WP131/1-A.30
Maximum delays (minutes)
100
Arrival spread = 0 min
Arrival spread = 30 min
Arrival spread = 60 min
10
1
120
140
160
180
200
220
240
260
Overall scan rate (bags per minute)
280
300
Figure A.30—Maximum Queuing Delays Versus System Scan Rate
If we use the high-speed EDS machines in a single-stage mode and assume the
same scan rates (350 bags per hour in tier 1, and 60 bags per hour in tier 2) and
the same probability of false positives, we end up with a single-machine
combined rate of 142 bags per hour. Figure A.31 shows the single-stage point
and expected values resulting from this scan rate. Comparison of the singlestage and two-tier results shows essentially no difference for point delays and
slightly better performance for the single-stage operation. Two-tier performance
is worse because of possible asymmetric EDS outages (e.g., more tier 2 than tier 1
machines fail). Single-stage EDS outages reduce both rapid scan and detailed
scan capabilities evenly, as each machine is doing both.
Single-stage systems naturally adapt to uncertainties in the balancing between
rapid scan (tier 1 operations) and detailed scan (tier 2 operations). One example
of the importance of this was mentioned in the discussion of false positive
detection rate. Figure A.32 plots the maximum point delays as a function of
assumed operational false positive detection rate, selecting the all flights at DFW,
2010, “preferred” option of 31 machines in tier 1 and 44 in tier 2. Also shown in
the figure is a plot of the maximum point delays assuming a single-stage
deployment with the same number of machines (75). The delays match at the
two-tier design point of a false positive detection rate equal to 0.25. But they
51
RAND WP131/1-A.31
120
Maximum baggage delays (minutes)
110
PE, single tier, rel = 100%
100
EV, single tier, rel = 90%
90
80
70
60
50
40
30
20
10
0
45
50
55
60
65
Number of operating machines
70
75
Figure A.31—Point and Expected Values for Maximum Delays as a Function of Total
EDS Machine Buy Size
diverge substantially at off-design points. The same outcome would occur if the
EDS machines could be swapped between tiers without degradation in
performance. Whether this is practical remains to be shown.
The single-stage scan rate (bags per hour) per EDS machines at a false positive
detection rate of 0.25 is about 142 bags per hour. This scan rate could be
improved if we could lower the false positive detection rate or the time needed
for the detailed scan. A number of briefings at the Technology Security
Conference proposed approaches that would do both. Others claimed that their
technology would achieve rates well over 1,000 bags per hour. Details on
realistic detection performance and other factors were generally missing or
classified. But it appears that promising technologies are being developed and
should allow significant scanning capabilities at reduced deployment numbers.
Trace Scanning Options
One technology that is well developed is trace detection. Trace detection
machines “sniff” molecules that emanate from explosive materials. These
52
RAND WP131/1-A.32
60
Maximum point delays (minutes)
55
Fixed 2-tier deployment: [31:44]
Variable 2-tier deployment: 75 machines
Single-stage deployment: 75 machines
50
45
40
35
30
25
20
15
10
5
0
0.20 0.21 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29 0.30 0.31 0.32 0.33
False positive detection rate
Figure A.32—Sensitivity of Performance of Different Concepts of Operation to System
False Positive Detection Rate
molecules could come from the bomb itself or from the environment around the
bomb, assuming that a small quantity of the bomb material was inadvertently
spread onto them. Trace detection equipment is already deployed at many carryon baggage inspection sites, and elsewhere. No surprisingly, trace detection
equipment has been mentioned as a potential stopgap measure for bomb
detection in case adequate EDS equipment is not available by the end of this
year.
Trace detection equipment can replace or augment EDS equipment at any stage
in the detection process. We initially considered two ETD (electronic trace
detection) options:41
1.
A stand-alone single-stage ETD system.
2.
A combined multistage EDS/ETD system, where the EDS equipment is used
in the first tier and ETD equipment is used in the second.
________________
41There is a third option—following the trace detection scan with a second-stage EDS scan
whenever the trace scan yielded evidence of a potential bomb. We will address this option
parenthetically later in this section.
53
In both options, we initially assumed that adequate trace detection performance
would require a full scan of the bag and its contents, including the opening of
individual bags and a thorough sweep of their interior contents. Less intrusive
options—those involving opening the bag but not intrusively sweeping the bag’s
contents or (even less intrusive) not opening the bag but only sweeping its
exterior—were not considered in our initial look because of their inherently
lower bomb detection probability. 42 In the first option we assumed that the
baggage scan would be undirected, i.e., there would be no a priori information
regarding which part of the bag has a suspicious object. In the second option, we
assumed that the EDS machine provides an approximate location in the bag
needing trace scanning, allowing a directed search.
Option 1: Single-Stage ETD Scanning Options
We will address the ETD single-stage standalone option first, assuming no a
priori knowledge about potential threats within the individual bags.
Fortunately, we have data supporting trace search options involving open-bag
scanning.
The FAA run operational evaluation tests on stand-alone ETD baggage scanning
at Eppley Airfield (OMA) in Omaha, Nebraska, and at Stewart Airport (SWF) in
upper New York.43 That data include two distinct cases:
•
An undirected search (labeled NDT) of the contents inside the bag, where no
prior x-ray existed, or
•
A prior x-ray scan that identified objects in the bag that needed to be
scanned, leading to a directed search (labeled MDT) of the contents inside the
bag.
Figure A.33 shows the overall results of these tests. The mean scanning time
results from the tests showed relatively small differences between the two search
strategies (directed versus nondirected) at the two airports, but a fairly large
difference between airports (just under 3 minutes per bag at Omaha, compared to
4 minutes per bag at Stewart). The distributions in scan time were broad in all
cases, suggesting a lot of variability from one bag to the next. This variability
________________
42A full scan would include both an external scan of the bag and a scan of the exterior of the
interior of the bag prior to intrusively sweeping the bag’s interior contents. We subsequently looked
at ETD options that did not require opening the bag (these options are covered later in this section).
43See Roy Mason, Directed Trace and Non-Directed Trace Efficiency Study, Proceeding of the Third
International Aviation Security Technology Symposium, November 2001
54
could easily lengthen queue times, extending them significantly over a mean
average, but we did not attempt to include this factor in our calculations.
Figure A.34 comments on these results and shows the number of single-stage
ETD machines (under the NDT column) that would be required at DFW if the
2010 baggage demand were assumed.44 These numbers are large and would
stress the airport’s floor space capacity even though the trace detector’s footprint
is relatively small. Furthermore, the likelihood of detecting the presence of the
bomb was only 85 percent in the tests, an outcome not as close to 100% as almost
everyone would like.
Single-stage trace detection systems prompt a question that needs addressing,
i.e., how to respond when the ETD machine reports a detection. Current
procedures at airports call for evacuation of the terminals when a serious bomb
threat exists. Such procedures would be unacceptable under the circumstance
where alarms were common and even frequent during the working day, as
would be likely even assuming the very low false alarm rates currently attributed
to ETD machines (less than 1.0 percent). An additional stage is needed. For
purposes of this analysis, we assumed that it would be a hand search, where the
full contents of the alerted bag would be “dumped” onto a table and each object
inspected.45 The above analysis added no time for this search, recognizing that it
would involve only a small number of bags.
Recent TSA work with the airports suggests a trace detection approach different
from what we assessed above. Rather than a full bag scanning procedure, TSA is
looking at a baggage scanning strategy that included only partial scans. It is our
understanding that current studies associated with ETD buy requirements are
assuming that only 20 percent of the bags would receive a full scan, i.e., the
nondirected open-bag intrusive scan that we assumed in our option 1. An
additional 40 percent of the bags would receive only an exterior scan, and the
remaining 40 percent would receive only a nonintrusive interior scan. 46 It is also
our understanding that the following scanning times have been specified for
assessment purposes:
•
ETD scan of the exterior of bag only: 60 seconds.
________________
44The MDT results require some sort of scan in a prior stage.
45For safety reasons, we also assume that this stage would be performed in a location well
removed from the passenger-heavy part of the airport, thus adding substantial delay to the search of
that bag.
46These fractions are obviously tentative and likely to change prior to full deployment.
However, random selection will be used to determine which bags are given which degree of search.
55
RAND WP131/1-A.33
• Performed by Hughes Technical Center at two
airports (OMA, SWF)
• Two operational concepts tested
– Directed search, using x-ray machine input
(MDT)
– Nondirected search (NDT)
• Results (time per bag)
Elapsed Time (seconds)
min
max
mean
Omaha (OMA)
MDT bag processing time
32
602
156.0
NDT bag processing time
12
761
176.7
MDT bag processing time
32
1131
239.1
NDT bag processing time
18
1015
243.1
Stewart (SWF)
• Percentage of bombs detected = 85.7%
Figure A.33—Experimental Experience on Trace Detection Performance in the Field
RAND WP131/1-A.34
• Trace detection equipment has small footprint, relatively easy to install
• But
– Detection probability substantially less than 100% (~85% in ops tests)
– Time for bag search is substantial
• Using 10-minute max delay scanning rate requirements, the number of
trace detection stations would be as follows:
Number of ETD stations required (DFW, 2010)
Bags/min=173.2
MDT
NDT
Omaha (OMA) results
451
511
Stewart (SWF) results
691
702
Figure A.34—Maximum Delays Versus Minimum Required Number of ETD Machines
in Tier 2
•
ETD scan of the exterior of an opened bag (the scan does not penetrate into
the bag’s interior contents): 105 seconds.
•
ETD scan of the contents of an opened bag (the equivalent of the above NDT
search): 165 seconds.
56
In all cases, a false positive detection probability for any scanned bag was
specified to be 0.6 percent. 47 The authors know of no comparable detection
probabilities for the partial scans.48
We are not aware of the data that support the above timelines. The previously
discussed FAA tests included neither an external-to-the-bag trace detection scan
only option nor one that involved only an exterior scan of the bag’s interior
contents. However, they are not unreasonable, albeit perhaps slightly optimistic
compared to the FAA’s test data.49
Using the above assumptions we find that the number of trace detection
machines needed at DFW is substantially smaller than calculated using the
FAA’s test results. Figure A.35 shows the maximum delay encountered as a
RAND WP131/1-A.35
120
First-stage trace detection only
8 2nd-stage stations
9 2nd-stage stations
10 2nd-stage stations
12 2nd-stage stations
20 2nd-stage stations
Maximum single-stage delays (mins)
105
90
75
60
2010 Demand
30 Minute spread
100% Reliability
FAR at 1st stage = 0.6%
45
30
15
0
200
225
250
275
300
325
350
375
400
Number of first-stage trace stations
Figure A.35—Required Second-Stage ETD Stations Versus Maximum Delays
________________
47This assumption is obviously subject to question. Without real test data, we had no reason to
select a different false alarm rate.
48Detection probabilities and false alarm rates are coupled; higher detection rates usually imply
higher false alarm rates, assuming a common sensor and a fixed target characteristics.
49The only comparable datum point is the time needed for an intrusive, non-directed interior
scan. The TSA-specified time of 165 seconds is somewhat shorter than the OMA test result of 176
seconds, and considerably shorter than the Stewart result of 243 seconds.
57
function of the number of trace stations in operation. If the ETD machines were
90 percent reliable, the required number of ETD stations to keep average delays
below 10 minutes would be approximately 300 machines.
The aforementioned studies also recognized the need for a second stage to
handle the infrequently false positive trace detections. They posited an
additional 10 minutes for the “dump the bag” hand search. We have used the 10
minute number to show in Figure A.35 how many stage-two stations would be
needed.
Option 2: A Combined EDS/ETD Scanning System
Trace detection systems can be used in conjunction with other machines. This
option assumes that the first stage uses EDS machines in their rapid scan mode
and the trace detection machines in the second tier to resolve any alarms that
may have resulted. Assuming the more favorable full baggage scan rates (under
the MDT label) recorded at Eppley Airfield in Omaha, we balanced trace
detection total scan rates to match the tier 1 rates where EDS machines are used.
The results are shown in Figure A.36. Not surprisingly, the total number of trace
RAND WP131/1-A.36
Maximum bag scanning delays (minutes)
60
23
50
40
24
Number of tier 1 EDS machines
30
25
20
26
27
10
0
80
85
90
95
28 29
30
31
32
33 34
100 105 110 115 120 125
Number of ETD machines in tier 2
130
35
135 140
Figure A.36—Maximum Delays Versus Minimum Required Number of Trace Detection
Machines in Tier 2
58
detection systems is still very large, more than 110 for maximum bag queuing
delays of less than 10 minutes. If we use instead the recent data from the studies
mentioned above, that number would fall to approximately 70.
The above simply confirms that trace detection equipment can substitute for EDS
machines, but the number of such machines will have to be large to get the
scanning done. Lacking sufficient EDS machines, trace detection equipment can
offer a degree of additional bomb detection capability that will increase overall
security. But, as was true for the EDS equipment, care must be taken not to force
all bags to be subjected to in-depth trace detection when the resultant delays
would grow beyond reasonable. And, although the footprint of trace detection
equipment is considerably smaller than for the large EDS machines, it is still
unlikely that the full amount of equipment and personnel needed for 100 percent
scanning can be installed and operated within the current configurations of the
large metropolitan airports. As noted in the main paper, facility space
considerations are likely to remain the long pole in the deployment tent.
Summary
Our overall results are as follows:
•
Using a 1998 OAG for Dallas–Fort Worth Airport (DFW), 37 in-operation
EDS machines were required to achieve a maximum delay of less than 1
hour. This is somewhat larger than the FAA’s assessment of only 30
machines. But the closeness of the two estimates gives us confidence that our
analysis is consistent with what the FAA did earlier.
•
Assuming an equipment reliability factor of 90 percent, and demanding that
only a small percentage of all outcomes produce delays larger than 60
minutes, at least 49 EDS machines need to be deployed in the 1998 OAG case.
•
Holding scanning performance constant, other factors could raise this
deployment requirement by 50 percent or more. Among those factors
considered were:
— A 10 percent increase for peak-hour load factors of 80 percent.
— A 15 percent increase for operational reliabilities of 80 percent.
— A 20–30 percent increase for lack of consolidation of all scanning
equipment at the airport.
59
•
Using a postulated 2010 OAG for DFW, similar assumptions lead to
deployments in the 75–85 range. In percentage terms, this growth is
substantially less than the forecast increase in passenger growth at DFW. 50
However, given the need for significant airport redesign and expansion to
house this number of machines, as well as the time required to achieve that
redesign/expansion, building the capability to handle this many machines
would appear to be wise. The lower of these two numbers is the factor we
used in estimating a 2.5 increase (or 5,000 total machines) needed before
2010.
•
Partitioning the baggage scanning process among individual airlines would
increase these numbers by up to 30 percent. Thus, total EDS requirements at
DFW could exceed 100 before everything is done.
•
Reducing the baggage scanning requirements through the use of passenger
profiling would lower the total buy size requirements commensurately. The
scaling is nearly one-to-one. It suggests a short-term approach to meeting
the spirit of the Aviation and Transportation Security Act’s “scanning 100%
of all checked-in bags (that have any likelihood of containing a bomb)” when
not enough scanning equipment is available to do the entire job. It also
suggests a longer-term approach that integrates profiling into the baggage
scanning system, thereby lowering the total costs of providing security while
meeting the highest standards of performance.
No matter how we slice the results, it is impossible to conclude that the FAA’s
estimates for EDS deployment requirements at DFW are going to produce
satisfactory results. We believe that, overall, a planned deployment of at least
5,000–7,000 machines is in the ballpark of what is truly needed.
________________
50The postulated OAG adds new flights in the “holes” in the existing aircraft arrival and
departure schedule. These additions tend to smooth out the peak demand, essentially adding new
baggage demand at times when there is a relative lull. As a result, the equipment is able to
accommodate the new demand without a concomitant increase in the number of machines needed.
60
B. Simulation Modeling of Airport
Security Operations1
Simulation modeling provides airports, airlines, and government agencies with a
tool for evaluating the performance of new security procedures and equipment
prior to implementation. The benefit of a simulation approach is that one can
evaluate not only the security process, but also all the up-line and down-line
effects that process has on the entire airport. Therefore, simulation modeling is
invaluable in developing procedures and equipment plans that achieve the
highest levels of security and, to the extent possible, preserve passenger
convenience.
Modeling Approach
Computer simulation modeling uses mathematical models to represent the
design and operations of airport terminals. Although flow models and other
simulation methods exist, the most robust approach is to use stochastic, discreteevent simulation modeling. With this approach, individual entities—such as
passengers and baggage—travel through the airport processes all dependencies
and interactions between them and those processes being captured as they go.
The simulation logic models both the physical layout of and the rules that govern
facility operations. For example, the model includes the travel distance between
the ticket counter and the security checkpoint, as well as ticket counter and
security checkpoint staffing by time of day, processing rate distribution, and
work method. The simulation model is thus a very realistic representation of an
operating airport terminal.
Because the objective of simulation modeling is to evaluate system performance,
it is critical that accurate passenger and baggage demand be used. A 24-hour,
design-day schedule best represents passenger and baggage demand. As the
simulation model is executed, the passenger and baggage demand from the
design-day schedule flows through the airport system, and passenger terminal
and baggage handling system statistics are collected. The model collects
________________
1This discussion of simulation modeling of airport security operations was provided to RAND
for inclusion in this paper by TransSolutions, an independent consulting group specializing in the
analysis of passenger and baggage flow at airports. TransSolutions has worked at more than 80
percent of the world’s largest airports, including 24 of the top 25 airports in the United States.
Currently, TransSolutions is helping several airport clients improve the efficiency of security
checkpoint areas and planning for 100 percent EDS inspection of all checked baggage.
61
performance statistics for all areas of interest within the airport process (e.g.,
passenger queue lengths and baggage inspection delay times before each EDS
machine). Finally, the appropriate number of replications of the design-day
schedule are made in order to reach statistically valid conclusions.
Benefits
Three important factors make analyses based on stochastic, discrete-event
simulation the best approach for evaluating prospective security systems for
airports.
1.
Variability of human behavior. In the airport environment, passengers travel in
different-size groups, move at various speeds, and are served in different
ways depending on their origination or destination point, citizenship, travel
class, etc. Other analytic methods, such as simple queuing theory or flow
modeling, are unable to capture these details and must make unrealistic,
simplifying assumptions. Stochastic, discrete-event simulation analysis
captures the effects of variability in human behavior, thus providing the
most accurate estimate of system performance.
2.
Interdependence of airport processes. A passenger’s path through an airport is a
sequence of processes (e.g., ticketing, security inspection) and decision
points. The discrete-event simulation model moves individual passengers
and baggage through this complex sequence. Each process within the airport
affects both up-line and down-line areas of terminal processing. These
rippling effects are impossible to capture using other analytic methods. A
discrete-event simulation model can capture all the cascading effects
resulting from a single change to the airport system.
3.
Many diverse solutions require what-if analyses. U.S. airports are all unique.
They were constructed at different times, to serve the needs of local markets
and air carriers. As a result, the passenger and baggage processing areas of
airports differ in size and layout, producing unique passenger and baggage
processing sequences. Moreover, many different technologies are used to
accomplish passenger and baggage security inspection. As a result, the
security solution for each airport will be unique. A computer simulation
model provides a tool that can be used to quickly and economically evaluate
numerous schemes. The objective performance measures produced by the
simulation can be used to identify the scheme that achieves the mosteffective screening for the least cost and least passenger delay.
62
Simulation Output
Discrete-event simulation modeling can provide performance data for every
passenger and every bag at each step in the journey through the airport.
Performance data can include queue lengths, wait times, and area occupancies
for all airport processes (e.g., ticket counters, primary security inspections).
Performance figures of merit include not only averages, but also wait time
histograms, occupancy by time-of-day graphs, and maximum or percentile
values. For example, simulation output for a security checkpoint may show an
average wait of 2.5 minutes, a maximum wait of 24 minutes, and a statement that
95 percent of passengers wait no more than 12 minutes. Histograms and time-ofday graphs, shown in Figure B.1, provide a clear picture of system performance.
Computer simulation can also provide an animation of passenger and baggage
movement through the airport. This animation is invaluable in illustrating the
benefits of the preferred system and how it will work prior to installation. Figure
B.2 shows examples of computer simulation animations.
RAND WP131/1-B.1
Number of Passengers by Time of Day
100
1200
80
900
60
600
40
300
20
0
0
[0–3) [3–6) [6–9) [9–12) [12–15)[15–18)[18–21)[21–24) [24–>)
Time in minutes
300
250
Number of people
1500
Cumulative percentage
Number of passengers
Passenger Wait Time Histogram
200
150
100
50
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Time of day
Figure B.1—Sample Output from Discrete-Event Simulation
RAND WP131/1-B.2
Security checkpoint
EDS screening in a baggage system
Figure B.2—Samples of Computer Simulation Animations
63
C. Trusted Traveler Program
Passenger profiling is widely used by civil aviation authorities throughout the
world as a way to focus security protocols. Some countries use techniques that
could not be used in the United States, such as singling out passengers because of
their national origin or religious preference. In America, profiling relies
extensively on behavioral characteristics that have been exhibited by people
involved in terrorist attacks. For instance, people who buy one-way tickets at the
last moment, pay cash for them, and have no checked baggage are likely to get
more attention from security personnel. In the past, civil aviation personnel
lacked access to information from law enforcement and intelligence
organizations about people who were on watch lists, who had overstayed their
visas, or had for other reasons drawn suspicion on themselves.
Establishing a “trusted traveler” program would turn the current approach on its
head. Rather than trying to identify suspicious people, the objective would be to
identify people who do not need to be subjected to intensive scrutiny except on a
random basis. For example, such a program might begin by issuing
biometrically verifiable identification cards to military personnel, government
employees, law enforcement personnel, and private sector employees who hold
security clearances or who have had recent, thorough background checks.
Encouraging people to volunteer and pay for an appropriate background check
could expand the number of people having trusted traveler cards. Anyone who
volunteers would pay an amount sufficient to cover the cost of the investigation
whether or not a card was issued.
To avoid penetration of the program by terrorists, security of the trusted traveler
database and of the nature of the adjudication process would have to remain
paramount. A voluntary program does not raise issues of civil liberties or
privacy so long as the database remains secure. Enrollment in a trusted traveler
program would not be a right. Government employment requires a background
check; many private companies scrutinize employment history and financial
status and require drug tests before offering jobs to people. People who do not
want trusted traveler status are not necessarily threats, but, compared to trusted
travelers, they will receive more-intense scrutiny at airports and are more likely
to suffer some inconvenience.
64
Department of Defense (DoD) experience shows that the most important parts of
background checks are the national agency check, verification of employment,
and an understanding of an individual’s financial history. The most timeconsuming portion of the traditional background investigation—interviews with
friends and neighbors—is the least revealing and predictive.
The Office of Personnel Management (OPM) and DoD have relied on contractors
to conduct background investigations. While safeguards need to be in place to
ensure the integrity of the contractors and the investigative process, ample
precedent exists in both OPM and DoD for managing contractor efforts. The
Department of Transportation (DOT) could also certify the investigative
programs of individual companies and accept their findings for the purposes of
issuing trusted traveler cards.
The information required to obtain trusted traveler status does not need to be as
extensive as that required for security clearances. The key indicators would be
•
An unexceptional National Agency Check
•
A fingerprint check against criminal and civil files
•
A verified, stable employment history
•
A record demonstrating firm community roots
•
A record of financial security with no unexplained deviations from typical
patterns
•
A travel history consistent with occupational history
•
An employer’s statement of confidence.
Some of this information can be obtained from existing databases (e.g., financial,
employment, and travel histories) provided the individual agrees to grant access
to them for the purpose of obtaining a trusted traveler card. The decision of
whether to grant a card should be made by the DOT.
A trusted traveler program used in conjunction with the Computer Assisted
Passenger Profiling System (CAPPS) augmented by information from
intelligence and law enforcement agencies (both domestic and foreign) will
provide both the security and efficiency air travelers and the nation deserve.