Algorithms for Port of Entry
Inspection for WMDs
Fred S. Roberts
DyDAn Center
Rutgers University
1
Port of Entry Inspection Algorithms
•Goal: Find ways to intercept illicit
nuclear materials and weapons
destined for the U.S. via the
maritime transportation system
•Currently inspecting only small
% of containers arriving at ports
•Even inspecting 8% of containers in Port of
NY/NJ might bring international trade to a halt
(Larrabbee 2002)
2
Port of Entry Inspection Algorithms
•Aim: Develop decision support algorithms that
will help us to “optimally” intercept illicit
materials and weapons subject to limits on delays,
manpower, and equipment
•Find inspection schemes that minimize total
“cost” including “cost” of false alarms (“false
positives”) and failed alarms (“false negatives”)
Mobile Vacis: truckmounted gamma ray
imaging system
3
Port of Entry Inspection Algorithms
•My work on port of entry inspection has gotten
me and my students to some remarkable places.
Me on a Coast Guard
boat in a tour of the
harbor in Philadelphia
4
5
Sequential Decision Making Problem
•Stream of containers arrives at a port
•The Decision Maker’s Problem:
•Which to inspect?
•Which inspections next based on previous results?
•Approach:
–“decision logics”
–combinatorial optimization methods
–Builds on ideas of Stroud
and Saeger at Los Alamos
National Laboratory
–Need for new models
– and methods
6
Sequential Diagnosis Problem
•Such sequential diagnosis problems arise in many
areas:
–Communication networks (testing connectivity, paging
cellular customers, sequencing tasks, …)
–Manufacturing (testing machines, fault diagnosis,
routing customer service calls, …)
–Medicine (diagnosing patients, sequencing treatments,
…)
7
Sequential Decision Making Problem
•Containers arriving to be classified into categories.
•Simple case: 0 = “ok”, 1 = “suspicious”
•Inspection scheme: specifies which inspections are
to be made based on previous observations
8
Sequential Decision Making Problem
•0’s and 1’s suggest binary digits (bits)
•Bit String: A sequence of bits:
0001, 1101, …
•Boolean Function: A function that assigns to
each bit string a 0 or a 1.
Bit String x
00
01
10
11
B(x)
1
0
0
1
B(00) = 1, B(10) = 0
9
Sequential Decision Making Problem
•Bit strings are of course crucial carriers of
information in modern computers
•They are used to encode detailed instructions and
are translated to a sequence of on-off instructions
for switches in the computer.
•We will use them differently.
10
Counting Bit Strings and Boolean
Functions
•To understand the difficulty of the container
inspection problem, we will need to count bit
strings and Boolean functions.
•Product Rule of Counting: If something can
happen in n1 ways, and no matter how the first
thing happens, a second thing can happen in n2
ways, then the two things together can happen in
n1 x n2 ways.
11
Counting Bit Strings and Boolean
Functions
•Product Rule of Counting (More Generally): If
something can happen in n1 ways, and no matter
how the first thing happens, a second thing can
happen in n2 ways, and no matter how the first two
things happen, a third thing can happen in n3 ways,
and …, then all the things together can happen in
n1 x n2 x n3 x … ways.
12
Counting Bit Strings and Boolean
Functions
•How many bit strings are there of 3 bits?
•There are 2 choices for the first bit.
•No matter how it is chosen, there are 2 choices for
the second bit.
•No matter how the first 2 bits are chosen, there
are 2 choices for the third bit.
•Get 2 x 2 x 2 = 23 = 8
13
Counting Bit Strings and Boolean
Functions
•How many bit strings are there of 3 bits?
000, 001, 010, 011, 100, 101, 110, 111
How many bit strings are there of 5 bits?
25
How many bit strings are there of n bits?
2n
14
Counting Bit Strings and Boolean
Functions
•How many Boolean functions are there of 3
variables?
15
Counting Bit Strings and Boolean
Functions
•How many Boolean functions are there of 3
variables?
•There are 23 = 8 bit strings of 3 variables
• For each such bit string, the Boolean function
assigns it one of 2 different values (0 or 1).
16
Counting Bit Strings and Boolean
Functions
•How many Boolean functions are there of 3
variables?
•There are 23 = 8 bit strings of 3 variables
• For each such bit string, the Boolean function
assigns it one of 2 different values (0 or 1).
•There are 28 =
223
such Boolean functions.
17
Counting Bit Strings and Boolean
Functions
•How many Boolean functions are there of n
variables?
18
Counting Bit Strings and Boolean
Functions
•How many Boolean functions are there of n
variables?
•There are 2n bit strings of n variables
• For each such bit string, the Boolean function
assigns it one of 2 different values (0 or 1).
19
Counting Bit Strings and Boolean
Functions
•How many Boolean functions are there of n
variables?
•There are 2n bit strings of n variables
• For each such bit string, the Boolean function
assigns it one of 2 different values (0 or 1).
•There are
22n
such Boolean functions.
20
Counting Bit Strings and Boolean
Functions
How big is
22n
?
When n = 4, this is 65,536
21
Counting Bit Strings and Boolean
Functions
The problem of making a detailed design of a
digital computer usually involves finding a
practical circuit implementation of certain
functional behavior.
A computer device implements a Boolean
function.
In the early days of computing, the goal was to
create a catalogue listing a good circuit
implementation of each Boolean function.
This would require 65,536 entries!
22
Counting Bit Strings and Boolean
Functions
•Of course, luckily, by symmetry, some of these
examples are “equivalent”
For example, if we interchange two variables.
•In the early days of computing (1951), a group
from the Harvard Computation Laboratory
painstakingly listed all 65,536 Boolean functions
of 4 variables and determined which were
equivalent.
•(There were 222 types.)
23
Sequential Decision Making Problem
For Container Inspection
•Containers have attributes, each
in a number of states
•Sample attributes:
–Levels of certain kinds of chemicals or
biological materials
–Whether or not there are items of a certain
kind in the cargo list
–Whether cargo was picked up in a certain port
24
Sequential Decision Making Problem
•Currently used attributes:
–Does ship’s manifest set off an “alarm”?
–What is the neutron or Gamma emission
count? Is it above threshold?
–Does a radiograph image come up positive?
–Does an induced fission test come up positive?
Gamma
ray
detector
25
Sequential Decision Making Problem
•We can imagine many other attributes
• Research at DyDAn is concerned with general
algorithmic approaches.
•We seek a methodology not tied to today’s
technology.
•Detectors are evolving quickly.
26
Sequential Decision Making Problem
•Simplest Case: Attributes are in state 0 or 1
•Then: Container is a bit string like 011001
•So: Classification is a decision function F that
assigns each binary string to a category.
011001
F(011001)
If attributes 2, 3, and 6 are present, assign container to
category F(011001).
27
Sequential Decision Making Problem
•If there are two categories, 0 and 1, decision
function F is a Boolean function.
Example:
F(000) = F(111) = 1, F(abc) = 0 otherwise
This classifies a container as positive iff it has
none of the attributes or all of them.
1=
28
Sequential Decision Making Problem
•What if there are three categories, 0 ½, and 1?.
Example:
F(000) = 0, F(111) = 1, F(abc) = 1/2 otherwise
This classifies a container as positive if it has all of
the attributes, negative if it has none of the
attributes, and uncertain if it has some but not all
of the attributes.
•How many such decision functions are there if
there are three attributes?
29
Sequential Decision Making Problem
•What if there are three categories, 0 ½, and 1?
•How many such decision functions are there if
there are three attributes?
30
Sequential Decision Making Problem
•What if there are three categories, 0 ½, and 1?.
•How many such decision functions are there if
there are three attributes?
•There are 23 bit strings of length 3.
31
Sequential Decision Making Problem
•What if there are three categories, 0 ½, and 1?.
•How many such decision functions are there if
there are three attributes?
•There are 23 bit strings of length 3.
•For each of them, there are 3 choices for the value
of F.
32
Sequential Decision Making Problem
•What if there are three categories, 0 ½, and 1?.
•How many such decision functions are there if
there are three attributes?
•There are 23 bit strings of length 3.
•For each of them, there are 3 choices for the value
of F.
•Thus, there are
323
such decision functions.
33
Sequential Decision Making Problem
•Given a container, test its attributes until know
enough to calculate the value of F.
•An inspection scheme tells us in which order to
test the attributes to minimize cost.
•Even this simplified problem is hard
computationally.
34
Sequential Decision Making Problem
•This assumes F is known.
•Simplifying assumption: Attributes are
independent.
•At any point we stop inspecting and output the
value of F based on outcomes of inspections so
far.
•Complications: May be precedence relations in
the components (e.g., can’t test attribute a4 before
testing a6.
•Or: cost may depend on attributes tested before.
•F may depend on variables that cannot be
directly tested or for which tests are too costly. 35
Sequential Decision Making Problem
•Such problems are hard computationally.
•There are many possible Boolean functions F.
•Even if F is fixed, problem of finding a good
classification scheme (to be defined precisely
below) is NP-complete – it is hard in a precise
computer science sense.
36
Sensors and Inspection Lanes
•n types of sensors measure presence or absence of the n
attributes.
•Many copies of each sensor.
•Complication: different characteristics of sensors.
•Entities come for inspection.
•Which sensor of a given type to
use?
•Think of inspection lanes and
waiting on line for inspection
•Besides efficient inspection
schemes, could decrease costs by:
–Buying more sensors
–Change allocation of containers to sensor lanes.
37
Trees
•A tree for us is a directed graph.
•It has nodes (vertices).
•Directed edges or arcs head
from a vertex to another.
•There are no “cycles”
(you can’t double back on
yourself)
•We will deal with rooted trees: One node is a root.
•All arcs point downwards in our diagrams, starting from
the root.
•If each node has two or zero outgoing (downwards) arcs,
we have a binary tree.
•Nodes with no outgoing arcs are called leaves.
38
Binary Decision Tree Approach
•Sensors measure presence/absence of attributes:
so 0 or 1
•Use two categories: 0, 1
•Binary Decision Tree:
–Nodes are sensors or categories
–Two arcs exit from each sensor node, labeled
left and right.
–Take the right arc when sensor says the
attribute is present, left arc otherwise
39
Binary Decision Tree Approach
•Reach category 1 from the
root only through the path
a0 to a1 to 1.
•Container is classified in
category 1 iff it has both
attributes a0 and a1 .
•Corresponding Boolean
function:
• F(11) = 1, F(10) = F(01)
= F(00) = 0.
Figure 1
40
Binary Decision Tree Approach
•Reach category 1 from the
root only through the path a1
to a0 to 1.
•Container is classified in
category 1 iff it has both
attributes a0 and a1 .
•Corresponding Boolean function:
• F(11) = 1, F(10) = F(01) = F(00)
= 0.
•Note: Different tree, same
function
Figure 1
41
Binary Decision Tree Approach
•Reach category 1 from the
root only through the path a0
to 1 or a0 to a1 to 1.
•Container is classified in
category 1 iff it has attribute
a0 or attribute a1 .
•Corresponding Boolean function:
• F(11) = 1, F(10) = F(01) = 1,
F(00) = 0.
Figure 1
42
Binary Decision Tree Approach
•Can you find another binary
tree that calculates the same
Boolean function?
•Sure, just interchange
nodes a0 and a1
Figure 1
43
Binary Decision Tree Approach
•Reach category 1 from
the root by:
a0 L to a1 R a2 R 1 or
a0 R a2 R1
•Container classified in
category 1 iff it has
a1 and a2 and not a0 or
a0 and a2 and possibly a1.
•Corresponding Boolean
function:
• F(111) = F(101) = F(011) = 1,
F(abc) = 0 otherwise.
Figure 2
44
Binary Decision Tree Approach
•This binary decision
tree corresponds to the
same Boolean function
F(111) = F(101) =
F(011) = 1, F(abc) = 0
otherwise.
However, it has one less
observation node ai. So,
it is more efficient if all
observations are equally
costly and equally likely.
Figure 3
45
Binary Decision Tree Approach
•So we have seen that a given Boolean function
may correspond to different binary decision trees.
•How do we find a binary decision tree
corresponding to a Boolean function?
•How do we find a least cost one?
Port of Long Beach
46
Binary Decision Tree Approach
•So we have seen that a given Boolean function
may correspond to different binary decision trees.
•How do we find a binary decision tree
corresponding to a Boolean function F?
•Brute force method:
Start with root a0, put both left and right arcs
to an a1.
From each a1 put both left and right arcs to
a2, etc.
At the end, each path through the tree
corresponds to a bit string x and let that
47
path end at F(x).
Binary Decision Tree Approach
a0
a1
a2
a1
a2
a2
a2
F(010)
48
Binary Decision Tree Approach
x
111
110
101
100
F(x)
1
1
0
1
x
011
010
001
000
F(x)
1
0
0
0
a0
a1
a2
0
a1
a2
0
0
a2
a2
1 1 0
1
1
49
Binary Decision Tree Approach
•Even if the Boolean function F is fixed, the
problem of finding the “least cost” binary
decision tree for it is very hard (NP-complete).
•For small n = number of attributes, can try to
solve it by trying all possible binary decision trees
corresponding to the Boolean function F.
Port of Long Beach
•Even for n = 4, not practical. (n = 4 at Port of
Long Beach-Los Angeles)
50
Binary Decision Tree Approach
Promising Approaches:
•Heuristic algorithms, approximations to optimal.
•Special assumptions about the Boolean function F.
•For “monotone” Boolean functions, integer
programming formulations give promising
heuristics.
•Stroud and Saeger (Los Alamos
National Lab) enumerate all
“complete,” monotone Boolean functions
and calculate the least expensive
corresponding binary decision trees.
•Their method practical for n up to 4, not n = 5.51
Binary Decision Tree Approach
Monotone Boolean Functions:
•Given two bit strings x1x2…xn, y1y2…yn
•Suppose that xi  yi for all i implies that
F(x1x2…xn)  F(y1,y2…yn).
•Then we say that F is monotone.
•Then 11…1 has highest probability of being in
category 1.
52
Binary Decision Tree Approach
Monotone Boolean Functions:
•Given two bit strings x1x2…xn, y1y2…yn
•Suppose that xi  yi for all i implies that
F(x1x2…xn)  F(y1,y2…yn).
•Then we say that F is monotone.
•Example:
•n = 4, F(x) = 1 iff x has at least two 1’s.
•F(1100) = F(0101) = F(1011) = 1, F(1000) = 0,
etc.
•Is this monotone?
•Yes
53
Binary Decision Tree Approach
Incomplete Boolean Functions:
•Boolean function F is incomplete if F can be
calculated by finding at most n-1 attributes and
knowing the value of the input string on those
attributes
•Example: F(111) = F(110) = F(101) = F(100) =
1, F(000) = F(001) = F(010) = F(011) = 0.
•F(abc) is determined without knowing b (or c).
•F is incomplete.
54
Binary Decision Tree Approach
Complete, Monotone Boolean Functions:
•Stroud and Saeger: algorithm for enumerating
binary decision trees implementing complete,
monotone Boolean functions.
•Feasible to implement up to n = 4.
•Then you can find least cost tree by enumerating
all binary decision trees corresponding to a given
Boolean function and repeating this for all
complete, monotone Boolean functions.
55
Binary Decision Tree Approach
Complete, Monotone Boolean Functions:
•Stroud and Saeger: algorithm for enumerating
binary decision trees implementing complete,
monotone Boolean functions.
•n = 2:
–There are 6 monotone Boolean functions.
–Only 2 of them are complete, monotone
–There are 4 binary decision trees for
calculating these 2 complete, monotone
Boolean functions.
56
Binary Decision Tree Approach
Complete, Monotone Boolean Functions:
•n = 3:
–9 complete, monotone Boolean functions.
–60 distinct binary trees for calculating them
–Counting methods more complicated than
simple ones we have described before
–All counts here are from Stroud and Saeger
57
Binary Decision Tree Approach
Complete, Monotone Boolean Functions:
•n = 4:
–114 complete, monotone Boolean functions.
–11,808 distinct binary decision trees for
calculating them.
–(Compare 1,079,779,602 BDTs for all Boolean
functions)
58
Binary Decision Tree Approach
Complete, Monotone Boolean Functions:
•n = 5:
–6894 complete, monotone Boolean functions
–263,515,920 corresponding binary decision
trees.
•Combinatorial explosion!
•Need alternative approaches; enumeration not
feasible!
•(Even worse: compare 5 x 1018 BDTs
corresponding to all Boolean functions)
59
Cost Functions
•So far, we have figured one binary decision tree
is cheaper than another if it has fewer nodes.
•This is oversimplified.
•There are more complex costs involved than
number of sensors in a tree.
60
Cost Functions
•Stroud-Saeger method applies to more
sophisticated cost models, not just cost =
number of sensors in the BDT.
•Using a sensor has a cost:
–Unit cost of inspecting one item with it
–Fixed cost of purchasing and deploying it
–Delay cost from queuing up at the sensor
station
•Preliminary problem: disregard fixed and delay
61
costs. Minimize unit costs.
Cost Functions
•Simplification so far: Disregard characteristics
of population of entities being inspected.
•Only count number of observation (attribute)
nodes in the tree.
•Unit Cost Complication: How many nodes of
the decision tree are actually visited during
average container’s inspection? Depends on
“distribution” of containers.
•Answer can depend on probability of sensor
errors and probability of bomb in a container.
62
Cost Functions: Delay Costs
•Tradeoff between fixed costs and delay costs:
Add more sensors cuts down on delays.
•More sophisticated models describe the process
of containers arriving
•There are differing delay times for inspections
•Use “queuing theory” to find average delay
times under different models
63
Cost Functions: Delay Costs
•There are differing delay times for inspections
•Use “queuing theory” to find average delay
times under different models
Explorations for Students: Explore the
fundamental principles of “queueing theory”
64
Cost Functions:
Unit Costs
Tree Utilization
•Complication: Assume cost depends on how many
nodes of BDT are actually visited during an “average”
container’s inspection. (This is sum of unit costs.)
•Depends on characteristics of population of entities
being inspected.
•I.e., depends on “distribution” of containers.
•In our early models, we assume we are given
probability of sensor errors and probability of bomb in
a container.
•This allows us to calculate “expected” cost of
65
utilization of the tree Cutil. (Details omitted here.)
Cost Functions
•Cost of false positive: Cost of additional
tests.
–If it means opening the container, it’s
very expensive.
•Cost of false negative:
–Complex issue.
–What is cost of a bomb going off in
Manhattan?
66
Cost Functions: Sensor Errors
•One Approach to False Positives/Negatives:
Assume there can be Sensor Errors
•Simplest model: assume that all sensors checking
for attribute ai have same fixed probability of
saying ai is 0 if in fact it is 1, and similarly
saying it is 1 if in fact it is 0.
•More sophisticated analysis later describes a
model for determining probabilities of sensor
errors.
•Notation: X = state of nature (bomb or no bomb)
Y = outcome (of sensor or entire inspection
67
process).
Probability of Error for The Entire Tree
State of nature is zero (X =
0), absence of a bomb
State of nature is one (X =
1), presence of a bomb
A
A
C
0
B
0
B
0
C
1
1
Probability of false positive
(P(Y=1|X=0))
for this tree is given by
0
1
1
Probability of false negative
(P(Y=0|X=1))
for this tree is given by
P(Y=1|X=0) = P(YA=1|X=0) * P(YB=1|X=0)
+ P(YA=1|X=0) *P(YB=0|X=0)* P(YC=1|X=0)
P(Y=0|X=1) = P(YA=0|X=1) +
P(YA=1|X=1) *P(YB=0|X=1)*P(YC=0|X=1)
Pfalsepositive
Pfalsenegative
68
Probability of Error for The Entire Tree
State of nature is zero (X =
0), absence of a bomb
A
B
0
C
0
1
1
Probability of false positive
(P(Y=1|X=0))
for this tree is given by
Note the use of the
product rule: probability
is ratio between number
of ways something we are
interested in can happen
and total number ways
things can happen.
P(Y=1|X=0) = P(YA=1|X=0) * P(YB=1|X=0)
+ P(YA=1|X=0) *P(YB=0|X=0)* P(YC=1|X=0)
Pfalsepositive
69
Probability of Error for The Entire Tree
State of nature is zero (X =
0), absence of a bomb
A
B
0
C
0
1
1
Probability of false positive
(P(Y=1|X=0))
for this tree is given by
Also uses the sum rule of
counting: If one event can
happen in n1 ways and a
second event in n2
different ways, then there
are n1+n2 ways in which
either the first event or the
second event can occur
(but not both).
P(Y=1|X=0) = P(YA=1|X=0) * P(YB=1|X=0)
+ P(YA=1|X=0) *P(YB=0|X=0)* P(YC=1|X=0)
Pfalsepositive
70
Aside: Illustrating the Sum Rule
For example, how many bit strings are there of 4
bits so that there are either exactly three 1’s or
exactly three 0’s?
For bit strings of three 1’s, we have four choices
for the place to put the 0, so there are 4 such
strings.
For bit strings with three 0’s, there are similarly 4
such strings.
By the sum rule, there are 8 bit strings with
exactly three 1’s or exactly three 0’s, since it is
impossible to have both.
1110, 1101, 1011, 0111, 0001, 0010, 0100, 1000
71
Cost Function used for Evaluating
the Decision Trees.
CTot = CFalsePositive *PFalsePositive + CFalseNegative *PFalseNegative +
Cutil
CFalsePositive is the cost of false positive (Type I error)
CFalseNegative is the cost of false negative (Type II error)
PFalsePositive is the probability of a false positive occurring
PFalseNegative is the probability of a false negative occurring
Cutil is the expected cost of utilization of the tree.
72
Cost Function used for Evaluating
the Decision Trees.
CFalsePositive is the cost of false positive (Type I error)
CFalseNegative is the cost of false negative (Type II error)
PFalsePositive is the probability of a false positive occurring
PFalseNegative is the probability of a false negative occurring
Cutil is the expected cost of utilization of the tree.
PFalsePositive and PFalseNegative are calculated from the tree.
Cutil is calculated from tree and probabilities of bomb in
container and probability of sensor errors.
CFalsePositive, CFalseNegative are input – given information.
73
Stroud Saeger Experiments
• Stroud-Saeger ranked all trees formed
from 3 or 4 sensors A, B, C and D
according to increasing tree costs.
• Used cost function defined above.
• Values used in their experiments:
– CA = .25; P(YA=1|X=1) = .90; P(YA=1|X=0) = .10;
– CB = 10; P(YC=1|X=1) = .99; P(YB=1|X=0) = .01;
– CC = 30; P(YD=1|X=1) = .999; P(YC=1|X=0) = .001;
– CD = 1; P(YD=1|X=1) = .95; P(YD=1|X=0) = .05;
– Here, Ci = unit cost of utilization of sensor i.
• Also fixed were: CFalseNegative, CFalsePositive, P(X=1)
74
Sensitivity Analysis
• When parameters in a model are not known
exactly, the results of a mathematical analysis
can change depending on the values of the
parameters.
• It is important to do a sensitivity analysis: let the
parameter values vary and see if the results
change.
• So, do the least cost trees change if we change
values like probability of a bomb, cost of a false
positive, etc?
75
Stroud Saeger Experiments: Our
Sensitivity Analysis
• We have explored sensitivity of the StroudSaeger conclusions to variations in values of the
three parameters:
CFalseNegative, CFalsePositive, P(X=1)
• (Work of Anand, Madigan, Mammone, Pathak,
Roberts)
• We estimated high and low values for the
parameters.
76
Stroud Saeger Experiments: Our
Sensitivity Analysis
– CFalseNegative was varied between 25 million and 10
billion dollars
• Low and high estimates of direct and indirect costs
incurred due to a false negative.
– CFalsePositive was varied between $180 and $720
• Cost incurred due to false positive
(4 men * (3 -6 hrs) * (15 – 30 $/hr)
– P(X=1) was varied between 1/10,000,000 and
1/100,000
77
Stroud Saeger Experiments: Our
Sensitivity Analysis
– CFalseNegative was varied between 25 million and 10
billion dollars
• Low and high estimates of direct and indirect costs
incurred due to a false negative.
– CFalsePositive was varied between $180 and $720
• Cost incurred due to false positive
(4 men * (3 -6 hrs) * (15 – 30 $/hr)
– P(X=1) was varied between 1/10,000,000 and
1/100,000
– Extra assignment: think about whether these ranges
are reasonable and how one would determine them?
78
Stroud Saeger Experiments: Our
Sensitivity Analysis
•
•
•
•
•
•
n = 3 (use sensors A, B, C)
Varied the parameters
CFalseNegative, CFalsePositive, P(X=1)
We chose the value of one of these parameters from the
interval of values
Then explored the highest ranked tree as the other two
were chosen at random in the interval of values.
10,000 experiments for each fixed value.
We looked for the variation in the top-ranked tree and
how the top-rank related to choice of parameter values.
Very surprising results.
79
Frequency of Top-ranked Trees when
CFalseNegative and CFalsePositive are Varied
7000
1st
2nd
3rd
4th
5th
6000
Frequency
5000
4000
3000
2000
1000
0
0
10
20
30
40
50
60
Tree no.
•
•
10,000 randomized experiments (randomly selected values of CFalseNegative and
CFalsePositive from the specified range of values) for the median value of P(X=1).
The above graph has frequency counts of the number of experiments when a
particular tree was ranked first or second or third and so on.
• Only three trees (7, 55 and 1) ever came first. 6 trees came second,
10 came third, 13 came fourth.
80
Frequency of Top-ranked Trees when
CFalseNegative and P(X=1) are Varied
8000
1st
2nd
3rd
4th
5th
7000
6000
Frequency
5000
4000
3000
2000
1000
0
0
10
20
30
40
50
60
Tree no.
• 10,000 randomized experiments for the median value of CFalsePositive.
• Only 2 trees (7 and 55) ever came first. 4 trees came second. 7
trees came third. 10 and 13 trees came 4th and 5th respectively.
81
Frequency of Top-ranked Trees when
P(X=1) and CFalsePositive are Varied
7000
1st
2nd
3rd
4th
5th
6000
Frequency
5000
4000
3000
2000
1000
0
0
10
20
30
40
50
60
Tree no.
• 10,000 randomized experiments for the median value of CFalseNegative.
• Only 3 trees (7, 55 and 1) ever came first. 6 trees came second. 10
82
trees came third. 13 and 16 trees came 4th and 5th respectively.
Most Frequent Tree Groups Attaining
the Top Three Ranks.
• Trees 7, 9 and 10
A
B
B
0
C
0
1
A
A
1
C 0
0
0
B
1
A
C
1
A 0
0
0
1
1
All the three decision trees have been generated from the same
Boolean function.
Both Tree 9 and Tree 10 are ranked second and third more than
83
99% of the times when Tree 7 is ranked first.
Most Frequent Tree Groups Attaining
the Top Three Ranks
• Trees 55, 57 and 58
A
1
B
1
1
1
C
1
C
0
B
1
A
1
C
0
B
1
A
0
1
All three trees correspond to the same Boolean function.
Tree ranked 57 is second 96% of the times and tree 58 is third
79 % of the times when tree 55 is ranked first.
84
Most Frequent Tree Groups Attaining
the Top Three Ranks
• Trees 1, 3, and 2
A
B
B
0
0
A
0
0
C
0
A
1
0
C
0
C
0
1
B
0
1
All three trees correspond to the same Boolean function.
Tree 3 is ranked second 98% of times and tree 2 is ranked third
80 % of the times when tree 1 is ranked first.
85
Stroud Saeger Experiments:
Sensitivity Analysis: 4 Sensors
• Second set of computer experiments: n = 4
(use sensors, A, B, C, D).
• Same values as before.
• Experiment 1: Fix values of two of CFalseNegative,
CFalsePositive, P(X=1) and vary the third through
their interval of possible values.
• Experiment 2: Fix a value of one of CFalseNegative,
CFalsePositive, P(X=1) and vary the other two.
• Do 10,000 experiments each time.
86
• Look for the variation in the highest ranked tree.
Tree Structure For Top Trees
a
a
b
b
b
b
c
d
0
c
1
0
d
c
1
1 0
Tree number 11485
1
1
1
c
1
d
c
d
0
d
0
0
1
1 0
1
d
1
1
Tree number 10129
87
Tree Structure For Top Trees
a
a
b
b
b
b
c
d
0
c
1
0
d
c
1
1 0
Tree number 11485
1
1
1
c
1
d
c
d
0
d
0
0
1
1 0
1
d
1
1
Tree number 10129
Explorations for Students: Compare the Boolean functions
corresponding to these two trees. See where they differ. How close88
are they? Can you make closeness precise?
Modeling Sensor Errors
•One Approach to Sensor Errors: Modeling
Sensor Operation
•Threshold Model:
–Sensors have different discriminating power
–Many use counts (e.g., Gamma radiation
counts)
–See if count exceeds
threshold
–If so, say attribute is present.
89
Conclusions from Sensitivity
Analysis
• Considerable lack of sensitivity to
modification in parameters for trees using 3 or
4 sensors.
• Very few optimal trees.
• Very few Boolean functions arise among
optimal and near-optimal trees.
• Surprising results.
90
Some Research Challenges
•Explain why conclusions are so insensitive to
variation in parameter values.
•Explore the structure of the optimal trees and
compare the different optimal trees.
•Develop less brute force methods for finding
optimal trees that might work if there are more
than 4 attributes.
•Develop methods for
approximating the optimal tree.
Pallet vacis
91
Closing Remark
•Recall that the “cost” of inspection includes the
cost of failure, including failure to foil a terrorist
plot.
•There are many ways to lower the total “cost”
of inspection:
Use more efficient
orders of inspection.
Find ways to inspect
more containers.
Find ways to cut down
on delays at inspection lanes.
92
Research Team
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Saket Anand, Rutgers, ECE graduate student
Endre Boros, Rutgers, Operations Research
Elsayed Elsayed, Rutgers, Ind. & Systems Engineering
Liliya Fedzhora, Rutgers, Operations Res. grad. student
Paul Kantor, Rutgers, Schl. of Infor. & Library Studies
Abdullah Karaman, Rutgers Ind. & Syst. Eng. grad. student
Alex Kogan, Rutgers, Business School
Paul Lioy, Rutgers/UMDNJ, Environmental and Occupational Health and
Sciences Institute
David Madigan, Rutgers, Statistics
Richard Mammone, Rutgers, Center for Advanced Information Processing
S. Muthukrishnan, Rutgers, Computer Science
Saumitr Pathek, Rutgers ECE graduate student
Richard Picard, Los Alamos, Statistical Sciences Group
Fred Roberts, Rutgers, DIMACS Center
Kevin Saeger, Los Alamos, Homeland Security
Phillip Stroud, Los Alamos, Systems Engineering and Integration Group
Hao Zhang, Rutgers Ind. & Systems Eng., graduate student
93
Collaborators on Sensitivity Analysis:
• Saket Anand
• David Madigan
• Richard Mammone
• Saumitr Pathak
Research Support:
• Office of Naval Research
• National Science Foundation
Los Alamos National Laboratory:
• Rick Picard
• Kevin Saeger
• Phil Stroud
94
Mathematics can help with homeland security!
95