Rule-Based Anomaly Pattern
Detection for Detecting
Disease Outbreaks
r
ente
pres
r
u
o
Y
y
toda
Andrew Moore
Weng-Keen Wong
Mike Wagner
Greg Cooper
In collaboration with many others, including Wendy Chapman, Bill Hogan,
Bob Olszewski, Jeff Schneider , Rich Tsui, Weng-Keen Wong
University of Pittsburgh and Carnegie Mellon University
www.cs.cmu.edu/~awm
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 1
The Auton Lab
Faculty: Andrew Moore, Jeff Schneider, Artur Dubrawski
Postdoctoral Fellows: Brigham Anderson, Alexander Gray, Paul Komarek,
Dan Pelleg
Graduate Students: Brent Bryan, Kaustav Das, Khalid El-Arini, Anna
Goldenberg, Jeremy Kubica, Ting Liu, Daniel Neill,
Sajid Siddiqi, Purna Sarkar, Ajit Singh, Weng-Keen
Wong
Head of Software Jeanie Komarek
Development:
Programmers: Patrick Choi, Adam Goode, Pat Gunn, Joey Liang, John
Ostlund, Robin Sabhnani, Rahul Sankathar
Executive Assistant: Kristen Schrauder
Head of Sys. Admin: Jacob Joseph
Undergraduate and
Masters Interns:
Recent Alumni:
Copyright © 2002-2004, Andrew Moore
Kenny Daniel, Sandy Hsu, Dongryeol Lee, Jennifer Lee, Avilay Parekh, Chris
Rotella, Jonathan Terleski
Drew Bagnell (RI faculty), Scott Davies (Google), David Cohn (Google), Geoff
Gordon (CMU), Paul Hsiung (USC), Marina Meila (U. Washington), Remi Munos
(Ecole Polytechnique), Malcolm Strens (Qinetiq)
Biosurveillance: Slide 2
1
Current Sponsors
•
•
•
•
•
•
•
•
•
•
•
•
•
•
National Science Foundation (NSF)
NASA
Defense Advanced Research Projects Agency (DARPA)
Other Government Sponsor
Department of Homeland Security (DHS)
Homeland Security Advanced Research Projects Agency
(HSARPA)
United States Department of Agriculture (USDA)
Health Canada
State
State of Pennsylvania
A Fortune-50 Pharmaceutical company
Caterpillar Corporation
Large industrial
A Fortune-50 Petrochemical company
Psychogenics Corporation
Small industrial
Transform Pharmaceuticals
Copyright © 2002-2004, Andrew Moore
Federal
Biosurveillance: Slide 3
Collaborators...
Ting Liu
Paul Hsiung
Andy
Connolly
Bob Nichol
Jeff Schneider
Istvan
Szapudi
Chris
Genovese
Larry
Wasserman
Alex Szalay
Alex Gray
Brigham
Anderson
Anna
Goldenberg
Kan Deng
Greg Cooper
Mike Wagner
Ajit Singh
Andrew Lawson
Daniel Neill
Scott Davies
Rich Tsui
Jeremy Kubica
Sajid
Drew
Paul Komarek
Weng-Keen Wong
Copyright © 2002-2004, Bagnell
Andrew Moore
Siddiqi
Dan Pelleg
Biosurveillance: Slide 4
2
1996
1997
1998
1999
2000
2001
2002
2003
M&M
Mars
Line
control
Adrenali
ne
(NOX
minimalizati
on)
Kodak
(Image
stabilizatio
n)
Digital
Equipm
ent
(pregna
n-cy
monitoring)
M&M
Mars
(manufa
c-turing)
NASA/
NSF
(Astrop
hysics
mining)
3M:
Textile
tension
control
Caterpillar
(Spare
parts)
US Army
(biotoxin
detection)
M&M
Mars:
Schedulin
g with
uncertainty
3M
(Adhesive
design)
DigitalMC
(Music
tastes)
Caterpillar
(emissions)
SmartMoney
(anomalies)
Unilever
(Brand
Management
)
Phillips
Petroleum
(work-force
optimization)
Cellomics
(screened
anomaly
detection)
Biometrics
company
(health
monitor)
Boeing
(intrusion)
Masterfoods
(new product
development)
Cellomics
(pro-teomics
screen)
ABB (Circuitbreaker
supply chain)
SwissAir
(Flight delays)
3M (secret)
Washington
Public
Hospital
System
(ER delays)
Unilever
(targeted
marketing)
NASA (National
Virtual
Observatory)
NSF (astrostatistics
software)
DARPA (national
disease monitor)
Masterfoods (biochemistry)
Pfizer (Highthroughput screen)
Caterpillar Inc.
(Self-optimizing
Engines)
Beverage
Company
(Ingredients/Manuf
acturing/Marketing/
Sales Bayes Net)
Transform Pharma
(massive
autonomous
experiment design)
Census Bureau
(privacy protection)
Psychogenics Inc:
Effects of
psychotropic drugs
on rats
NSF (astrostatistics
software)
Masterfoods (biochemistry)
State of PA (National
Disease Monitor [with
Mike Wagner of U. Pitt])
State of PA (Anti
Cancer [collaboration
with CMU Biology]
DARPA (detecting
patterns in links)
Other Government
Departments
(identifying dangerous
people, potential
collaborators, and
aliases)
Other Government
Departments (detecting
a class of clusters)
Other Pharma
Research Co. Life
Science specific data
mining
United States
Department of
Agriculture: Early
warning system for food
terrorism
NSF: Biosurveillance
Algorithms
Au
De ton/S
plo
ym P R
ent
s
Copyright © 2002-2004, Andrew Moore
Our 5 biggest
applications in 2004
Biosurveillance: Slide 5
Biomedical Security (with Mike
Wagner, University of Pittsburgh)
Autonomous selftweaking engines
Drug Screening
Intelligence Data
Big Astrophysics Automated Science
Record # 456621:
Big Noisy Database of Links
There must be
something useful
here… but I’m
drowning in noise
Doe and Smith
were booked in
the same hotel
on 3/6/93
Analyst
Potential coll-aborators
of Doe:
Big Noisy Database of Links
This is more
useful
objective
information
Copyright © 2002-2004, Andrew Moore
TGAT
The pattern of travel
among the group
consisting of (Doe,
Smith, Jones, Moore)
during 93-96 can’t be
explained by
coincidence
Biosurveillance: Slide 6
3
Our 5 biggest
applications in 2002
Biomedical Security (with Mike
Wagner, University of Pittsburgh)
Autonomous selftweaking engines
Drug Screening
Security Surveillance Mining
Big Astrophysics Automated Science
Record # 456621:
Big Noisy Database of Links
There must be
something useful
here… but I’m
drowning in noise
Doe and Smith
were booked in
the same hotel
on 3/6/93
Analyst
Potential coll-aborators
of Doe:
Big Noisy Database of Links
TGAT
This is more
useful
objective
information
Copyright © 2002-2004, Andrew Moore
The pattern of travel
among the group
consisting of (Doe,
Smith, Jones, Moore)
during 93-96 can’t be
explained by
coincidence
Biosurveillance: Slide 7
..Early Thursday Morning. Russia. April 1979...
Sverdlovsk
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 8
4
Sverdlovsk
•During April and
May 1979, there
were 77
Confirmed
cases of
inhalational
anthrax
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 9
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 10
5
What if this happened in 2002?
Salt-lake city, Feb 2002
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 11
What if this happened in 2002?
For a realistic attack,
during peak period of
importance, each
extra hour of early
detection could save
2000 lives.
Salt-lake city, Feb 2002
Copyright © 2002-2004, Andrew Moore
[Tsui et al., 2000]
Biosurveillance: Slide 12
6
UPMC BRADDOCK
UPMC MCKEESPORT
MAGEE-WOMENS HOSPITAL
UPMC PRESBYTERIAN
UPMC SHADYSIDE
UPMC SOUTH SIDE
UPMC ST MARGARET
LEE HOSPITAL
CONEMAUGH MEMORIAL
MINERS HOSPITAL
ALLEGHENY GENERAL
FORBES REGIONAL
MEYERSDALE
MERCY
WINBER
30% of ER visits in 13-county
metropolitan area
(population 3.0 Million)
52% of ER visits in
Allegheny County
(population 1.3 Million)
How is RODS Used?
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 13
When an alarm sounds...
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 14
7
When an alarm sounds...
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 15
How is RODS Used?
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 16
8
The task
How strange is everything that’s
happened in the last n hours?
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 17
The task
How strange is everything that’s
happened in the last n hours?
Should we launch
active data
collection?
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 18
9
The task
How strange is everything that’s
happened in the last n hours?
How strange is it, under the
hypothesis that we have the
following population of diseases
with the following characteristics?
Should we launch
active data
collection?
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 19
The task
How strange is everything that’s
happened in the last n hours?
How strange is it, under the
hypothesis that we have the
following population of diseases
with the following characteristics?
Should we launch
active data
collection?
Copyright © 2002-2004, Andrew Moore
What is the chance that
disease X is currently in the
population?
Biosurveillance: Slide 20
10
The task
How strange is everything that’s
happened in the last n hours?
Can we find a
pattern in the
strange stuff?
Should we launch
active data
collection?
How strange is it, under the
hypothesis that we have the
following population of diseases
with the following characteristics?
What is the chance that
disease X is currently in the
population?
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 21
WSARE v2.0
• What’s Strange About Recent Events?
• Designed to be easily applicable to any
multivariate many-attribute date/timeindexed biosurveillance-relevant data
stream.
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 22
11
WSARE v2.0
• Inputs:
1. Date/time-indexed
biosurveillancerelevant data stream
2. Time Window
Length
3. Which
attributes to use?
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 23
WSARE v2.0
• Input
s:
1. Date/time-indexed
biosurveillancerelevant data stream
2. Time Window
Length
Primary Date
Key
Time
“ignore key and
weather”
“last 24 hours”
Example
3. Which
attributes to use?
Hospital ICD9 Prodrome Gender Age Home
Work
Recent Recent (Many
Flu
Weather more…)
Large Medium Fine Large Medium Fine
Levels
Scale Scale
Scale Scale Scale
Scale
h6r32 6/2/2 14:12 Down- 781 Fever
town
M
20s NE
15217 A5
NW 15213 B8
2%
70R
…
t3q15 6/2/2 14:15 River- 717 Respirat M
side
ory
60s NE
15222 J3
NE
15222 J3
2%
70R
…
t5hh5 6/2/2 14:15 Smith- 622 Respirat F
field
ory
80s SE
15210 K9
SE
15210 K9
2%
70R
…
:
:
:
:
:
:
Copyright © 2002-2004, Andrew Moore
:
:
:
:
:
:
:
:
:
:
:
Biosurveillance: Slide 24
12
WSARE v2.0
• Inputs:
1. Date/time-indexed
biosurveillancerelevant data stream
• Outputs: 1. Here are the
2. Time Window
Length
3. Which
attributes to use?
2. Here’s why
3. And here’s
how seriously you
should take it
records that most
surprise me
Primary Date
Key
Time
Hospital ICD9 Prodrome Gender Age Home
Work
Recent Recent (Many
Flu
Weather more…)
Large Medium Fine Large Medium Fine
Levels
Scale Scale
Scale Scale Scale
Scale
h6r32 6/2/2 14:12 Down- 781 Fever
town
M
20s NE
15217 A5
NW 15213 B8
2%
70R
…
t3q15 6/2/2 14:15 River- 717 Respirat M
side
ory
60s NE
15222 J3
NE
15222 J3
2%
70R
…
t5hh5 6/2/2 14:15 Smith- 622 Respirat F
field
ory
80s SE
15210 K9
SE
15210 K9
2%
70R
…
:
:
:
:
:
:
:
:
:
:
:
:
:
:
Copyright © 2002-2004, Andrew Moore
:
:
:
Biosurveillance: Slide 25
Simple WSARE
• Given 500 day’s
worth of ER cases at
15 hospitals…
Date
Cases
Thu 5/22/2000
C1, C2, C3, C4 …
Fri 5/23/2000
C1, C2, C3, C4 …
:
:
:
:
Sat 12/9/2000
C1, C2, C3, C4 …
Sun 12/10/2000 C1, C2, C3, C4 …
Copyright © 2002-2004, Andrew Moore
:
:
Sat 12/16/2000
C1, C2, C3, C4 …
:
:
Sat 12/23/2000
C1, C2, C3, C4 …
:
:
:
:
Fri 9/14/2001
C1, C2, C3, C4 …
Biosurveillance: Slide 26
13
Simple WSARE
• Given 500 day’s
worth of ER cases at
15 hospitals…
• For each day…
• Take today’s cases
Date
Cases
Thu 5/22/2000
C1, C2, C3, C4 …
Fri 5/23/2000
C1, C2, C3, C4 …
:
:
:
:
Sat 12/9/2000
C1, C2, C3, C4 …
Sun 12/10/2000 C1, C2, C3, C4 …
:
:
Sat 12/16/2000
C1, C2, C3, C4 …
:
:
Sat 12/23/2000
C1, C2, C3, C4 …
:
:
:
:
Fri 9/14/2001
C1, C2, C3, C4 …
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 27
Simple WSARE
• Given 500 day’s
worth of ER cases at
15 hospitals…
• For each day…
• Take today’s cases
• The cases one week ago
• The cases two weeks ago
Copyright © 2002-2004, Andrew Moore
Date
Cases
Thu 5/22/2000
C1, C2, C3, C4 …
Fri 5/23/2000
C1, C2, C3, C4 …
:
:
:
:
Sat 12/9/2000
C1, C2, C3, C4 …
Sun 12/10/2000 C1, C2, C3, C4 …
:
:
Sat 12/16/2000
C1, C2, C3, C4 …
:
:
Sat 12/23/2000
C1, C2, C3, C4 …
:
:
:
:
Fri 9/14/2001
C1, C2, C3, C4 …
Biosurveillance: Slide 28
14
Simple WSARE
• Given 500 day’s worth
of ER cases at 15
hospitals…
• For each day…
DATE_ADMICD9
12/9/00
12/9/00
12/9/00
12/9/00
:
PRODROMGENDER
786.05
789
789
786.05
:
12/16/00
12/16/00
12/16/00
12/16/00
12/23/00
12/23/00
12/23/00
3
1
1
3
:
787.02
782.1
789
786.09
789.09
789.09
782.1
• Take today’s cases
:
:
:
12/23/00
786.09
786.09
• The cases one week ago 12/23/00
12/23/00
780.9
• The cases two weeks ago12/23/00 V40.9
2
4
1
3
1
1
4
3
3
2
7
F
F
M
M
:
M
F
M
M
M
F
M
:
M
M
F
M
place2
s-e
s-e
n-w
s-e
:
n-e
s-w
s-e
n-w
s-w
s-w
n-w
:
s-e
s-e
n-w
s-w
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
• Ask: “What’s different
about today?”
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 29
Simple WSARE
• Given 500 day’s worth
of ER cases at 15
hospitals…
• For each day…
DATE_ADMICD9
12/9/00
12/9/00
12/9/00
12/9/00
:
12/16/00
12/16/00
12/16/00
12/16/00
12/23/00
12/23/00
12/23/00
PRODROMGENDER
786.05
789
789
786.05
:
3
1
1
3
:
787.02
782.1
789
786.09
789.09
789.09
782.1
2
4
1
3
1
1
4
F
F
M
M
:
M
F
M
M
M
F
M
:
M
M
F
M
• Take today’s
cases
Fields
we use:: 12/23/00 : 786.09 :
3
786.09
3
• The cases one week ago 12/23/00
12/23/00
780.9
2
Date, Time of Day, Prodrome,
ICD9,
12/23/00 V40.9
7
•
The
cases
two
weeks
ago
Symptoms, Age, Gender, Coarse Location,
place2
s-e
s-e
n-w
s-e
:
n-e
s-w
s-e
n-w
s-w
s-w
n-w
:
s-e
s-e
n-w
s-w
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
Fine Location,
ICD9
Derived Features,
• Ask:
“What’s
different
Census Block Derived Features, Work
about
today?”
Details, Colocation Details
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 30
15
Example
Sat 12-23-2001 (daynum 36882, dayindex 239)
35.8% ( 48/134) of today's cases have 30 <= age < 40
17.0% ( 45/265) of
other cases have 30 <= age < 40
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 31
Example
Sat 12-23-2001 (daynum 36882, dayindex 239)
FISHER_PVALUE = 0.000051
35.8% ( 48/134) of today's cases have 30 <= age < 40
17.0% ( 45/265) of
other cases have 30 <= age < 40
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 32
16
Searching for the best score…
•
•
•
•
•
Try ICD9 = x for each value of x
Try Gender=M, Gender=F
Try CoarseRegion=NE, =NW, SE, SW..
Try FineRegion=AA,AB,AC, … DD (4x4 Grid)
Try Hospital=x, TimeofDay=x, Prodrome=X,
…
lert!
A
g
• [In future… features of census rblocks]
n
fitti
Ove
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 33
Example
Sat 12-23-2001 (daynum 36882, dayindex 239)
FISHER_PVALUE = 0.000051 RANDOMIZATION_PVALUE = 0.031
35.8% ( 48/134) of today's cases have 30 <= age < 40
17.0% ( 45/265) of
other cases have 30 <= age < 40
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 34
17
Multiple component rules
• We would like to be able to find rules like:
There are a surprisingly large number of children with
respiratory problems today
or
There are too many skin complaints among people from the
affluent neighborhoods
• These are things that would be missed by casual screening
• BUT
• The danger of overfitting could be much worse
• It’s very computationally demanding
• How can we be sure the entire rule is meaningful?
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 35
Checking two component rules
• Must
pass
both
tests to
be
allowed
to live.
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 36
18
WSARE v2.0
• Inputs:
1. Date/time-indexed
biosurveillancerelevant data stream
• Outputs: 1. Here are the
2. Time Window
Length
3. Which
attributes to use?
2. Here’s why
3. And here’s
how seriously you
should take it
records that most
surprise me
Primary Date
Key
Time
Hospital ICD9 Prodrome Gender Age Home
Work
Recent Recent (Many
Flu
Weather more…)
Large Medium Fine Large Medium Fine
Levels
Scale Scale
Scale Scale Scale
Scale
h6r32 6/2/2 14:12 Down- 781 Fever
town
M
20s NE
15217 A5
NW 15213 B8
2%
70R
…
t3q15 6/2/2 14:15 River- 717 Respirat M
side
ory
60s NE
15222 J3
NE
15222 J3
2%
70R
…
t5hh5 6/2/2 14:15 Smith- 622 Respirat F
field
ory
80s SE
15210 K9
SE
15210 K9
2%
70R
…
:
:
:
:
:
:
:
:
:
:
:
:
:
:
Copyright © 2002-2004, Andrew Moore
:
:
:
Biosurveillance: Slide 37
WSARE v2.0
• Input
s:
•Output
s:
1. Date/time-indexed
biosurveillancerelevant data stream
1. Here are the
records that most
surprise me
2. Time Window
Length
3. Which
attributes to use?
2. Here’s why
3. And here’s
how seriously you
should take it
Primary Date Time Hospita ICD Prodrom Gende Ag Home
Work
Recen Recent (Many
Normally,l 8% of9cases
in the
Key
e
r East
e Large Mediu Fine Large Mediu Fine t Flu Weathe more…
)
are over-50s with respiratory Scale m
Scale Scale m
Scale Levels r
Scale
Scale
781 Fever M
20 NE 15217 A5 NW 15213 B8 2% 70R …
h6r32 6/2/2 14:12 Downproblems.
town
t3q15 6/2/2 14:15
River- it’s
717been
Respira
M
But today
15%
side
tory
t5hh5 6/2/2 14:15 Smith- 622 Respira F
field
tory
:
:
:
:
:
:
:
Copyright © 2002-2004, Andrew Moore
s
60 NE
s
80 SE
s
: :
Don’t be too impressed!
15222 J3
NE
15222 J3
2%
70R
…
Taking into account all the patterns
I’veK9been
15210
SE searching
15210 K9 over,
2% there’s
70R a
…
20% chance I’d have found a rule
just
:
:this dramatic
:
:
: by: chance
:
:
Biosurveillance: Slide 38
19
WSARE on recent Utah Data
Saturday June 1st in Utah:
The most surprising thing about recent records is:
Normally:
0.8% of records (50/6205) have time before 2pm and prodrome = Hemorrhagic
But recently:
2.1% of records (19/907) have time before 2pm and prodrome = Hemorrhagic
Pvalue = 0.0484042
Which means that in a world where nothing changes we'd
expect to have a result this significant about once
every 20 times we ran the program
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 39
Which of the 500 days have
Irregularities?
• WARNING: Yet another overfitting
opportunity.
• If we took 500 days of randomly
generated data, then about 5 days
would have a p-value below 0.01
• This can be solved with…
• A Bonferroni correction
• The FDR (False Discovery Rate) method
[Benjamini & Hochberg, 1995, J. R. Stat
Soc, 57 289]
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 40
20
Results on
Emergency
Dept Data
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 41
Sanity check
• What happens if we generate a fake
database in which we know that there can
be no relation between date and case
features?
• This can be achieved by shuffling all the
dates in the database.
• The days detected by FDR are then…
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 42
21
Survived Randomized
• No days reported as
abnormal
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 43
Simulation-based validation
• We need to understand sensitivity and specificity.
• Have developed a very fast simulator in which people
go to work, go out for food, live in families.
• Background of mild diseases: some food-borne, some
occupational, some contagious etc.
• A nasty disease is introduced at some point.
• Can WSARE detect it?
• How does WSARE compare to a standard sensible
epidemiological alternative?
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 44
22
WSARE Simulation Sensitivity
vs Specificity on GRIDSIM
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 45
WSARE Simulation Sensitivity vs
Specificity on GRIDSIM
STANDARD
WSARE
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 46
23
WSARE 3.0
•
•
•
•
•
“Taking into account recent flu levels…”
“Taking into account that today is a public holday…”
“Taking into account that this is Spring…”
“Taking into account recent heatwave…”
“Taking into account that there’s a known natural
Food-borne outbreak in progress…”
Bonus: More
efficient use of
historical data
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 47
Signal
Conditioning on observed environment: Well
understood for Univariate Time Series
Time
Example Signals:
•
•
•
•
•
Copyright © 2002-2004, Andrew Moore
Number of ED visits today
Number of ED visits this hour
Number of Respiratory Cases Today
School absenteeism today
Nyquil Sales today
Biosurveillance: Slide 48
24
An easy case
Signal
Upper Safe Range
Mean
Time
Dealt with by Statistical Quality Control
Record the mean and standard deviation up
the the current time.
Signal an alarm if we go outside 3 sigmas
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 49
Signal
Conditioning on Seasonal Effects
Time
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 50
25
Signal
Seasonal Effects
Time
Fit a periodic function (e.g. sine wave) to previous
data. Predict today’s signal and 3-sigma
confidence intervals. Signal an alarm if we’re off.
Reduces False alarms from Natural outbreaks.
Different times of year deserve different thresholds.
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 51
Example [Tsui et. Al]
Weekly counts of P&I from week 1/98 to 48/00
From: “Value of ICD-9–Coded Chief
Complaints for Detection of
Epidemics”, Fu-Chiang Tsui, Michael
M. Wagner, Virginia Dato, ChungChou Ho Chang, AMIA 2000
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 52
26
Seasonal Effects with Long-Term Trend
Weekly counts of
IS from week
1/98 to 48/00.
From: “Value of ICD-9–Coded Chief
Complaints for Detection of
Epidemics”, Fu-Chiang Tsui, Michael
M. Wagner, Virginia Dato, ChungChou Ho Chang, AMIA 2000
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 53
Seasonal Effects with Long-Term Trend
Called the Serfling
Method [Serfling, 1963]
Weekly counts of
IS from week
1/98 to 48/00.
From: “Value of ICD-9–Coded Chief
Complaints for Detection of
Epidemics”, Fu-Chiang Tsui, Michael
M. Wagner, Virginia Dato, ChungChou Ho Chang, AMIA 2000
Copyright © 2002-2004, Andrew Moore
Fit a periodic function (e.g. sine wave) plus a linear
trend:
E[Signal] = a + bt + c sin(d + t/365)
Good if there’s a long term trend in the disease or
the population.
Biosurveillance: Slide 54
27
Day-of-week effects
Total Sales of all Available Products (HEALTH)
60000
12/30/1999
50000
s ale s
40000
12/23/1999
30000
20000
1/1/2000
1/1/2001
7/4/2000
10000
11/23/2000
11/25/1999
4/23/2000
99
9
20
2 / 00
8/
20
3 / 00
8/
20
4 / 00
8/
20
5 / 00
8/
20
6 / 00
8/
20
7 / 00
8/
20
8 / 00
8/
20
9 / 00
8/
20
10
0
/8 0
/2
1 1 00 0
/8
/2
1 2 00 0
/8
/2
0
1/ 00
8/
20
01
/1
1/
/8
12
8/
9
99
/1
/8
11
10
/8
/1
99
9
99
19
8/
9/
8/
8/
19
99
0
dates
From: “Using Grocery Sales Data for the
Detection of Bio-Terrorist Attacks”,
Goldenberg, Shmueli and Caruana,
2002
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 55
Day-of-week effects
Total Sales of all Available Products (HEALTH)
60000
12/30/1999
50000
s ale s
40000
12/23/1999
30000
20000
1/1/2000
10000
1/1/2001
Another simple
form of ANOVA
7/4/2000
11/23/2000
/1
8/
99
9
20
2 / 00
8/
20
3 / 00
8/
20
4 / 00
8/
20
5 / 00
8/
20
6 / 00
8/
20
7 / 00
8/
20
8 / 00
8/
20
9 / 00
8/
20
10
0
/8 0
/2
1 1 00 0
/8
/2
1 2 00 0
/8
/2
0
1/ 00
8/
20
01
4/23/2000
1/
9
12
/8
/1
99
99
/1
/8
/8
10
11
19
99
99
19
8/
8/
9/
8/
9
11/25/1999
0
dates
Fit a day-of-week component
E[Signal] = a + deltaday
From: “Using Grocery Sales Data for the
Detection of Bio-Terrorist Attacks”,
Goldenberg, Shmueli and Caruana,
2002
Copyright © 2002-2004, Andrew Moore
E.G: deltamon= +5.42, deltatue= +2.20, deltawed=
+3.33, deltathu= +3.10, deltafri= +4.02, deltasat= 12.2, deltasun= -23.42
Biosurveillance: Slide 56
28
Analysis of variance
• Good news:
If you’re tracking a daily aggregate (e.g. number
of flu cases in your ED, or Nyquil Sales)…then
ANOVA can take care of many of these effects.
• But…
What if you’re tracking a whole joint distribution
of transactional events?
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 57
Idea: Bayesian Networks
“Patients from West Park Hospital
are less likely to be young”
“On Cold Tuesday Mornings the
folks coming in from the North
part of the city are more likely to
have respiratory problems”
“The Viral prodrome is more
likely to co-occur with a Rash
prodrome than Botulinic”
Copyright © 2002-2004, Andrew Moore
“On the day after a major
holiday, expect a boost in the
morning followed by a lull in
the afternoon”
Biosurveillance: Slide 58
29
WSARE 3.0
All historical
data
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 59
WSARE 3.0
All historical
data
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 60
30
WSARE 3.0
All historical
data
Today’s
Environment
What should
be happening
today?
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 61
WSARE 3.0
All historical
data
Today’s
Environment
What should
be happening
today?
Copyright © 2002-2004, Andrew Moore
Today’s
Cases
What’s strange
about today,
considering its
environment?
Biosurveillance: Slide 62
31
WSARE 3.0
All historical
data
Today’s
Environment
Today’s
Cases
What should
be happening
today?
What’s strange
about today,
considering its
environment?
And how big a deal is
this, considering how
much search
I’ve done?
Biosurveillance:
Slide 63
Copyright © 2002-2004, Andrew Moore
WSARE 3.0
All historical
data
Today’s
Environment
Today’s
Cases
Cheap
What should
be happening
today?
Expensive
Copyright © 2002-2004, Andrew Moore
What’s strange
about today,
considering its
environment?
And how big a deal is
this, considering how
much search
I’ve done?
Biosurveillance:
Slide 64
32
WSARE 3.0
All historical
data
Today’s
Environment
• All-dimensions
Trees
Today’s
Cases
• Racing
Randomization
• Differential
Randomization
Cheap
• RADSEARCH
Expensive
Copyright © 2002-2004, Andrew Moore
What should
be happening
today?
What’s strange
about today,
considering its
environment?
And how big a deal is
this, considering how
much search
I’ve done?
Biosurveillance:
Slide 65
Results on Simulation
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 66
33
Standard
WSARE2.0
WSARE2.5
WSARE3.0
Results on Simulation
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 67
Conclusion
• One approach to biosurveillance: one algorithm
monitoring millions of signals derived from
multivariate data
instead of
Hundreds of univariate detectors
• Modeling historical data with Bayesian Networks to
allow conditioning on unique features of today
• Computationally intense unless we’re tricksy!
WSARE 2.0 Deployed during the past year
WSARE 3.0 about to go online
WSARE now being extended to additionally exploit
over the counter medicine sales
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 68
34
Conclusion
• Searching over thousands of contingency tables on a
large database...
...only
we have to
to do
it 10,000 times on
thealgorithm
replicas
•• One
approach
biosurveillance:
one
during randomization
monitoring
millions of signals derived from
• multivariate
...we also need
to learn Bayes Nets from databases with
data
millions of records...
instead
of
• ...and
keep relearning
them as data arrives online...
detectors
• Hundreds
...in the endofweunivariate
typically search
about a billion alternative
Bayes net structures for modeling 800,000 records in 10
minutes
• Modeling historical data with Bayesian Networks to
allow conditioning on unique features of today
• Computationally intense unless we’re tricksy!
WSARE 2.0 Deployed during the past year
WSARE 3.0 about to go online
WSARE now being extended to additionally exploit
over the counter medicine sales
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 69
Conclusion
• One approach to biosurveillance: one algorithm
monitoring millions of signals derived from
multivariate data
instead of
Hundreds of univariate detectors
• Modeling historical data with Bayesian Networks to
allow conditioning on unique features of today
• Computationally intense unless we’re tricksy!
• WSARE 2.0 Deployed during the past year
• WSARE 3.0 about to go online
• WSARE now being extended to additionally exploit
over the counter medicine sales
Copyright © 2002-2004, Andrew Moore
Biosurveillance: Slide 70
35