Clinical Decision Support
Dr Jeremy Rogers MD MRCGP
Senior Clinical Fellow in Health Informatics
Northwest Institute of Bio-Health Informatics
Talk Outline
Why we need it
What does ‘decision support’
mean ?
Work so far
Why we don’t use it
Talk Outline
Why we need it
What does ‘decision support’
mean ?
Work so far
Why we don’t use it
Drivers for decision
support
► Growth of medical knowledge
► Approx 100 articles were published in 1966 from RCTs;
► Over 10,000 annually by 1995 (Chassin, 1998)
► ‘The scarcely tolerable burden of information that is
imposed taxes the memory but not the intellect’ (GMC 1993)
► Pressures to use knowledge
► Evidence based medicine
► National service frameworks
► Clinical Governance
► Cost – e.g. $5.5M in 37 Days
for one patient at Duke
► ‘Post genomic’ individualised medicine
Drivers for decision
support
► Public recognition of medical error
► IOM “To err is human” (2000)
& “Crossing the quality chasm” (2001)
► More people die from medical errors than from breast
cancer or AIDS or motor vehicle accidents
► Jessica Santillan case
17 year old who had a heart and lung
transplant from a donor with an incompatible
blood group in Feb 2003 at Duke, and died
after a re-do 13 days later
Committee on Quality of
Health Care in America
US Institute of Medicine : Quality Chasm Report, 2001
(The American) health care delivery system is in
need of fundamental change
The current care systems cannot do the job
Trying harder will not work
Changing systems of care will
Talk Outline
Why we need it
What does ‘decision support’
mean ?
Work so far
Why we don’t use it
Kinds of decision
Diagnosis
Intervention
Prognosis
Kinds of support
► Active vs Passive support
► Making specific suggestions – one off, or continuing ?
► Critiqueing recorded actions – screw-up detection
► Tweaking / filtering information display
► Intelligent image processing
► Reminders ? Alerts ?
► Decision support, or decision making ?
► Do we expect human to learn from device ?
Drowning in data
The case for DS in display filtering
EPR - Dr Kildare - 26th Oct 2000
John Doe
36 yrs
Engineer
Married, 2 children
Encounters
12.10.96 Coryza: chest NAD: reassure
13.10.96 URTI: wheezy: amoxycillin
20.10.96 Anxiety: child admitted to H: reassure
24.10.96 PEFR : 300 :
10.11.96 PEFR : 400: CXR requested
12.11.96 CXR Basal Consolidation: : erythromycin
27.11.96 : Chest clear :
07.03.97 Depression: death in family: paroxetine
19.04.97 Gastoenteritis: : reassure
01.06.97 : : rpt Rx paroxetine
18.10.97 Sick note : :
03.03.98 Viral URTI: PEFR 350: salbutamol
04.03.98 WCC NAD : :
30.06.98 PMR report : BP, ECG NAD :
15.09.98 Eczema : : hydrocortisone
05.11.98 Depression : : paroxetine
03.01.99 Fibrositis: trigger spot lwr back: ibuprofen
17.02.99 Allergic Asthma: PEFR 300: salbutamol
21.03.99 Chest Inf: L base: erythromycin
07.10.99 Med4: anxious :
26.01.00 Asthma Review: :Repeat Rx Salbutamol
Active Problems
Current Medication
Asthma
Salbutamol
Hydrocortisone
Letters
Results
Appt
This Visit
Code
PEFR
Asthma
C/o Low Mood
Notes
550 l /min
Chest NAD. No Problems.
Declined antidepressant
BP
Action
Salbutamol inh 2 puff qds 1op
Influvac im BN #035679A4
PEFR
WCC
Drowning in data
The case for DS in display filtering
EPR - Dr Kildare - 26th Oct 2000
John Doe
36 yrs
Engineer
Married, 2 children
Encounters
12.10.96 Coryza: chest NAD: reassure
13.10.96 URTI: wheezy: amoxycillin
20.10.96 Anxiety: child admitted to H: reassure
24.10.96 PEFR : 300 :
10.11.96 PEFR : 400: CXR requested
12.11.96 CXR Basal Consolidation: : erythromycin
27.11.96 : Chest clear :
07.03.97 Depression: death in family: paroxetine
19.04.97 Gastoenteritis: : reassure
01.06.97 : : rpt Rx paroxetine
18.10.97 Sick note : :
03.03.98 Viral URTI: PEFR 350: salbutamol
04.03.98 WCC NAD : :
30.06.98 PMR report : BP, ECG NAD :
15.09.98 Eczema : : hydrocortisone
05.11.98 Depression : : paroxetine
03.01.99 Fibrositis: trigger spot lwr back: ibuprofen
17.02.99 Allergic Asthma: PEFR 300: salbutamol
21.03.99 Chest Inf: L base: erythromycin
07.10.99 Med4: anxious :
26.01.00 Asthma Review: :Repeat Rx Salbutamol
Active Problems
Current Medication
Asthma
Salbutamol
Hydrocortisone
Letters
Results
Appt
This Visit
Code
PEFR
Asthma
C/o Low Mood
Notes
550 l /min
Chest NAD. No Problems.
Declined antidepressant
BP
Action
Salbutamol inh 2 puff qds 1op
Influvac im BN #035679A4
PEFR
WCC
Goal of support
► Influence outcome
► Good things more likely; bad things less likely
► Outcomes…
► Fatal events are only the tip of the iceberg
►Easiest to measure, and most dramatic, but….
► Non fatal events
►Side effects
►Sub-optimal treatment
►Inappropriate treatment
► Non harmful events
►Inefficiency & Confusion
►Inappropriate resource consumption
►Bed stay
►Repeated re-investigation
Kinds of DS technology
► Statistical
► ‘93.467% of the time, things that quack
and have webbed feet are ducks”
► Model-based
► ‘It’s definitely a duck because
you told me its mother was a duck’
► Neural Networks
► ‘Of all the things you’ve shown me so far,
it looks most like the ones you said were ducks.’
Talk Outline
Why we need it
What does ‘decision support’
mean ?
Work so far
Why we don’t use it
The Story so far…
“Three decades of research into
computer aids for medical decision making
have resulted in thousands of systems and a
growing number of successful clinical trials…”
BMJ 1997;315:891 (4 October)
Projects past and present
► Acute Abdominal Pain (1972)
► Mycin (1977)
► ProFORMA
► Internist/QMR (1980s)
► Protégé
► DXPlain
► PRODIGY (1997-)
► ILIAD
► Prescribing Indicators
► Sophie
► Isobel (2000)
► Medical Logic Modules ► NHS Direct (2000)
(Arden Syntax) (1989) ► Knowledge Coupling (PKC.com)
1970s & 1980s
Behold, the Oracle…
Mycin – Shortliffe 1970s,
Stanford
> (mycin)
What is the AEROBICITY of ORGANISM-1? aerobic
------ PATIENT-1 ------
Is Sylvia Fischer a compromised host? yes
Patient's name: Sylvia Fischer
Is Sylvia Fischer a burn patient? If so, mild or serious? why
Sex: female
It is known that:
Age: 27
1) THE SITE OF THE CULTURE IS BLOOD
------ CULTURE-1 ------
2) THE GRAM OF THE ORGANISM IS NEG
From what site was the specimen for CULTURE-1 taken? blood
3) THE MORPHOLOGY OF THE ORGANISM IS ROD
How many days ago was this culture (CULTURE-1) obtained? 3
Therefore,
------ ORGANISM-1 ------
Rule 52:
Enter the identity (genus) of ORGANISM-1: unknown
The gram stain of ORGANISM-1: ?
A GRAM must be of type (MEMBER ACID-FAST POS NEG)
The gram stain of ORGANISM-1: neg
If
1) THE BURN OF THE PATIENT IS SERIOUS
Then there is weakly suggestive evidence (0.4) that
1) THE IDENTITY OF THE ORGANISM IS PSEUDOMONAS
Is Sylvia Fischer a burn patient? If so, mild or serious? serious
Is ORGANISM-1 a rod or coccus (etc.): rod
Findings for ORGANISM-1:
What is the AEROBICITY of ORGANISM-1? why
IDENTITY: ENTEROBACTERIACEAE (0.800) PSEUDOMONAS (0.760)
It is known that:
1) THE GRAM OF THE ORGANISM IS NEG
Is there another ORGANISM? (Y or N) Y
2) THE MORPHOLOGY OF THE ORGANISM IS ROD
------ ORGANISM-2 ------
Therefore,
Enter the identity (genus) of ORGANISM-2: unknown
Rule 107:
The gram stain of ORGANISM-2: (neg .8 pos .2)
If
Is ORGANISM-2 a rod or coccus (etc.): rod
1) THE AEROBICITY OF THE ORGANISM IS AEROBIC
What is the AEROBICITY of ORGANISM-2? anaerobic
Then there is suggestive evidence (0.8) that
1) THE IDENTITY OF THE ORGANISM IS ENTEROBACTERIACEAE
Findings for ORGANISM-2:
IDENTITY: BACTEROIDES (0.720) PSEUDOMONAS (0.646)
Abdominal Pain:
De Dombal (1972)
A multicentre study of computer aided diagnosis for patients with
acute abdominal pain was performed in eight centres with over
250 participating doctors and 16,737 patients.
Performance in diagnosis and decision making was compared
over two periods: a test period (when a small computer system
was provided to aid diagnosis) and a baseline period (before the
system was installed). The two periods were well matched for
type of case and rate of accrual.
The system proved reliable and was used in 75.1% of possible
cases.
User reaction was broadly favourable.
Abdominal Pain:
De Dombal
During the test period improvements were noted in diagnosis,
decision making, and patient outcome.
Initial diagnostic accuracy rose from 45.6% to 65.3%. The
negative laparotomy rate fell by almost half, as did the perforation
rate among patients with appendicitis (from 23.7% to 11.5%). The
bad management error rate fell from 0.9% to 0.2%, and the
observed mortality fell by 22.0%.
The savings made were estimated as amounting to 278
laparotomies and 8,516 bed nights during the trial period-equivalent throughout the National Health Service to annual
savings in resources worth over 20m pounds and direct cost
savings of over 5m pounds. Computer aided diagnosis is a useful
system for improving diagnosis and encouraging better clinical
practice.
Br Med J (Clin Res Ed) 1986 Sep 27;293(6550):800-4
Medical Logic Modules
(Arden Syntax)
maintenance:
title: ;;
filename: template;;
version: 1.00;;
institution: ;;
author: ;;
specialist: ;;
date: 1993-01-01;;
validation: testing;;
library:
purpose: ;;
explanation: ;;
keywords: ;;
citations: ;;
knowledge: type: data-driven;;
data: ;;
evoke: ;;
logic: ;;
action: ;;
end:
An MLM…
maintenance:
title: Check for adequacy of therapeutic anticoagulation with warfarin;;
filename: warfarin_anticoagulation;;
version: 1.07;;
institution: Columbia-Presbyterian Medical Center;;
author: Randolph C. Barrows, Jr., MD (barrows@cucis.cis.columbia.edu);;
specialist: ;;
date: 1994-04-28;;
validation: testing;;
library:
purpose: To warn the health care provider that a patient maintained on
warfarin is NOT in a therapeutic range for low-intensity or full-intensity
anticoagulation. Low-intensity anticoagulation is defined as a prothrombin
INR in the range of 2.00 - 3.00 (roughly corresponding to a PT in the range
of 1.2-1.5 times control). Full-intensity anticoagulation is defined as an INR
in the rage of 3.00 - 4.50 (roughly corresponding to a PT in the rage of 1.5 2.0 times control).;;
explanation: ;;
keywords: ;;
citations: Scientific American Medicine;;
…and (some of) its logic
/* the INR-containing procedures */
storage_of_INR := EVENT {
'32506~service event', ‘2256~presbyterian coagulation profile';
'32506~service event', ‘2302~stat coagulation profile' };
/* See if patient has a warfarin order. Probably need to add 31058
Bishydroxycoumarin Preparations Here I only want header table info, no
components. Is it ok to say null components? */
(start_time, status, order_key, frequency):= READ LAST {
'dam'="PDQORD1", display_header'="TRSKF",'display_comp'=""; ;
'28612~CPMC Drug: Coumadin 10 Mg Tab',
'28613~CPMC Drug: Coumadin 2 Mg Tab',
'28614~CPMC Drug: Coumadin 2.5 Mg Tab',
'28615~CPMC Drug: Coumadin 5 Mg Tab',
'29932~CPMC Drug: Ud Coumadin 10 Mg Tab',
'29933~CPMC Drug: Ud Coumadin 2 Mg Tab',
'29934~CPMC Drug: Ud Coumadin 2.5 Mg Tab',
'29935~CPMC Drug: Ud Coumadin 5 Mg Tab',
'33033~CPMC Drug: Coumadin 7.5 Mg Tab' };
Knowledge Couplers:
PKC.com
Larry Weed MD
Some CPOE Success Stories
►
►
►
►
►
►
Barnes-Jewish Hospital, St. Louis, Missouri
130 potentially dangerous drug interactions identified
two-thirds of those involving the drug cisapride averted
Brigham and Women’s Hospital, Boston
81% decline in medical errors after implementation CPOE
64% of decline due to first, and simplest, version of the technology, which included
features such as predetermined lists of medications and doses, display of patient data,
basic drug dosage, interaction, and duplication checking.
Montefiore Medical Center, New York City
50% decrease in medication errors following CPOE
Time from placing an order to its arrival in pharmacy reduced to two hours.
Ohio State University Medical Center, Columbus, Ohio
Length of stay decreased by two days following CPOE
Pharmacy orders turnaround reduced by two hours
Pharmacy charges per admission reduced by $910
University Community Hospital, Tampa, Florida
77% reduction in all adverse drug events, and 85% in severe ADEs
Cost of drugs for one family reduced by more than $200,000 per year.
Children’s Hospital of Pittsburgh
50% reduction in harmful error
Virtual elimination of weight-related adverse drug events
Complete eradication of transcription/handwriting errors
50% reduction in medication delivery times.
Other successes…
► Strong evidence suggests that some CDSSs can improve physician
performance. Additional well-designed studies are needed to assess their
effects and cost-effectiveness, especially on patient outcomes
(Johnston 1994)
► Mothers receiving computer-generated reminders had 25% higher on-time
immunization rate for their infants
(Alemi, 1996)
► Decision support system was safe and effective and improved the quality of
initiation and control of warfarin treatment by trainee doctors
(BMJ 1997;314:1252)
► Computerized physician order-entry reduced adverse drug events by 55%
(Bates, 1998)
► 9% of redundant lab tests at a hospital could be eliminated using a
computerized system
(Bates, 1998)
► 74% of the studies of preventive healthcare reminder systems and 60% of
the evaluations of drug dosing models reported a positive impact
(Trowbridge & Weingarten, AHRQ, 2001)
..and some failures
► (PRODIGY) - No effect was found … on the management of
asthma or angina in adults in primary care
BMJ 2002; 325: 941-944
► ..decision support system did not confer any benefit in
absolute risk reduction or blood pressure control
BMJ 2000;320:686-690
► Computerised decision support systems have great potential
for primary care but have largely failed to live up to their
promise
BMJ 1999;319:1281
My own failure:
Prescribing Indicators
► General Practice Repeat Prescribing
► Patients get more drug without seeing doctor
►typically, enough for 1-3 months
► 35% of population at any one time on repeat Rx
► Medication Review
► Accepted part of good clinical practice
► Requirement in NSF for Older People
► But: signing authorities is daily batch process
►>30 scrips per GP per day
►No time for careful review
What is ‘Medication Review’ ?
Indicators of ‘quality’ prescribing
► Cantrill et al: 13 indicators:
► Dose too high or too low?
► Course too long ?
► Expensive or useless drug ?
► Interaction with another drug ?
► Contraindicated ?
► By brand ?
► REASON FOR USE DOCUMENTED ?
► Manual system: impractical
► Our project: (2000-2002)
► computerise the indicators
Complex implementation..
Patient ID:
Medication:
Problem List:
4578
DITA906 DISR10514B
183... (Oedema) 1B17..
(Depressed) G5732.
(Paroxysmal Atrial fibrillation)
G73z0. (Intermittent
claudication) H3.... (Chronic
obstructive pulm.dis.) 137S.. (Ex
smoker) 246... (O/E - blood
pressure reading) 442...
(Thyroid hormone tests) 44P...
(Serum cholesterol) 7L172.
(Blood withdrawal for testing)
Ontology ID
Product
Rubric
345031(oral dig)
DITA905
Digoxin 125 mcg tab
345031
DITA906
Digoxin 250 mcg tab
345031
DITA908
Digoxin 62.5 mcg tab
9099269
Systemic Digoxin
G57.. Cardiac dysrhythmias
G573. Atrial fibrillation and flutter
G5730 Atrial fibrillation
G5731 Atrial flutter
G5732 Paroxysmal atrial fibrillation
G573z Atrial fibrillation and flutter NOS
IDENT “9099269”
MAIN digoxin
PROPERTIES
HAS_DRUG_FEATURE physiological action
WHICH_IS process
ACTS_ON heart
Indication
Code
Rubric
Atrial fibrillation
14AN.
H/O atrial fibrillation
3272.
ECG: atrial fibrillation
3273.
ECG: atrial flutter
7936A
IV pacer control of A Fib
G573.
Atrial fibrillation / flutter
305084
Digoxin
Liquid
HAS_DRUG_FEATURE indication
FOR treating
ACTS_ON supraventricular arrhythmia
HAS_DRUG_FEATURE indication
FOR treating
ACTS_ON atrial fibrillation
HAS_DRUG_FEATURE information source
IS_PART_OF interaction appendix
329308
Digoxin elixir
345031
Oral
Digoxin
tablet
305075
Digoxin
injection
305093
Digoxin Paed inj
..and disappointing
results
► Machine says there is no recorded indication in
33% of prescribing events
► BUT high false positive rate: 62%
► => it is wrong, most of the time
► Why ?
Of all alerts where machine says
‘no indication’…
BNF Omits
5%
Idiosyncratic
record
27%
Human could
infer
3%
Mapping error
27%
No Record
38%
Problems with the
oracle
► Painful data acquisition
► Exhaustive
► Includes exhaustive negative findings
►(which clinicians traditionally largely omit)
► Slow to use
► Poor support for clinical workflow
► Clinician is passive
► Infrequent recognised need
1990s –
More modest aspirations
► Narrow Domain systems
► ECG interpretations
► Arterial blood gas interpretation
► Predicting drug-drug interaction
► Alerts and Reminders
► Out of range test flagging
► But plans for the oracle are resurfacing in
expectation of imminent EPR
Talk Outline
Why we need it
What does ‘decision support’
mean ?
Work so far
Why we don’t use it
You can lead a horse to
water…
“Three decades of research into computer
aids for medical decision making have
resulted in thousands of systems and a
growing number of successful clinical
trials…”
“Yet only a handful of applications are in
everyday use”
BMJ 1997;315:891 (4 October)
Decision Support Systems in Use Today (2003)
QMR
PUFF
HELP
Diagnostic decision-support system for internists
1972
routine use
Pulm onary function tests
1977
?
Know ledge-based HIS
1980
?routine use
ACORN
Coronary care adm ission
1987
decom m issioned
DXplain
Liporap
MDDB
Epileptologists' Assistant
Cancer, Me?
Hepaxpert I, II, III
Interpretation of acid-base disorders
Managed Second Surgical Opinion System
Colorado Medicaid Utilization Review System
Geriatric Discharge Planning System
Microbiology/ Pharmacy Expert System
PEIRS
NéoGanesh
POEMS
SETH
Jeremiah
Clinical Event Monitor
VIE-PNN
CEMS
GermAlert
Germwatcher
Orthoplanner
RaPiD
DoseChecker
Coulter® FACULTYT M
SahmAlert
Reportable Diseases
TxDENT
RetroGram
Automedon
ERA
Therapy Edge
ATHENA
Clinical decision support
1987
routine use
Dyslipoproteinaem ia phenotyping
1987
?routine use
Diagnosis of dysm orphic syndrom es
1988
?routine use
Nurse progress note assistant
1989
decom m issioned
Patient cancer advice
1989
?
Hepatitis serology
1989
routine use
acid-base disorders
1989
?routine use
Managed care
1989
?
Prescription quality review
1990
?
Patient discharge planning
1990
?
Drug sensitivity
1991
?routine use
Pathology reports
1991
decom m issioned
Ventilator m anager
1992
2001
Post-operative care
1992
?
Clinical toxicology
1992
?routine use
Orthodontic treatm ent planner
1992
?routine use
Clinical alerts
1992
?routine use
Neo-natal parentral nutrition
1993
?In use
Mental health decision support system
1993
routine use
Infection control
1993
?routine use
Infection control
1993
?routine use
Orthodontic treatm ent planner
1994
?routine use
Designs rem ovable partial dentures
1994
?routine use
Drug dose checker
1994
?routine use
Haem atology
1995
?routine use
Drug sensitivity
1995
?routine use
Infection control
1995
?routine use
Screeing dental patients
1997
?routine use
Decision support for drug regim ens for HIV-infected patients
1999
routine use
Ventilator m anager
2001
routine use
Web-enabled electronic decision support and referrals system for cancer
2001
Under evaluation
Web-enabled decision support system for the treatm ent of HIV
2001
routine use
DSS for the m anagem ent of hypertension in prim ary care
2002
routine use
http://www.openclinical.org/aisinpractice.html
Why ? – the domain
► Rigid criteria difficult to apply in chaotic settings
► Medical data doesn't fit quantised definitions
► Even complex decision support algorithms require simplified
and standardised inputs by users
► And descriptive data is very hard to quantise
► Rules are situation specific
► localising decisions to available resource is costly
► When are decisions actually made ?
► To be effective, system needs to be physically
available in situation where decision is made
Why ? - the technology
► Highly mobile workforce vs highly static computers
► Slow computers
► Crude knowledge bases  poor performance
► Lack of stats for bayesian approaches
► Crude KR technology for model-based
► Closed software architectures
► Can’t integrate 3rd party DS modules with EPR
Why ? – the law
► Medicolegal aspect of EPR
► Confidentiality & Consent
► HIPAA
► Medicolegal aspects of DS technology
► Responsibility for action rests with clinician
► Systems that are as effective as clinician overall no help if
behaviour includes obvious clinical howlers
► Burden of recording why did not follow DS advice
Why ? – the people
► Poor data quality
► Numerical data easy to obtain
► Much of medicine not numerical
► Inconsistent data entry
Data Quality
(Frequency of recording per GP per year)
READ CODE
Sore Throat Symptom
Visual Acuity
ECG General
Ovary/Broad Ligament Op
Specific Viral Infections
Alcohol Consumption
H/O Resp Disease
Full Blood Count
Practice A
0.6
0.4
2.2
7.8
1.4
0
0
0
Practice B
117
644
300
809
556
106
26
838
Why? – the people
► Poor data quality
► I know what I’m doing
► Numerical data easy to obtain
► Perception of infallibility
► Much of medicine not numerical
► 88% of the time users requested to
bypass PRODIGY
(Beaumont 1988)
► Inconsistent data entry
► What happened to my clinical ► Reluctance to change clinical
practice to fit the tool
autonomy ?
► Interface issues
BMJ 1999;318:1527-1531
► Weed’s knowledge couplers
► Users intolerant of less than
perfect performance
BMJ 2003;326:314
Why ? - money
Through more improved
choice of initial antibiotics to
treat pneumonia, a group of
mid-west hospitals
decreased complications,
mortality rates and hospital
days and costs…
Improved management of
diabetic patients through
frequent e-mail
communication can produce
better outcomes and fewer
visits…
…but hospital revenues also
decreased as patients
shifted from higher paying to
lower paying DRGs.
…but lower physician group
revenues under ‘fee for
service’ payment.
Summary
► Research and commercial products pre-date
IOM by almost 30 years
► Widespread adoption has not occurred even
where results were positive
► Significant hurdles remain
► Legal
► Technical - EPR is harder than it looks
► Human factors