Clinical Research Informatics
in Pediatric Critical Care
J. MICHAEL DEAN, MD, MBA
KATHERINE SWARD, PHD, RN
Context
http://cpccrn.org/
http://www.pecarn.org/
Critically ill and injured children typically receive care in the ED and/or the
pediatric intensive care unit (PICU). A spectrum of heterogeneous conditions
lead to need for “intensive care” : traumatic brain injury, other traumas, lung
injury, sepsis, postoperative care
Epstein, D. & Brill, J.E. (2005). A history of pedicatric critical care medicine. Pediatric Research, 58, 987–996;
doi:10.1203/01.PDR.0000182822.16263.3D
Unfortunately, much of the technology and many therapies in pediatric critical
care have evolved without adequate study or have been adopted uncritically
from adult, neonatal, or anesthetic practice…
NIH/NICHD has a new branch (Pediatric Trauma and Critical Illness),
recognizing the need for research in this environment.
Rigorous use of appropriate scientific methodology, deployed across a network
structure, achieves the numbers of patients required to provide answers...
http://www.nichd.nih.gov/research/supported/Pages/cpccrn.aspx
Context
Clinical Research Informatics involves the use of informatics tools and
methods in the discovery and management of new knowledge relating
to health and disease.
It includes management of information related to clinical trials and …
secondary research use of clinical data.
http://www.amia.org/applications-informatics/clinical-research-informatics
Context
Intensive Care
Informatics – a
relatively new
working group in
AMIA
Data Coordinating Center
U of Utah Division of Pediatric Critical Care encompasses (but is not limited
to): clinical services, Intermountain Injury Control Research Center, and a
data coordinating center that supports a large number of research networks
and clinical studies.
http://medicine.utah.edu/pediatrics/critical_care/
Informatics tools and methods are threaded throughout DCC activities
 study conception & protocol development LaTex, GitHub data models
 data collection standard CRFs, TrialDB OpenClinica RedCap CheckBox
 custom software development
 data quality QueryManager
 data analysis and publication
 network communications and logistics eRoom, teleconferencing
 regulatory/compliance monitoring; IT infrastructure (FISMA; HIPAA …)
Two examples
PECARN: registry project (Dean)
Pediatric emergency care applied research network.
CPCCRN: CDS tools (Sward)
Collaborative pediatric critical care research network
PECARN
•TBI prediction CDS
•Neuroimaging decision rule
•Decision rule for intraabdominal injuries
•Pediatric Emergency Care
Quality *
http://www.pecarn.org/pecarnNetwork/
documents/BrochureFall2013.pdf
PECARN Registry Protocol Objectives
1. Develop the registry by merging EHR data from
participating EDs.
2. Use the registry to collect stakeholder-prioritized
performance improvement measures
3. Report performance improvement measures to
individual clinics and to sites and measure subsequent
changes in quality performance.
PECARN Registry Protocol
Study Procedures
Database Construction
Deidentification Procedures
Natural Language Processing Procedures
Determining Benchmarks for Report Card
Report Card Feedback
PECARN Registry Protocol
Database Construction
Identify potential sources of relevant data elements in the specific EHR at each
site.
Finalize the types of data elements that will be extracted.
Extract data for one day of data at each clinical site.
Transmit one day data to the DCC for de-identification.
Establish de-identification procedure at each clinical site.
Extract and de-identify one month data from calendar year (CY) 2012 at each site.
Transmit one month de-identified data to DCC from each site.
Finalize and test import procedures from one month extracts into Registry.
Analyze frequencies of missing, out of range, or unexpected values for key data
elements.
Extract, de-identify, and transmit entire CY 2012 from each site to the DCC.
Create Registry with entire CY 2012 from all participating sites.
The process
WHAT IT REALLY LOOKS LIKE
PECARN Registry ETL
PECARN Registry ETL
Report Card Development:
Expert Panel
Panel met on 2/24/14 to review data for pre-selected Performance
Measures
Develop “ideal” benchmarks
ABC calculated by stats, presented in summary & full documents
◦ New methodology, difficult to understand
◦ Works well for dichotomous; continuous causes confusion
Definitions of Performance Measures ever changing
Huge stats efforts
Report Card:
Designing
Formatting the Report Card, Simplify
Finalize Performance Measure definitions (yes. Still changing…)
~17 Performance Measures, reported with 4-5 benchmarks, want
visuals & simple reports
◦ What will we loose
Comparison with past data, visual, quick
Redesign data warehouse to improve performance
Examples of Report Card draft today
Differences between
development & production
Automation (eventually)
Only data needed for report cards is kept ‘active’
◦ 4 months of data kept in active database for trendlines
◦ Older data is archived
Data is locked & not allowed to update after a cut-off
◦ Includes grouped & manually derived data
Data becomes more static – no resubmissions past deadlines!
Next Steps
Data collection
◦ “Real-time” monthly submission of 2014 data
Test the whole production cycle
◦ What happens when we really do this? How does it look?
Report Card delivery
◦ Automation is critical, several IT methods/approaches – will develop over
time
◦ Start small – email a report card
Measuring provider improvement once they get a Report Card
◦ Staggered starts
Collect Report Card feedback from providers
◦ Implement into the Report Card – improve!
Future directions?
Conversations about data for “this study” (Report Cards) vs “registry as
a whole” (future uses, add-on projects)
Adding more sites? Roll-up?
Adding more data?
Using the data for clinical trials?
CDS tools
CDS tools
• ICU
is information-dense environment
(information overload is likely)
• Many interventions in PICU were adopted from
adult practice, neonatal practice, anesthesiology, or
other areas – lack evidence from pediatric
environment
• For many critical care conditions, the
“intervention” is not single point in time, but is the
cumulative effect of multiple decisions and actions
across days
Critical Care Medicine, 2008
Even for a relatively simple
protocol like glucose/insulin
A single “decision” in reality
requires multiple steps that
are conducted in sequence.
Challenge – production rules
systems like Drools are designed
to run every rule at the same time.
Each node
is a set of
rules
Replicating eProtocol-insulin
allowed us to validate the
Java/Drools approach – could we
generate the same recommendations
in both versions of the software.
A subsequent project (Hypertonic Saline)
allowed us to examine issues related to
timing – labs not synchronous w
clinical data entry
Next project : ventilator management for ARDS/ALI
ARDS in children
•Estimates of ARDS in children range from 1.4 – 2.8% of all PICU
admissions
•Estimated: 3-4 ARDS cases per year per 100,000 population < 15
years of age
•The one prospective US publication suggested a rate triple this at 9.5
per year per 100,000 admissions
•Therefore, likely there are 1800 – 5700 cases per year in US
pediatric population < 15 years of age (2011 US Census = 60*106)
•Hence, ARDS remains a significant Public Health issue as overall
there is a significant (~18%) rate of mortality
•ALI is less severe form…even more common?
Adult vs Pediatric ICU
Developmental differences and differences in
clinical practices may contribute to the selective
application of evidence derived in other settings.
Ventilator modes such as high-frequency oscillatory
ventilation (HFOV) are more common in the
pediatric setting than in adults, for example; while
invasive arterial monitoring is increasingly less
common in the PICU.
Khemani, R & Newth, CJL (2010). The design of future pediatric mechanical ventilation trials for acute lung injury.
Am J Respir Crit Care Med 182, 1465–1474
Adult vs Pediatric ICU
Inspired oxygen fraction changes
◦ Size of FiO2 change (0.1 vs. 0.05)
◦ SpO2 ranges: <88, 88-93, >93%;
PaO2 ranges: <55, 56-68, >68 Torr
pH ranges
◦ Adult: >7.45, 7.30 – 7.45, 7.15 – 7.29, <7.15
◦ Pediatric: >7.45, 7.34 – 7.45, 7.25 – 7.34, 7.15 – 7.24, <7.15
Body weight for calculating tidal volume
◦ Adult: predicted BW (obesity, BW calculated from height & gender)
◦ Pediatric: actual BW (obesity & FTT, contractures; now formula for height from ulnar length )
Tidal Volume (VT exhaled) measurement
◦ Adult: measured at ventilator – use SET volume
◦ Pediatrics: should be measured at ETT
Mode of Ventilation
◦ ARDSNet – volume controlled
◦ Pediatric – Pressure control - PC or PRVC (volume targeted)
◦ Evolution in thinking re HFOV mode
High volume, High cost, High risk
Mechanical ventilation
Frequently used intervention in ICU
◦ Care of patients on mechanical ventilator was a motivating factor in the development of
ICUs (Watson, R.S. & Hartman, M.E. (2009). Epidemiology of Critical Illness. In D.S. Wheeler et al. (eds). Science and Practice of Pediatric
Critical Care Medicine. London: Springer-Verlag.)
◦ Primary treatment for respiratory failure (ARDS/ALI).
◦ Also a common intervention for other conditions.
◦ 20-64% (mean 30%) of PICU children require mechanical ventilation for some portion of
their stay
◦ A common outcome measure in pediatric trials (vent free days, days on ventilator, etc.)
Labor intensive, accounts for disproportionate amount of resource usage and costs
◦ 12% of overall hospital costs (Wunsch, H., Linde-Zwirble, W.T., Angus, D.C. et al. (2010). The epidemiology of mechanical ventilation
use in the United States. Crit Care Med, 38 (10), 1947-53)
Although life saving, mechanical ventilation has inherent risks (Bezzant & Mortensen,
1994; Newth et al., 2014)
◦ Oxygen toxicity
◦ Barotrauma, pneumothorax, damage to lung tissue from excessive pressure, volume, and
flow
◦ Complications from intubation (tracheal damage)
◦ Ventilator associated pneumonia
◦ Dangers from drugs, stress; nutritional problems
◦ Discomfort, pain, distress
Need for MV protocols
Heterogeneous patient characteristics
Many possible causes of ALI/ARDS
Best/optimal practices are not well
understood
MV protocols
Reported benefits of MV protocols in adult ICUs include decreased duration and costs of
mechanical ventilation and improved collaboration between health care team members
Variable results in peds
Schultz et al (2001) showed reduced time to
extubation
Randolph et al (2002) showed no decrease in weaning
time.
◦ two complex paper-based protocol arms with poor
compliance in each arm
Both of these studies were limited to the weaning
phase alone.
Mechanical Ventilation Course
Stable
Acute phase
Weaning
MV course
Stabilization
“Routine
management”
Weaning
Extubation
Intubation
End time
Definitions
NIV
criteria
Intubation
criteria
Stabilization
criteria
Weaning
Readiness
test
Extubation
Readiness
test
Extubation
criteria
End NIV
criteria
Weaning
extubation
failure
Variable results in peds
Variability occurs throughout the entire length
of ventilator management, not just the weaning
phase.
◦ Restepro et al (2004) found reduced time to spontaneous
breathing but no difference in overall ventilator duration.
◦ That study used a paper protocol to manage the overall
course of ventilation, but the authors noted as a limitation
their inability to determine compliance with the protocol.
MV research in peds
Willson et al. used a paper protocol outlining a broadly
defined lung protective strategy.
Curley et al. used the adult ARDSNet ventilation paper
protocol.
◦ Neither manuscript addressed protocol compliance.
◦ Neither protocol was explicit
Jouvet et al.17 used a closed loop protocol for mechanical
ventilation, but provided no details regarding its derivation
from an adult protocol .
Intensive Care Medicine, 2009
Providers say they adopted ARDSnet lung protective ventilation – but we saw high variability in practices
(single center)
MV in CPCCRN
• Early CPCCRN ideas about trials of different modes of
ventilation…
• Led to discussions of outcomes, paper on weaning and
extubation, recognition that reducing practice variation can
improve signal-to-noise (both for studies of MV; and for
studies in which MV is a surrogate outcome.)
•This led to thinking about how MV decisions are made
in the PICU … which led to our R21 grant
MV Protocol - Adult
ARDSnet studies – most sites used a paper based protocol
Intermountain: Tom East, Alan Morris, and colleagues developed an explicit,
computer-based protocol for mechanical ventilation, for care of adult ICU
patients with ALI/ARDS
MV purposes (simplified)
1.
Increase oxygen level
2.
Reduce CO2 (reflected
in the pH)
Protocol has rule sets for
each (oxygenation and
“ventilation”)
Different sets for different
MODE and patient state
Translation for Pediatrics
CPCCRN investigators recommended changes to adult protocol that they
believed were necessary for practitioners at their site to accept the
protocol recommendations.
Most of the changes were a matter of granularity (size)
* We also planned to update infrastructure from VB to Java/Drools
eProtocol MV – Peds
KE challenges
Complexity
o multiple modes (VC, PRVC, PC, HFOV, extubated)
o rules evaluating timing of ABG
◦ Non-invasive (O2 sat) vs invasive (pO2) measures
Large rule set
o 2824 “main” rules, plus others
o DROOLS file 56,410 lines
Little evidence of “best” or optimal practices
◦ e.g., increasing PEEP vs increasing FiO2 for low oxygenation
R21
1. Examine usual care ventilator management practices
◦ compared to what the protocol would have recommended
◦ Premise – when usual care and protocol are similar, the rule is probably going to
be seen by clinicians as acceptable
◦ When different – either rule needs to be examined, OR this is an opportunity to
improve care
◦ Prospective observational study – 8 hospitals
2. Examine issues of granularity – larger versus smaller changes; is this
affected by large versus small child? And issues of potential acceptability of
computer protocol
• Survey with 50 fabricated scenarios. ICU attending MD and fellows.
• Included attitude questions from UTAUT
• Chose survey approach to focus on content (rules/recommendations), rather
than delivery method (CDS)
Actual Changes Made
to PIP or VR (ventilation)
Δ
N
2449
%
No Δ
1564
63.9%
↑
363
14.8%
↓
561
23%
↑↓
39
1.6%
Stratified actual changes
by protocol bins…compared
what was done
to what protocol would have
recommended.
Single Institution
PC mode MV
N = 1484 ALI/ARDS
Heat map:
correspondence
between usual care
and protocol rules
We found WIDE variability in usual care practices within sites
FiO2
PEEP/FiO2 relationship
PEEP
PEEP/FIO2 Data – 8 CPCCRN PICUs
We found WIDE variability in usual care practices across sites
ALI
2012
120
Patients
3894
ventilator
changes
Set and forget?
Little change in mode
across course of care
Within a mode – the
most frequent decision
was to NOT CHANGE
any setting.
Aim 2
What are providers WILLING to do, at least in the context of
a research study?
Even though they might not make changes to settings in
usual care – are they willing to make changes if asked to do
so?
Are they willing to have the research protocol communicated
by means of a computer protocol (or are they resistant to
the idea of using computer protocol)?
UTAUT constructs
Performance expectancy
Effort expectancy
Attitudes toward technology
Social influence
Facilitating conditions
Anxiety
Self-efficacy
63.7% agree/strongly agree “using a
computer protocol for ventilator
management is a good idea”
and 68% agree that they have
the knowledge necessary to use a
computer protocol
But want help files or a person who can
assist with learning to use the protocol
Granularity, thresholds, modes
About evenly split (36.4%, 38.8%) on whether
evaluation for ventilator change should occur every
2 or every 4 hrs
48% ventilate to the weight on admission to the
PICU, 35.8% use predicted body weight
Wide range of responses as to what OI would
trigger change from conventional to HFOV mode
Aim 1 showed VC/AC mode was rare in our sites
but 71.9% say they use this mode in their PICU
Acceptability
Overall ~ 80% accepted
Higher acceptance for FiO2 instructions (> 95%)
Acceptance rate varied by mode (highest acceptance for HFOV
recommendations)
Recommendations “like protocol” were accepted at a higher rate than
recommendations “not like” protocol – except lower acceptance for PEEP
instructions
◦ Prefer to adjust FiO2 rather than PEEP to support oxygenation?
◦ Clinicians seem uncomfortable with how high the protocol might push pressures
(PEEP and PIP)
Clinicians WILL decline “poor” recommendations (they stay vigilant)
eVentilator – Pediatric Version, May 2013
New CDS tool: Java/Drools
Architecture
Core
Domain specific extension to core
GUI, IE, knowledge base, pt data etc. are
separated
FDA IDE
FDA considers this type of software a “device”: software
that…accepts clinical findings … and generates
recommendations for treatment…
FDA may or may not choose to require an IDE
◦ If so this requires extensive
documentation and
several types of validation
FDA-IDE
Working with colleagues in a U of Utah basic
science lab for technical and safety evaluation of
the CDS tool (e.g., run the software over extended
time and watch for “crashes”), initial evaluations of
usability, initial estimates of impact on workflow…
(naïve users; willing to enter data hourly 24x7)
Future Directions
Infrastructure updates
◦ model driven architecture
◦ JFX – GUI authoring
Other delivery platforms for multi-site research
Web based?
Mobile?
Map the terminology; represent knowledge according to
standards – Health eDecisions format?
Alvin Feinstein, MD, 1977 (Yale)