SUPPLEMENTARY MATERIALS AND METHODS SECTION
The institutional review board approved the study protocol. A waiver of informed
patient consent was granted based on the minimal risk to patients. Verbal consent
was obtained from physician study participants.
Study Design
Designed as a prospective observational study, the observation period was between
October 2008 and May 2010 in 3 ICUs at Mayo Clinic, Rochester, MN, an
academic tertiary-referral health care center, equipped since 2005 with a
comprehensive EMR. The study units consisted of a 24-bed general medical ICU
(“Medical”), a 20-bed surgical ICU (“Surgical) and a 20-bed medical-surgical ICU
(“Mixed”). The staff and patient mix are described in the results section. The
electronic resources available are briefly described below.
The standard EMR utilized in our clinical ICU practice complies with the definition of a
comprehensive EMR and has at its core the Mayo Integrated Clinical System (MICS)
LastWord, (unless otherwise stated all EMR components have been developed at Mayo
Clinic, Rochester, MN). This interface provides access to demographic and admission
information, laboratory data, medications record, nursing flow sheets, clinical notes and
procedure reports.. These data are displayed either in a free-text or table format. An
additional electronic resource is the Chart+ application which is used in all operating and
recovery rooms, ICUs and many procedural areas throughout the hospitals to capture and
display physiologic monitor data as well as allowing users to manually enter specific
data, such as medication and fluid administration. An electronic high-fidelity viewer,
QReads, is utilized for the display of radiologic and other medical images. The final
component of the standard EMR environment is the computerized physician order entry
system (CPOE) that incorporates comprehensive order sets, drug and allergy alerts as
well as rudimentary decision support. Access to all components of the EMR except
CPOE is achieved through Synthesis, a Windows-based custom user interface. The
CPOE system is accessed through the MICS Last Word System (Centricity, General
Electric Company, Fairfield, Connecticut, US). All ICUs are equipped with multiple
computer workstations, with password-protected EMR and clinical database access.
The ICU “Data Mart” is a Microsoft SQL relational data warehouse that provides direct
access to EMR data of critically ill patients across all ICUs in Mayo Clinic Rochester
hospitals. The copy of the EMR data is available in near real-time with a delay window
of 15 minutes for monitoring data and four hours for clinical notes. Technology
standards, architectures and polices are based on a strategic direction established by the
Mayo Clinic information technology department. The ICU Data Mart is updated from
original sources using technically different approaches, including direct SQL transactions
(“push”), store procedures, web-services and the capture of Health Level 7 (HL7)
messages from CPOE. For calculation of the quantity of data available within the EMR,
data stored within the DataMart were accessed with JMP statisticalsoftware(v 9.0, SAS
Institute Inc., Cary, NC), to query data tables throughOpenDatabaseConnectivity
(ODBC). Returned data werestoredas JMP data tables andanalyzedfurther.
Study Patients
Any admission episode (24-7) in any of the 3 study ICUs was eligible for inclusion. An
admission episode was excluded if the patient was less than 18 years old or was admitted
for low risk monitoring (as defined by the Simplified Therapeutic Intervention Scoring
System) [14] or had previously indicated that they did not wish to have their medical
record reviewed for research purposes.
Study Subjects
All critical care service physician staff – both attendings and trainees – who participated
in the care of a newly admitted patient,were eligible to be subjects for the study of
interactions relating to that patient. After verbal consent, physician subjects were
approached once per patient admission and no more than twice per shift. During the
study period they could complete the survey more than once. To facilitate the
administration of surveys in the live clinical environment, study subjects were not
allocated a unique study number. Subject characteristics, including base specialty and
grade (attending or trainee physician) were collected.
During the study period, the ICUs were staffed as follows: The critical care team
primarily managed all patients, in conjunction with surgical or consult services as
appropriate. Admitting teams consisted of an attending physician, a senior (fellow or
senior resident) and a junior resident. In the medical ICU, both a fellow and senior
resident were part of the admitting team. All physicians provided 24-7 in-house coverage
of the ICU, except for the mixed ICU in which the attending physician was not in-house
for 4 nights out of 7, but was immediately available to answer questions and to come to
the hospital as appropriate. Attending physicians were all U.S. board certified (or
international equivalent) in anesthesiology or pulmonary medicine and all had
subspecialty qualifications in critical care. Fellows were all in approved Accreditation
Council for Graduate Medical Education (ACGME) critical care training positions in
internal medicine, pulmonary medicine or anesthesiology. Residents came from
institutional internal medicine, anesthesiology, surgery or emergency medicine training
programs and rotated through the ICUs for periods of 1-3 months at a time.
Data Collection
Instruments
An expert panel [BP, OG, MK] generated a list of candidateclinical information concepts
which were considered by them to be useful in the diagnosis and treatment of patients
admitted to the study ICUs. Once identified, this list was distributed for comment to the
ICU attending physicians practicing within the institution. Using a combination of Likert
scale rankings and Delphi process a list of 33 high-value concepts were agreed upon after
review by the investigators [BP, OG, MK]. The final study instrument, a paper-based
questionnaire, had separate sections for these pre-identified concepts as well as free text
areas where the study subject (physician) could record additional concepts not present in
the pre-identified data section but which they had used in the assessment, diagnosis and
treatment of the admitted patient (Figure 1). Those additional, self-reported concepts
were captured as broad categories, (history, examination, investigation or other), and later
mapped to an agreed ontology [BP, AA, JA].
Procedures
Once an admission episode was identified for inclusion by a study coordinator, all
physician members of the admitting team were approached within 3 hours of the
admission time and were requested to complete the study questionnaire. The
questionnaire was delivered with a standard script and approach. Study coordinators then
collected the study forms, or recorded if the subject was unavailable, declined or did not
complete the form for some other reason. Recognizing the busy nature of the ICU
environment and the task load associated with the completion of each questionnaire, it
was decided to approach each physician no more than twice per shift, irrespective of the
number of admissions during that shift. Study coordinators were instructed not to
approach subjects if they appeared to be actively engaged in clinical care or had indicated
that they did not want to participate in the study. The timing of that approach was
determined by the study coordinator.
The EMRs of admitted patients on whom the study subjects responded were
retrospectively accessed through the ICU Data Mart described above. For each patient,
the total amount of clinical data accumulated between ICU admission and 3 hours after
admission was calculated for each of the following categories: vital signs, microbiology,
medications, laboratory results, fluids, nursing flow sheet items, and clinical notes.To
calculate the total clinical data in the ICU DataMart, only the incidence of data entry in
the database was considered towards the total and no attempt was made to weight the
quantity of data contained within each DataMart cell. Thus an entry associated with a
single systolic blood pressure was counted in the same way as an entry associated with a
clinical note. All vital sign data were stored at 15 minute intervals, all flowsheet data at 1
hour intervals and all other data were stored whenever new data became available. Each
data entry was associated with an identifying label, (e.g. Heart Rate), in the EMR. The
total quantity of data entries accumulated within the EMR was calculated for each
identifying label (see supplementary Table A for additional data).
In order to compare clinical information use to EMR data availability, all EMR data
labels were reviewed and mapped to appropriate subject-reported clinical concepts [BP,
VH].For each concept the quantity of DataMart data entriesand number of EMR data
labels were calculated. For example, the clinical information concept, “Blood Pressure”,
mapped to the followingEMR labels; Noninvasive blood pressure (NIBP) – Mean; NIBP
– Diastolic; NIBP-Systolic; Arterial blood pressure (ABP) – mean; ABP – Diastolic;
ABP- Systolic; ABP #3 – Mean;ABP #3 – Diastolic; ABP #3 – Systolic. This
represented a total of 22,544 data entries. Thus the subject-reported clinical information
concept “blood pressure” was represented in the EMR by 9 unique labels and accounted
for a total of 22,544 data points in the 408 study patients
Outcomes
The primary outcome measure was subject-reported frequency of utilization of preidentified and additional clinical information concepts. The secondary outcome measure
was the quantity of EMR data points and the number of unique EMR data labels which
mapped to each subject-reported clinical information concept.
Descriptive statistics are reported as median (interquartile range, IQR) for non-normally
distributed data. Comparisons were between groups containing categorical data only and
utilized chi square testing. All statistical procedures were performed in JMP (v 9.0, SAS
Institute Inc.)