THE EFFECT OF AN EMERGENCY MEDICINE STRUCTURED DATA COLLECTION
FORM IN AIDING IN ACUTE ABDOMINAL PAIN EXAMINTATION AND DIAGNOSIS
Student Team: Craig Donnelly, Kelsey LeBeau, Hannah McClellan, Matthew Thompson
Faculty Advisors: Stephanie Guerlain and Dr. Donald Brown
Department of Systems and Information Engineering
Client Advisors: Dr. Forrest Calland and Dr. Scott Syverud
University of Virginia Health System
KEYWORDS: Emergency Medicine, abdominal pain,
medical diagnostic aids.
Unfortunately, this technique does not yield
standardized data suitable for statistical data analysis.
Abdominal pain is a complaint that could benefit from
standardized methods because it is commonly seen in
emergency medicine, is often resource intensive and
ranges significantly in types of causes and severity of
conditions. This study examined two emergency
medicine research topics that medical researchers have
actively studied for several decades: the documentation
of medical records using a structured approach, and the
management of patients complaining of acute
abdominal pain.
ABSTRACT
Background: The process by which patients in
Emergency Departments in the United States are
evaluated is irregular and not standardized. Abdominal
pain is a common emergency medicine complaint with
a complicated and resource intensive diagnostic process
that would likely benefit from standardization.
Objective: To evaluate the effectiveness of a
structured patient data collection form in standardizing
and improving the quantity and quality of data collected
during patient encounters in the University of Virginia
Emergency Department involving patients with
abdominal pain.
Design: This study is a prospective assessment of a
structured patient data collection form on the quality of
patient documentation by emergency medicine
residents on all non-traumatic adult patients
complaining of abdominal pain during a four-week
period.
Results: The use of the structured data collection
form resulted in significantly higher average cumulative
scores for subjective and objective data collection, but
with low compliance.
Conclusions: Standardized data forms can be
effective in increasing the quality and quantity of data
collected from patients presenting to an academic
tertiary care center with abdominal pain, but are likely
to be of limited overall utility unless designed and
implemented with great care.
BACKGROUND INFORMATION
According to a 1993 study at the University Hospital,
abdominal pain is the chief complaint in 4-5% of all
emergency department patients. 1 Although many
causes of abdominal pain are neither surgical issues nor
life-threatening conditions, there are certainly those that
demand immediate medical or surgical attention in
order to avoid traumatic or even fatal outcomes.
Unfortunately, differentiating between the multiple
causes of abdominal pain is one of the greatest
challenges emergency room physicians face at present.
One study reported that 41% of abdominal pain patients
are diagnosed with undifferentiated abdominal pain
because physicians are unable to identify a specific
diagnosis. 2 Moreover, the same paper reported that
patients who received a specific diagnosis during the
emergency room visit were misdiagnosed 55% of the
time. 2 A 1976 study at the University Hospital
reviewed 1,000 abdominal pain cases and found twenty
instances of patients who underwent unnecessary
surgical procedures. 3 In addition, there were eleven
cases in which physicians did not initially recognize a
surgical condition and thus erroneously released a
patient. 3 The poor performance of the abdominal pain
diagnostic process is a result of both the inexperienced
resident physicians performing the initial examination,
as well as the unquestionable difficulty of accurately
INTRODUCTION
Emergency medicine physicians work in a high stress
environment where they rely on experience to diagnose
patients. In many emergency departments, physicians
use a narrative method of recording a patient’s history,
physical examination and diagnostic plan.
1
information for physicians. Such a “snap shot” of
abdominal pain patient demographics guide physicians
in establishing adequate sample sizes and parameters
for clinical research experiments. The last formal effort
of this sort at the University Hospital occurred in 1993
when Powers and Guertler reviewed 1,000 abdominal
pain patient records from the emergency room to
“assess…evaluation and epidemiology of abdominal
pain”. 1 Nearly a decade has passed since this study
took place. The numerous advancements in medical
technology and inevitable fluctuation of patient
community attributes certainly warrant a reexamination
and updating of the results from the 1993 study.
As this literature review illustrates, improving the
abdominal pain diagnostic process is a widespread topic
in today’s medical research community. An effective
investigation of abdominal pain cases requires detailed
accounts of physicians’ observations and decisions
during patient examinations.
diagnosing patients with abdominal pain. In the
resident physician’s defense, studies show that even
master surgeons struggle to pinpoint the exact cause of
an abdominal pain patient. 4 Over the past few decades,
these circumstances have prompted numerous studies to
better understand abdominal pain cases and develop a
successful diagnostic strategy.
In light of the exceptional capabilities of modern
technology, it is not surprising that many of the efforts
to improve the abdominal pain diagnostic process are
computer-based solutions. Nonetheless, numerous
studies offer convincing arguments for entirely nontechnical solutions. Namely, these studies argue the
importance of using a standardized data collection form
and a general evaluation protocol. According to the
American College of Emergency Physicians, “Careful
collection of a uniform data set appears to be the most
important factor in attaining the greatest diagnostic
accuracy.”2 Dr. R. Rusnak’s investigation of
malpractice claims resulting from the misdiagnosis of
appendicitis suggests creating a uniform data collection
form like the one currently used for patients with chest
pain to “ensure a more thorough examination of
patients.”6 Cope wholly agrees with this premise
stating, “Radiologic or ultrasonic examinations…and
the vast array of lab tests available will not compensate
for a poor or incomplete history and physical
examination. If one is to make a correct diagnosis, a
complete history and physical examination should be
the rule.”5
The obvious importance of collecting a complete,
uniform data set prompted F.T. de Dombal to
incorporate a standardized data collection form into his
study of a computer based training package for
physicians. He studied the effect of these two methods,
combined and individually, on the abdominal pain
diagnostic process in both an urban and rural hospital.
The investigation yielded extremely encouraging results
including a decrease in the admission of non-surgical
patients, a decrease in the number of misdiagnosed
appendicitis cases, and a notable increase in physicians’
diagnostic accuracy.7 Using standardized forms in
addition to the training package produced the greatest
results, although using forms alone still made
significant improvements in the diagnostic process. De
Dombal concludes his report by stating, “…the
evidence in support of formally structured patient
interview pathways is now overwhelming, and these
should be routine.” 7
Because patients with abdominal pain make up such
a significant portion of emergency room cases, a
thorough characterization of the clinical epidemiology
in the emergency room is a valuable collection of
METHODS
A structured data collection form for patients
complaining of abdominal pain was developed using
existing examples from other institutions, medical
literature and input from experts at the University of
Virginia Health System.
The clinical experiment took place in the University
of Virginia Emergency Department. The experiment
restricted the participating physicians to emergency
medicine residents to minimize the effect of a learning
curve experienced by the non-emergency medicine
residents on an emergency medicine rotation. The onemonth time period was the maximum time allowable
without the threat of inconsistencies associated with the
monthly rotation schedules of the emergency medicine
residents.
The experiment was comprised of three specific
periods. A one-week baseline evaluation period served
as the control period from which changes in patient data
documentation was compared. In this phase, the
emergency medicine residents used the generic
University of Virginia Health System Emergency Room
Record as they were accustomed to doing. During the
two-week trial period that followed the baseline period,
the emergency medicine residents were instructed to
use the structured form instead of the generic form for
non-traumatic adults (18 years and older) complaining
of abdominal pain. The final week, the reevaluation
period, marked crossover change in the documentation
of the general form as a result of experience with the
structured form.
2
During all four weeks, the team reviewed all of the
charts for patients admitted into the Emergency
Department complaining of abdominal pain and
“graded” the thoroughness of the residents’ data
collection during the examination of abdominal pain
patients. Two teams comprised of two members each
graded the residents according to the constructed
grading criteria sheet. Both teams graded all of the
residents’ charts to ensure accuracy and inter-rater
reliability was measured to ensure consistency. The
two teams awarded the residents points based on the
residents’ documentation of specific symptoms, patient
history, physical examination, lab results, etc.
Subsequently, the teams recorded the “score” for each
data point, as well as the total score for each patient
examination, in a constructed Microsoft Access
database. The teams also entered the actual results for
each data entry into the database. Microsoft Access
was the chosen database software package due to its
ease of use, user-friendly interface, and ability to
transport data easily into Microsoft Excel or SAS for
statistical analysis.
Physical Examination Documentation Score: While
the improvements in comprehensive scores illustrated
the general benefit of using the new form, further
analysis focusing on a specific aspect of exam
documentation revealed encouraging results as well. In
this analysis, the team compared the total points earned
for all data fields associated with a patient’s symptoms
and medical history to measure how well physicians
documented information during the physical exam and
patient interview. The main difference between this
analysis and that of the comprehensive scores was that
this test excluded points earned for recording diagnostic
test results. The results of this analysis are shown in
Figure 2. As seen in the graph, each physician showed
a marked improvement when using the new abdominal
pain data form. Three of the four physicians
demonstrated statistically significant increases in their
scores (p<.05). Table 1 reports the summary statistics
for this analysis.
Figure 2
Mean Scores for Symptoms and Medical History
60
Demographics: During the four-week testing period
there were a total of 238 non-traumatic abdominal pain
patients analyzed. The patients’ ages ranged from 1892 years, with a median age of 38.5. Females
represented 64 % of the study population. The racial
make-up was 68% white, 31% black, and 1% other.
General Documentation Score: The average score
for all study cases during the baseline period was 35.4.
Implementing the form resulted in a score of 50.8, a
15.4% increase. This average score includes the scores
of the structured forms only. The average score for the
testing period including structured forms and general
forms was 38.6%. The post-trial evaluation period had
an average evaluation score of 31.6. These average
scores are depicted in Figure 1.
Figure 1
40
score
RESULTS
using new
abdominal
pain form
20
0
B*
I*
D*
H
physician
* indicates statistically significant improvement, p < .05
- error bars denote 1 standard deviation
Table 1
Symptom and Past Medical
History Score
physician
I
Scores Based on Study Period
60
using generic
form
50.8
score using
score using new
P
generic form
abdominal pain form
Value
29.04
41.58
.05
.05
B
29.2
43.37
D
25.88
42.77
.05
H
30.42
41.63
NSS
50
40
35.4
Documentation of Guarding and Rebound:
Guarding and rebound, signs observed during the
physical examination, are usually commented on
simultaneously, regardless of whether they are present.
Excluding rebounding from the structured form design
tests the effect of the form by comparing the
improvement in guarding documentation and decline in
31.6
30
20
10
0
Baseline
Trial
Re-evaluation
3
Table 2
rebounding documentation from the generic form
documentation and structured form documentation. The
design of the structured form included a check box for
guarding and not for rebound. When using the general
forms, the physicians commented both in 21% cases
(25.1% rebound, 23.1% guarding). The structured
form showed an increase in the documentation of
guarding (61.6%). In only one case, a physician
documented rebounding (7.1%). These results can be
seen in Figure 3.
Figure 3
Resident
A
B
C
D
E
F
H
I
J
Although the team anticipated an average compliance
rate of 70%, the actual average compliance rate of only
34% generated many unanswered questions regarding
the residents’ reasons for not using our form during the
abdominal pain patient examinations. Therefore, the
team created and administered a survey to the residents
to obtain their opinions on the effectiveness of the data
collection form and the reasons why they chose to use,
or not use, our form during the two-week period of the
testing phase. According to the returned surveys, the
team generated the following pie charts in Figures 5
and 6 depicting the residents’ reasons for using, or
failing to use, our form during abdominal pain patient
examinations.
Figure 5
Sensitivity of Structured Form
100
80
61.6
%
60
40
25.1
23.3
20
7.2
0
Form
No Form
Guarding
Rebound
Reasons the Residents Used the Form
Compliance: The team attempted to balance the
usability vs. thoroughness tradeoff while creating our
data collection form. This trade off is based on past
studies concerning patient collection forms and states
that if a form becomes too comprehensive, it will
become too time consuming and will not gain support
from the medical personnel using the form even if it
elicits a complete data set of patient information.
Despite our efforts, the residents only used our form
during 40 of 116 total patient examinations during the
second and third weeks of January. Figure 4 and Table
2 illustrate each resident’s compliance rate.
30%
Figure 6
Reasons the Residents Did Not Use the Form
Residents' Compliance with Using Our Form
During the 2nd and 3rd Weeks of January
40%
50%
Total # of AAP
Cases During the
2nd and 3rd Weeks
of January
20
# of AAP Cases
30%
20%
20%
Required
Easier to Document Multiple
Less
Time
Symptoms
More Thorough than Current Patient Data Form
Consuming
Figure 4
25
Compliance Rate
16.67%
45.45%
12.50%
72.22%
25.00%
8.70%
50.00%
50.00%
16.67%
15
10
10%
Forgot
Inconvenient to Acquire Form
Hard to Use for Multiple Chief Complaints
# of AAP Cases on
Which Our Form
was Used
5
The major reasons justifying the residents’ low
compliance rate were: the residents forgot to use our
data form and our form did not lend itself to recording
multiple chief complaints.
0
D
H
I
B
E
A
J
C
F
Residents
4
specific information. However, this result poses a
concern that structured forms are too constricting.
Significant research and several iterations ought to be
done before implementing a structured form
permanently. In order to be beneficial for special
situations, physicians ought to be encouraged to writein anything not included on the structured form.
To combat the problems associated with the
residents’ low compliance, future teams must
constantly monitor the residents and have someone
present in the emergency department twenty-four hours.
Only stringent monitoring can ensure the desired
compliance rates will be met. In addition, future
iterations of the form should make it easier to document
multiple chief complaints for a patient and provide a
method other than the current “checkbox method” to
clearly indicate positive or negative patient symptoms.
Furthermore, a longer “learning period” allowing the
residents more time to become familiar with the layout
and content oft the form should also be investigated in
future iterations.
INTERPRETATION OF RESULTS
The results revealed that physicians recorded much
more information from the physical exam and patient
interview when using the new data forms. The
predominant cause of improvement was that the new
form included specific prompts for this detailed
information. Thus, the new data form reminded
physicians to consider and document critical factors
pertaining to symptoms and medical histories. In a
sense, the form may have served as a script of questions
to guide physicians as they recorded a patient’s
medical, surgical and family history. Thus, physicians
using the new form were much more likely to record
detailed accounts of valuable, historical data. The new
form also reminded physicians to document precise
descriptions of physical exam results, such as the
quality, duration, and appearance of various symptoms.
Furthermore, physicians could record such information
quickly because the form provided lists of descriptive
adjectives following many of the symptom data fields.
Essentially, the new form caused physicians to
document more qualitative data from the physical exam
and patient interview by prompting them for specific,
descriptive information and providing an efficient
means for recording that information.
The fact that the new form encouraged physicians to
conduct and document a more thorough physical exam
and patient interview is advantageous for two reasons.
First, it ensures that physicians will document the
qualitative data required for the data warehouse when
they use the new data form. Equally important,
however, is that the results indicate the new data form
presumably caused physicians to devote notable time
and effort to the physical exam and patient interview.
As Dr. Zachary Cope states, “If one is to make a correct
diagnosis, a complete history and physical examination
should be the rule.”5 According to Cope, many
physicians are quick to rely on advanced diagnostic
testing technologies instead of their own medical
knowledge and intuition. The new data form opposed
this problem because it discouraged physicians from
neglecting a crucial part of the diagnostic process.
Thus, while the initial purpose was to gather high
quality data to populate the data warehouse, this
analysis indicated that simply using the new data form
may have caused physicians to conduct more thorough
examinations. Consequently, physicians and patients
can benefit from the new abdominal pain data form
even before the data warehouse is fully operable.
The results of the “guarding and rebound” test
demonstrated the power of a structured form to solicit
CONCLUSION
Despite the residents’ low compliance, the team’s study
depicts the usefulness and effectiveness of a
standardized data collection form for abdominal pain
patients. Average scores depicting the quality and
quantity of patient information recorded during a
patient examination increased from 35.4 to 50.8 for
those examinations during which the created
standardized form was used.
Future iterations of this study are needed to verify
the statistical results of the study and improve the
compliance rate amongst the residents during the
designated testing periods.
REFERENCES
1.
2.
3.
5
Powers, Robert D., MD, and Andrew T. Guertler,
MD. “Abdominal Pain in the ED: Stability and
Change Over 20 Years.” Division of Emergency
Medicine, University of Virginia Health Sciences
Center. Charlottesville:1995.
American College of Emergency Physicians.
“Clinical Policy for the Initial Approach to Patients
Presenting With a Chief Complaint of
Nontraumatic Acute Abdominal Pain.” Annals of
Emergency Medicine. Vol. 23, no. 4. April 1994:
906-922.
Brewer, R.J., G.T. Golden, D.C. Hitch, et al.
“Abdominal Pain, an Analysis of 1,000
Consecutive Cases in a University Hospital
Emergency Room.” American Journal of Surgery.
February 1976: 219-223.
Abernathy, Charles M., and Robert M. Hamm.
Surgical Scripts. Philadelphia: Hanley & Belfus,
1994.
Cope, Zachary. Cope’s Early Diagnosis of the
Acute Abdomen. Revised by William Silen, MD.
New York: Oxford University Press, 1996.
Rusnak, Robert A., MD, Joseph Borer, MD, and
Joseph Fastow, MD. “Misdiagnosis of Acute
Appendicitis: Common Features Discovered in
Cases After Litigation.” American Journal of
Emergency Medicine. Vol.12, no. 4: 397-402.
De Dombal, F.T., V. Dallos, and W.A.F. McAdam.
“Can Computer Aided Teaching Packages Improve
clinical Care in Patients with Acute Abdominal
Pain?” Yearbook of Medical Informatics 1992:
286-288.
4.
5.
6.
7.
.
BIOGRAPHIES
Craig Donnelly is a fourth-year Systems Engineering
student from Colonia, NJ. His primary contribution
was the qualitative analysis of the effectiveness of the
structured patient data collection form. Mr. Donnelly
will be working for PriceWaterhouseCoopers in New
York City.
Kelsey LeBeau is a Systems Engineering student from
Charlottesville, VA. Her primary contribution was
conducting the experiment. Miss LeBeau plans to
spend the next several months traveling.
Hannah McClellan, a fourth year Systems Engineering
student from Knoxville, TN, is also pursuing a minor in
Biomedical Engineering. Her principle contribution
was in the area of statistical analysis. Miss McClellan
has accepted a position with A.T. Kearney in
Alexandria, VA as a Business Analyst.
Matthew Patrick Thompson is a fourth year Systems
Engineering student from Woodbridge, VA. His
principle contribution was the construction and
management of the database. Mr. Thompson will
attend graduate school in the fall.
6