JAMES O. PAGE
“No ones death comes to pass without making
some impression, and those close to the
deceased inherit part of the liberated soul, and
thus become richer in their humaneness.”
An Introduction to
EMS Research
Bryan Bledsoe, DO, FACEP
Midlothian, Texas
Definitions

Research: careful,
systematic, patient
study and
investigation in some
field of knowledge,
undertaken to
discover or establish
facts or principles.
Definitions

Science:
 The
state or fact of knowing; knowledge.
 Systematized knowledge derived from
observation, study, and experimentation
carried on in order to determine the nature or
principles of what is being studied.
Is EMS Art or Science?
The knowledge of
EMS is science.
 The way it is applied
is art.
 Excellent EMTs know
the science of EMS
and use the art of
EMS to apply the
science.

Art or Science?
Some health practices
are more about art
than science.
 Chiropractic, for
example, has little
science and a lot of
art.

Art or Science?

Neurology today has
a great deal of
science and little art.
Art or Science?

When you have a life
threatening illness or
injury, would you seek
out a health care
provider whose
practice was more art
or more science?
Art or Science?
EMS is the safety net
of society.
 We are often the first
to provide care to the
injured and the
infirm.
 Our practices must be
based on science.

But we must apply them with art!
Research

Most medical research
is based on the
application of the
scientific method.
The Scientific Method

The scientific method is the process by
which scientists, collectively and over
time, endeavor to construct an accurate
(that is, reliable, consistent and nonarbitrary) representation of the world.
The Scientific Method

Steps:
1.
2.
3.
4.
5.
Observe some aspect of the universe
Invent a tentative description, called a hypothesis,
that is consistent with what you observed.
Use the hypothesis to make predictions.
Test those predictions by experiments or further
observation and modify the hypothesis in the light
of your results.
Repeat steps 3 and 4 until there are no
discrepancies between theory and experiment
and/or observation.
The Scientific Method

Hypothesis:
groundwork,
foundation
supposition; an
unproven theory.
The Scientific Method
The Scientific Method

The great advantage of the scientific method is
that it is unprejudiced.
One does not have to believe a given researcher, one
can redo the experiment and determine whether his
or her results are true or false.
 The conclusions will hold irrespective of the state of
mind, or the religious persuasion, or the state of
consciousness of the investigator and/or the subject
of the investigation.
 Faith, defined as a belief that is not based on logical
proof or material evidence, does not determine
whether a scientific theory is adopted or discarded.

The Scientific Method
A theory is accepted not based on the
prestige or convincing powers of the
proponent, but on the results obtained
through observations and/or experiments
which anyone can reproduce.
 The results obtained using the scientific
method are repeatable.

The Scientific Method

There are many types of “pseudoscientific”
theories which seem based on a mantle of
apparent experimental evidence but that,
when examined closely, are nothing but
statements of faith.
The Scientific Method

“Faith is believing
what you know ain’t
so.”
Mark Twain, 1894
“Following the Equator”

Science versus Pseudoscience

How to tell the difference.






Anecdotes do not make a science.
Scientific language does not make a science.
Bold statements do not make claims true.
Heresy does not equal correctness.
Reversal of the burden of proof.
Rumors do not equal reality.
Science versus Pseudoscience

How to tell the difference.







Failures are rationalized.
Use of emotive words and false analogies.
Ad ignorantiam reasoning (if you can’t
disprove a claim, it must be true).
Ad hominem or tu quoque statements.
Overreliance on authorities.
Circular reasoning.
Reductio ad absurdum reasoning.
The Scientific Method

Pseudoscience is still
a major part of our
culture. Many
practitioners resort to
pseudoscience and
other anecdotal
practices.
The Scientific Method

Typically uses terms
and images to appear
“scientific”.
The Scientific Method


Reliance on “names” and
anecdotes.
Dr. Greg Cynaumon?





Physician?
Psychologist?
Nutritionist?
Chiropractor?
Master’s and Doctorate in
psychology from Sierra
University (known
diploma mill).
The Scientific Method
No clinical evidence CortiSlim works.
 No information about contents (natural or
herbal).
 Dr. Talcott fringe nutritionist with loose
affiliations with University of Utah.
 One of the largest scams in recent history.

Use of the Scientific Method in EMS

Use of the scientific method in EMS.
 Observation:
During a discussion you and
your coworkers bring up the idea that there
are more psychiatric emergencies when the
moon is full.
 Hypothesis: Psychiatric emergencies are
more common when the lunar cycle is in the
full moon phase.
Use of the Scientific Method in EMS

Use of the scientific method in EMS.
 Prediction:
You predict that there will be
more EMS calls for psychiatric emergencies
when the moon is full as compared to other
times of the month.
Use of the Scientific Method in EMS

Use of the scientific method in EMS.
 Testing:
You set up an experimental protocol
to test your hypothesis.
 For this you:
 Determine
you will conduct the study for a year.
 Determine when a full moon is scheduled to
appear.
 Determine that you will declare two days before
and two days after the full moon a five-day “full
moon period.”
Use of the Scientific Method in EMS

Use of the scientific method in EMS.
 For
this you:
 Define
what a psychiatric emergency will be.
 Gather information over the course of the year that
includes the date of all psychiatric emergency
patients.
 When the data has been gathered for the period
previously determined, you determine:


1. How many psychiatric patients were there?
2. How many were treated during the full moon periods?
Use of the Scientific Method in EMS

For this you:
 Analyze
the data based on the number of
emergencies that arose during full moon
periods and compare to the number of
emergencies during the other periods.
 Perform any statistical tests necessary to
understand the data.
Use of the Scientific Method in EMS

Results:
Full Moon Period
Days=65
Other Period
Days = 300
Total
Days = 365
Ψ patients = 10
Ψ patients = 74
Total Ψ patients = 84
Total Patients =
2,105
Total Patients = 9,300 Total Patients = 11,315
Use of the Scientific Method in EMS

Use of the scientific method in EMS.
 Results:
 During
Your study finds:
full moon period 0.5% of patients were Ψ
patients.
 During other period 0.8% of patients were Ψ
patients.
 During the entire study period 0.7% of patients
were Ψ patients.
Use of the Scientific Method in EMS

Use of the scientific method in EMS.
 Results:
 The
Your study finds:
incidence of Ψ patients during the full moon
period was 0.15 patients per day.
 The incidence of Ψ patients during the other
moon phases was 0.25 patients per day.
 The incidence of Ψ patients overall was 0.23
patients per day.
Use of the Scientific Method in EMS

Use of the scientific method in EMS.
 Psych
patients are less common during the
full moon.
 HYPOTHESIS DISSPROVED!
Use of the Scientific Method in EMS

Use of the scientific method in EMS.

Revise hypothesis:




New hypothesis: Psychiatric emergencies are no more
frequent during full moon lunar phases than during other
lunar phases.
HYPOTHESIS ACCURATE.
Report findings. Paper should be detailed enough that
anybody can follow it and repeat your experiment.
Repeat study to determine whether findings can be
repeated.
EMS Research

The more valid a
study, the closer it is
to the truth!
EMS Research

Bias:
A
mental leaning or inclination; partiality;
prejudice; bent.
 The more a study design adheres to the
scientific method, the less chance for bias to
affect the outcomes.
EMS Research
Bias (sometimes flat
deception) has been a
common practice in
medicine and EMS.
 Quackery and bias
still permeates many
aspects of modern
healthcare.

EMS Research

Validity:




Whether the study measures what it was supposed to
measure.
Validity refers to the appropriateness of the
interpretation of the results of a study.
External Validity: The extent and appropriateness
of the generalizability of results.
Internal Validity: The basic minimum control,
measure, analysis and procedures necessary to make
results interpretable.
EMS Research

Constant:


A characteristic or condition that is the same for all
individuals in a study.
Variable:

A characteristic that takes on different values or
conditions for different individuals.



Dependent Variable: the variable being affected or
assumed to be affected my the independent variable.
Independent Variables: the variables that affects (or is
assumed to affect) the dependent variable under study.
Experimental Variable: at least one independent variable
being manipulated by the researcher.
Levels of Evidence
Levels of Evidence

Center for Evidence-Based Medicine
(Oxford)
Ia. Meta-analysis of RCTs
Ib. One RCT.
IIa. Controlled trial without randomisation.
IIb. One other type of quasi-experimental study.
III. Descriptive studies, such as comparative studies,
correlation studies, and case-control studies.
IV. Expert committee reports or opinions, or clinical
experience of respected authorities or both.
Levels of Evidence

American Heart Association
1. Positive randomized controlled trials.
2. Neutral randomized controlled trials.
3. Prospective, non-randomized controlled trials.
4. Retrospective, non-randomized controlled trials
5. Case series (no control group)
6. Animal studies
7. Extrapolations
8. Rational conjecture (common sense)
Study Designs

Randomized Controlled Trial (RCT):
A
group of patients are randomized into an
experimental group and a control group.
These groups are followed up for the
variables/outcomes of interest.
 RCTs most closely approximate the scientific
method and are the most valid of studies.
Randomized Controlled Trial
EMS Research

Houston MAST Study:
 Constant:
All patients with abdominal
trauma with hypotension transported by the
HFD to Ben Taub Hospital.
 Dependent Variable: survival from trauma.
 Independent Variables: age, sex, location
of injury, transport time, etc.
 Experimental Variable: Application or nonapplication of the MAST.
Example
Mattox KL, Bickell B, Pepe PE, Burch J, Feliciano
D. Prospective MAST study in 911 patients. J.
Trauma. 1989;29:1104-12
 911 trauma patients > 15 years of age with
systolic BP < 90 mmHg were randomized by
alternate day assignment. All transported by
HFD to Ben Taub hospital trauma center.
 Experimental variable: MAST or no-MAST
 Dependent variable: survival from trauma

Example
Independent variables: etiology, age, race,
sex, location of injury, trauma scores,
injury severity scores. Scores revealed the
two groups to be statistically identically
matched.
 Findings: Mortality rate 31% in the MAST
group and 25% in the non-MAST group.
 Difference statistically significant.

Double Blind Study
A double blind study is one in which
neither the patient nor the physician
knows whether the patient is receiving the
treatment of interest or the control
treatment.
 It is a type of RCT.

Double Blind Study
Double Blind Study
Bracken MB, Shephard MJ, Collins WF, et al. A
randomized, controlled trial of
methylprednisolone or naloxone in the treatment
of acute spinal-cord injury. NEJM
1990;322(20):1405-11
 Patients with spinal cord injury randomized by
computer to receive methyprednisolone,
naloxone, or placebo.
 Researchers and patients did not know which
drug was which (prepared in pharmacy)

Quasi-Experimental Studies

Quasi-experimental studies use intact
groups of subjects rather than assigning
subjects to groups at random.
Quasi-Experimental Studies

Non-randomized controlled trial:
1
group receives intervention
 1 group receives no-intervention (control)
 Subjects assigned to groups by methods other
than randomization.
Cohort Studies

A Cohort Study is a study in which
patients who presently have a certain
condition and/or receive a particular
treatment are followed over time and
compared with another group who are not
affected by the condition under
investigation.
Cohort Study
Cohort Studies

Framingham Heart Study:
 People
in Framingham, MA have been
followed for over 50 years.
 Conducted by National Heart, Lung, and Blood
Institute (NHLBI) and Boston University.
 5,209 residents between 30 and 60 years of
age initially enrolled.
 In 1971, 5,124 children (and their spouses) of
the original cohort added.
 500 minority members have been added.
Cohort Studies

Framingham Heart Study:
 Every
two to four years, study participants are
given extensive medical examinations
including a medical history, blood tests and
other sophisticated tests such as bone scans,
eye exams and echocardiograms assessing
multiple aspects of their current health status.
Cohort Studies

Framingham Heart Study (results):
 Risk
factors for heart disease identified.
 Better understanding of the effects of
lifestyle.
 More than 1,000 papers published
significantly adding to the knowledge base of
heart disease.
 Study continues….
Case-Control Studies

Case Control Studies: Case control studies
are studies in which patients who already
have a certain condition are compared
with people who do not.
Case-Control Studies
Case Series

A case series is a report on a series of
patients with an outcome of interest. No
control group is involved.
Case Report

A case report is a narrative report of an
interesting case.
Historical Studies

Historical studies are systematized
searches for the facts and then using the
information to describe, analyze, and
interpret the past.
Historical Studies

Bledsoe BE, Smith
MG. Medical
Helicopter Accidents
in the United States:
A 10-Year Review. J
Trauma.
2004;56:1325-1329
Historical Studies

Methods:Review of
all air medical
helicopter accidents
in the US from 19972002 from the NTSB
database.
Systematic Reviews

A summary of the medical literature that
uses explicit methods to systematically
search, critically appraise, and synthesize
the world literature on a specific issue.
Systematic Reviews

Bledsoe BE. Critical
Incident Stress
Management (CISM):
Benefit or Risk for
Emergency Services?
Prehosp Emerg Care.
2003;7:272-279
Meta-Analysis

A meta-analysis is a systematic review
that uses quantitative methods to
summarize the results.
Meta-Analysis
Meta-Analysis
Meta-analysis of RCTs represents the
highest level of scientific evidence.
 Allows for more objective appraisal of the
evidence.
 Reduces the possibility of false negative
results.

Meta-Analysis

van Emmerik AAP,
Kamphuis JH,
Hulsbosch AM,
Emmelkamp PMG.
Single-session
debriefing after
psychological trauma:
a meta-analysis.
Lancet.
2002;360:766-771
Meta-Analysis
Meta-Analysis (Observational Studies)
Less valid than RCTs.
 Still valid in that it gives better information
of the population as a whole.
 Many EMS interventions cannot be tested
with an RCT as it might be unethical to
withhold care from the control group.

Meta-Analysis (Observational Studies)
Bledsoe BE, Wesley AK, Eckstein M, Dunn
TM, O’Keefe MF. Helicopter transport of
trauma patients: a meta-analysis (in
press).
 Observational meta-analysis examining
validated trauma scoring systems in
trauma patients transported from the
scene to a trauma center.

Meta-Analysis (Observational Studies)

Conclusions:
2
out of 3 trauma patients transported from
the scene to a trauma center have minor
injuries based on validated trauma scoring
criteria.
 1 out of 4 patients transported are discharged
from the emergency department.
Animal Studies
Animal studies provide insight into biology.
 For ethical reasons, some studies cannot
be carried out on humans.
 Computer modeling are replacing many
animal studies.

Expert Opinion

Expert opinions can take various forms:
 Systematic
reviews
 Narrative reviews
 Pure opinion pieces
Rational Conjecture
Lowest level of
scientific validity.
 But, overall very
important.
 “You don’t have to
run a Chi-Square test
on common sense”

Summary
EMS must be driven by science.
 Science is based upon quality research.
 EMS providers of the future must be able
to understand, and in certain cases,
conduct valid research.

Summary

Evidence-Based Medicine:
 EMS
must start adhering to the tenets of
evidence-based medicine.
 Third-party payers will soon stop paying for
care and procedures not supported by
science.
 Litigation may follow if non-evidence-based
practices continue.
Evidence-Based Medicine

Evidence-Based Medicine is:


The conscientious, explicit and judicious use of the
current best evidence in making decisions about the
care of individual patients.
Evidence-Based Medicine is not:



“Cook-book” medicine
“Cost-cutting” medicine
“Old-hat” nor impossible to practice
Evidence-Based Medicine

Evidence-based medicine is not
restricted to randomized controlled trials
and similar studies. It involves tracking
down the best external evidence with
which to answer our clinical questions.
The End

Don’t be afraid of
research. It can
actually be fun!