EVIDENCE BASED PRACTICE
AND ITS IMPACT ON THE BOC
EXAMINATION
Athletic Training Educators’ Conference
Washington D.C.
February 25, 2011
Susan McGowen, PhD, EMT, ATC
University of New Mexico
Bernadette Olson, EdD, ATC
South Dakota State University
DRIVING CHANGE IN ATHLETIC TRAINING
PRACTICE:
Implementing Evidence Based Practice Skills into
Athletic Training Education Curriculum
 Educate future practitioners, students should
successfully demonstrate:



Use of evidence based practice during didactic/clinical experiences.
Implement EBP strategies into their own entry level practice careers.
Stretch:

As new practitioners, students will participate/drive change in the
organizations for which they work to advance EBP environments.
 Educate current practitioners
 Educate the educators:

Are we ready to serve as role models?
And the survey says…
 72% of participants
completed the survey…
 Program Directors (51%)
 Faculty (41%)
 Clinical Instructors (8%)
 Need participation from
all in curriculum to
integrate EBP
Why We Should Care - Improving Patient Outcomes
7
 (1) It is ethical: Care about patients/do no harm
 (2) Smarter patient(s): We are in an information era –
 patient advocacy
 (3) It is legal: We are held to a standard of care...
 (4) It’s contemporary: Everyone else is doing it…
 “The train is leaving the station…”
 (5) Profession Integrity: Making our mark as experts…
 (6) It is a competitive market: If you don’t get participants
back faster, then someone else will.
 (7) Cost Effectiveness: We can’t afford to continue treating
conditions the same way.
 (8) How the public views continuing competence: From seat
time to demonstrating a change in patient outcomes.
Barriers to EBP (in the classroom & practice)
 Knowledge base of educators and practitioners…
 Do we know enough to be teaching practices?
 Where do we get our information from?
 How much EBP is out there?...
 Is the research available for us to answer our clinical
questions?
 Relationship to theoretical research…
 Time to…
 learn
 practice
 disseminate
Evidence-based practice is the integration of best research evidence with
clinical expertise and patient values to improve patient outcomes.
9
Best Research Evidence
√ Clinically relevant research - patient-centered clinical research
√ Diagnosis, Prognosis, Therapy, Rehabilitation, Prevention.
Patient Values/Patient
Circumstances
Clinical Expertise
√ Clinical Skills/Past Experience
√ Rapidly identify each patient’s
unique health state and
diagnosis, individual risks and
benefits of potential
interventions, and personal
values and expectations.
Clinical
Decision
Making
√ Unique preferences,
concerns and expectations
that each patient brings to a
clinical encounter and that
must be integrated into
clinical decisions if they are to
serve the patient.
Improved Patient Outcomes
Crossing the Quality Chasm: A New Health System for the 21st Century (IOM, 2001)
Critical Appraisal…
10
How does this effect my
professional practice?
Clinical Bottom Line
Statement relating the study
being analyzed to clinical
question
Assigning Level of
Evidence/Ranking Evidence
Assess Validity: Internal
Validity, External and
Statistical, Key Evidence in
the Study Design,
Participants; Type of Sample
Utilized, Procedures followed
within the study, Outcome
Measures administered, Final
Results
Best Research
Evidence
Clinical
Expertise
Patient
Values
Formulate Clinical
Questions (PICO)
Intervention, Assessment,
Diagnosis or Prevention
Strategies
Locate Clinically Relevant
Literature
Medline, CINHAL. Web of
Science, Web of Knowledge,
Cochrane Library, Agency for
Healthcare Research and
Quality (AHRQ), ACP Journal
Club, Institute of Medicine
(IOM), Clinicaltrials.gov
Crossing the Quality Chasm: A New Health System for the 21st Century (IOM, 2001)
What are we preparing our students to do?
11
Be Consumers…
Be Producers…
 Inquisitive
 Is it time to participate in a
practitioners/lifelong
learners
 Effectively locate and
analyze literature
 Making decisions
 Collaborating with other
practitioners

Communication networks, Site
practice guidelines, Practice
patterns, Journal Clubs
study design?



Randomized control trials
Cohort Studies
Case Studies
 Disseminate information to
benefit other providers


Via publications
Via presentations
 Teaching others how to
participate in EBP practice
At minimum, an entry-level clinician
should be able to…
 Formulate sound clinical questions
 Locate literature relevant to question
 Critically appraise literature:
 A single article/study or arranged around a topic
 Quantitative or Qualitative
 Reflect and determine if this affects practice
 Others…
Take (10)
 How are you starting the conversations
in your programs…
What conversations is the
BOC having?
14
HOW DOES EBP INFLUENCE THE “BUSINESS”
OF THE BOC?
References
Role Delineation Study / Practice Analysis
Entry-Level Examination
REMEMBER – THE BOC EXAMINATION DOES NOT
DRIVE CHANGE, EDUCATION DRIVES CHANGE
Evidence used in the BOC examination reflects what is currently
accepted as common practice in athletic training.
BOC References
 How is the BOC Examination Reference List
determined?
EBP permeates
clinical and didactic
settings and
references
Program Director Reference Survey
 January 2011
 Survey Purpose:
 To determine references most commonly used by entry-level
ATEPs
 To determine preferred references when writing, reviewing
and validating items within exam development process by BOC
Exam Development Committee (EDC)
 Overall: program director references coincided with
what EDC members preferred
Program Director Reference Survey Summary
Educational Content Areas
as determined by the NATA ECE
% Used
by PDs
% Preferred
by EDC
Risk Management and Injury Prevention
87.5%
96.3%
Pathology of Injuries and Illnesses
74.4%
85.45%
Orthopedic Clinical Examination and Diagnosis
73.7%
70.91%
Medical Conditions and Disabilities
64.0%
61.54%
Acute Care of Injuries and Illnesses
82.6%
85.45%
Therapeutic Modalities
69.5%
74.55%
Conditioning and Rehabilitative Exercise
62.2%
77.78%
Pharmacology
58.4%
64.44%
Psychosocial Intervention and Referral
39.0%
36.54%
Nutritional Aspects of Injuries and Illnesses
52.0%
46.94%
Health Care Administration
80.3%
86.54%
Professional Development
81.8%
86.27%
18
Relationship between EBP and RDS
EBP permeates
clinical and didactic
settings and
references
RDS reflects use of
EBP
Role Delineation/Practice Analysis
 The RD/Practice Analysis is:
 A review of existing RD/Practice Analysis material
 Subject matter expert knowledge
 A large scale survey
 Findings are used to evaluate and support an entry
level athletic trainer competency test
RD/Practice Analysis 6
 Example from Domain 3: Immediate and Emergency
Care
How EBP affects the Exam Process
EBP permeates
clinical and didactic
settings and
references
RDS reflects use of
EBP
BOC Examination
utilizes evidence
based items
Influenced
by:
Clinical
Education
Influenced
by:
Didactic
24
Education
What does an
evidence
based practice
environment
look like
NOW?
Reflected
in current
6th Role
Delineation
Study
(RDS)
BOC
Examination
Design of the BOC Examination…
25
Role
Delineation/
Practice Analysis
Content Validity
Determination of
Passing Point
Development of
Test Blueprint
Review &
Revision
Test
Administration
Item
Development,
Review &
Validation
Test Assembly
Psychometric
Analysis
Multiple Choice Item Development
 Triangulation
Subject Matter Experts
Statistical
Analysis
Double
Reference
Multiple Choice Item Development
 Subject matter experts
 Represent 5 domains
 Items based on:
 Importance
 Criticality
 Frequency
Multiple Choice Item Development
 Importance
 Is competence in a domain essential for a minimally
competent certified athletic trainer?
 Criticality
 Can adverse effects result if the certified athletic trainer is not
competent in a domain?
 Frequency
 What percentage of time does the certified athletic trainer
spend performing duties associated with the performance
domain?
Breakout Session
29
 Divide into groups
 Construct 3 multiple choice items
 Recall
 Analysis
 Application
 Include question, correct answer and four distracters
(incorrect responses)
Multiple Choice Item Principles
 Entry-level
 Clear and concise
 Written in question form
 Third person
 Stem (question) stated in a positive matter
 Distracters clearly wrong, but plausible
 Double reference
 Be aware of state/local/regional differences
Adapted from the BOC Style Guide
MC Item Examples: Recall
INITIAL ITEM
REVISED ITEM
 According to Richard Ray, there are
 What budget type requires an
specific budget types used by athletic
training facilities to project future
incomes and track expenses. Which of
the following budget type requires an
athletic trainer to list anticipated
expenditures for specific categories of
program functions, including
equipment repair, insurance, and
expendable supplies? Choose only
one.





Fixed
Line-item
Lump-sum
Performance
Zero-based
athletic trainer to list anticipated
expenditures for specific categories
of program functions? Choose only
one.





Fixed
Line-item
Lump-sum
Performance
Zero-based
(clear and concise stem)
MC Item Examples: Analysis
REVISED ITEM
INITIAL ITEM
ABC Athletic Training Clinic
Anywhere, USA
Budget: First Quarter 2014
Expenditures by Type
Section A
ABC Athletic Training Clinic
Anywhere, USA
Budget: First Quarter 2014
Expenditures by Type
Section A
2” white tape
10 cs
Rub. alcohol
2” ACE wraps
8 pt
2” white tape
Rubbing alcohol
10 cases
8 pints
10/box
2” elastic wraps
10/box
Section B
Section B
Staff meeting refreshments
Varies by week
Staff meeting refreshments
Varies by week
Copy costs when copier is broken
Variable
Copy costs when copier is broken
Variable
Immediate purchase of expendable supplies
Variable
Immediate purchase of expendable supplies
Variable
Section C
Section C
Taping/treatment tables
4 tables
Taping/treatment tables
4 tables
Treadmill for rehabilitation room
1 unit
Treadmill for rehabilitation room
1 unit
Section C of the budget shown below includes which
type of expenditures? Choose only one.
Capital
Cash
Line item
Operating
Departmental
(use of abbreviations
and trademarked terms)
MC Item Examples: Application
INITIAL ITEM
REVISED ITEM
 A patient presents to the athletic
 A patient presents to the athletic
trainer with pain, moderate effusion,
and instability of their left knee after
experiencing an incident where they
felt a “pop” while making a quick
lateral movement about a planted
foot. Which of the following
structures are MOST likely affected?
Choose only one.





ACL
MCL
LCL
ATF
CF
trainer with pain, moderate effusion,
and instability of their left knee after
experiencing an incident where they
felt a “pop” while making a quick
lateral movement about a planted
foot. Which of the following
structures are MOST likely affected?
Choose all that apply.





Anterior cruciate ligament
Medial collateral ligament
Lateral collateral ligament
Meniscus
Patella Tendon
(length of distracters, wrong but
plausible, abbreviations)
*Influence of
Vision for the
Profession
(what do we
want to look
like in 5 yrs /
10 yrs)
*Influence of
Educators –
Drive Change in
Practice and
Culture
*Influence of
Current
Practitioners
Influenced
34
Changes
in
Professional
Education (4th
Competencies and
Proficiencies)
Influenced by
Professional
Development we
seek…
What does an
evidence
based practice
environment
look like in
5 years?
Reflected
in 7th
(RDS)/
Practice
Analysis
Closing Comments…
35
The BOC does not create evidence
The examination reflects what is currently accepted as
common practice in athletic training
Discussion
THANK YOU!