Curricular Management Track
Ron LeFebvre, MA, DC
Jerrilyn Cambron, DC
7/9/15
EBM
 EIP
 EBP
 EBCP

 Teaching
interventions
informed by
clinical
research
 Teaching
how to find,
assess and
use clinical
research.
“Research findings from underpowered, earlyphase clinical trials would be true about one in
four times, or even less frequently if biases
present.”
Ioannidis JPA, Why Most Published Research Findings Are False. PLoS
Medicine. August 2005, 2(8), e124 www.plosmedicine.org

A total of 756 articles re-addressing a medical
practice were examined. 165 were back to the
drawing board, 146 were medical reversals, 138
were reaffirmations, and 139 were
inconclusive. Of the 363 articles testing
standards of care, 146 (40.2%) reversed that
practice, whereas 138 (38.0%) reaffirmed it.
Prasad V, MD; Vandross A, MD; Toomey C, MD; Cheung M, MD; Rho J, MD;
Quinn S, MD; Chacko SJ, MD; Borkar D, MD; Gall V, MD; Selvaraj S, MD; Ho
N, MD; Cifu A, MD. A decade of reversal: an analysis of 146 contradicted
medical practices. Mayo Clinic Proc. 2013;88(8):790-8.
• Good research is hard to do
• Most is flawed, much fatally flawed
• Much of it is intended for other
researchers—not ready for primetime
0.50
The Research Evidence





7 Administrators (skills?)
7 General faculty (with “good skills”)
7 Clin faculty
5 EIP “experts”
Everyone else





21 Chiropractic
5 Oriental Medicine
3 Massage Therapy
3 Midwifery
1 Integrative Health Program
#1: Get ideas
#2: Identify
resources
People &
Programs
Organizations
Course
Assignment/
Focus
Microbiology/ Lit search,
Public Health HARM
studies, odds
ratios.
Nutrition
Lit search,
study types
Genetics
Presentation
on prevalence
and post-test
probability
Immunology Poster
assignment
Faculty
College
Contact Information
Kara
Burnham
UWS
kburnham@uws.edu
Jim Gerber UWS
jgerber@uws
Mark
Kaminski
UWS
mkaminski@uws.edu
Verna van
Fleet
Northwestern vvanfleet@nwhealth.edu
#3: Ask for more
help/clarification
#4 Make an Action Plan
Thursday
1:00-3:00
One example: UWS
Library
Research
(Admin)
R-25
Investigator
(Faculty)
Course Instructor
(Basic Science Faculty)
Chair of
Clinical Education,
Chiro Science & Clinical
Science
YEAR 3
YEAR 2
YEAR 1
1.
2.
3.
4.
5.
6.
7.
Decide what you want
Adopt learning objectives
Build a core series of courses by re-structuring
what you already have
Integrate with the rest of the curriculum
Train the faculty
Survey, map & measure (OUTCOMES)
Do it better
Research Methodology vs EIP
The graduate must be able to do the following:



ASK a searchable clinical question,
ACQUIRE/ACCESS the best evidence available,
APPRAISE the quality and applicability of that
evidence,


APPLY the evidence into the care being offered.
Self ASSESS
(based on the Sicily conference, Dawes 2005).







Explain the role of EBCP in chiropractic practice
and education.
Ask a focused clinical question based on the
PICO model.
Conduct an effective evidence-based literature
search.
Utilize electronic information sources including
various search engines.
Critically appraise a clinical study.
Synthesize and interpret the evidence.
Apply the evidence to the management of a
patient case.
The EBP competent practitioner
 can present an overview of EBP.
 can translate an issue of clinical uncertainty into an
answerable question.
 can efficiently and effectively search for and retrieve
useful and up-to-date healthcare information and
evidence.
 critically appraises the evidence for validity and
clinical importance.
 applies the relevant evidence to practice.
 engages in self-evaluation of his/her process for
accessing, appraising and incorporating new evidence
into practice.
General Competency Statement: Explain, evaluate, and apply scientific evidence
in the context of practitioner experience and patient preferences and apply
evidence informed decision-making in integrated healthcare delivery.







EP1. Explain the role of scientific evidence in healthcare in the context of
practitioner experience and patient preferences.
EP2. Describe common methodologies within the context of both clinical and
mechanistic research, focusing on an assessment of your own field.
EP3. Discuss contemporary issues in integrative practice research, including
those relative to evaluating whole practices, whole systems, disciplines,
patient-centered approaches and health outcomes.
EP4. Analyze the research base within one’s own discipline including the
positive and negative interactions, indications and contraindications for one’s
own modalities and agents.
EP5. Apply fundamental skills in research evaluation.
EP6. Demonstrate evidence informed decision making in clinical care.
EP7. Discuss the value of evidence informed risk management planning and
risk management behavior.
1.
2.
3.
4.
Can find, assess the quality, and build a set of preappraised lit sources (including push services) for
their own use.
Can conduct a search in PUBMED (using multiple
search terms and limiters)
Can understand the language of therapy outcomes
(e.g., RR, OR, NNT, etc.) and diagnostic studies
(e.g., LRs, sensitivity, specificity).
Can judge at a rudimentary level if study results
are precise and statistically & clinically significant.
5.
6.
7.
8.
Can do a quick, simple assessment of the quality of
a therapy study (e.g., using the ABCDFIX
mnemonic) a diagnostic study, and a harm study.
Can interpret test results in the light of pre-test
probability to guessitimate the post-probability of
a patient having a condition.
Know what to consider when applying research
results to patient care (i.e., generalizability issues,
patient preferences/values, clinical experience)
Can assess the quality of a systemic review and a
clinical practice guideline and knows when a
clinical prediction rule is ripe for use.

Once the curricular changes are in place,
someone must continue to manage it.
Who?
 How?
 By what authority?

Core EIP
courses
Clinic
Training
Larger program
curriculum
Journal
clubs
Consolidate core
courses (research,
library, an hour or
two here or there)
Shift core courses
from doer to user
of research
literature
Information Mastery
(1 hr Q2)
Clin Topics I
EBP 1 (2 hr Q4)
Clin Topics II
EBP 2 (2h Q5)
Therapy
Research Method 1
Systematic reviews, harm,
diagnosis
Research Method II
EBP 3 (1 hr Q10)
Journal club
EBP 4 (1 hr Q11)
Journal Club
National
Northwestern Palmer
UWS
Number of
courses
4 1/3
3
2
5
Number of
contact hours
58.5
46
60
70
1, 2, 3
1,3
1, 3
1,2,4
Program
Years
Thursday
4:30-6



TIME
EXPERTISE
WILLINGNESS
“Every new patient work up includes two
‘Learning Objectives.’ The exercise includes
reflecting on a case work up to identify
knowledge gaps, convert them into searchable
questions and then search the literature. The
findings are appraised and incorporated into
management.”


When medical
doctors were asked
how often they
need to search for
an answer for a
patient case, they
answered about
1x/week
But when patient
encounters were
observed, a clinical
question was
identified for every
2-3 patients.
Bate L, Hutchinson A. How clinical decisions are made. British Journal of Clinical
Pharmacology. 2012; 74:4 614-620
Identifying searchable,
clinical questions is a learned
skill!
Search
question
Student
search
Report &
feedback

Background questions

Foreground questions
FEEDBACK
On the search
 On quality & quantity of the evidence
 On how robust the results are
(clinically significant ? clinically useful?)
 The applicability to the patient

1.
2.
3.
4.
5.
“Quick hits”
Case Presentations
CATS & “Super CATS”
Annotated Treatment Plans
Intern Lounge Activities





Patient based, clinical supervisor driven
Focused clinical question
Informal
“Just in Time” to affect care or report to patient
Undocumented

Individual case
presentation (one on
one with supervisor)

Part of Grand
Rounds/small group
presentation



What if the student’s patient population
doesn’t lend itself to searchable clinical
questions?
Then play the “what if” game.
Change a factor in the case and formulate a
clinical question that you know there is an
answer to.


Exercises (SPs, paper cases) that can be
documented in which treatment plans have
references.
Complex Case Credits


Focus is on the study more than the patient
UWS: 9Q interns present 1 “Super CAT” in
small group with clinician.
CLINIC CAT
CAT Author:
Date Completed: 6/2/15
1) Brief patient scenario: 26 year old female with symptomatic cam-type femoroacetabular impingement1) Results: Answer to the study question
 Express results quantitatively including the following:
(FAI)
o P value & statistically significance: Not reported
2) Clinical Question: P Patients with femoroacetabular impingement_ I Conservative care C_surgery O
o Clinical significance: Not clinically significant due to lack of numerical results. This review only
Decreased symptoms .
included the qualitative results from the 2 primary research articles that they graded as “low”
3) Search string used: (femoroacetabular OR hip) AND impingement AND (conservative OR chiropractic
quality (vs the 3 that were “very low”), and no statistical values were included for either. Both
OR manipulation OR manual therapy)
studies suggested that FAI patients benefit from non-operative therapy, but this review fails to
provide further details. The article by Hunt et al studied patients with “prearthritic hip disease,”
4) Databases/Search engines accessed: PubMed, PEDRO, TRIP, CINAHL
which included FAI as a subtype, but could not analyze each subtype separately due to low
numbers of participants. Six of the 17 patients with FAI reported “satisfaction” with conserveative care, but 11 ended up receiving surgery. The entire group of patients (all categories)
showed improvements from baseline in HHS (69.4 to 78.96) and NAHS (70.4 to 81.6).
The study by Emara et al was more definitive in defining patients with cam-type FAI, and their
7) Study Type (Design): Mark all that apply.
results suggested that non-operative treatment can help patient’s symptoms and function, but
no statistical analysis was included. In the Emara study, 33 patients treated nonoperatively
Systematic review (qualitative)
Systematic review (Meta-analysis)
RCT
Cohort
Case
showed mean improvement in HHS and NAHS, from 72 to 91 for both. Four patients “failed”
control
Guidelines
Clinical Prediction Rule
Cross sectional
Other (identify)
the nonoperative treatment and received surgery, after which no outcomes were reported. It
Prospective design
Retrospective design
was not stated what was used to classify a patient as “failed” conservative care. No p values
8) Bottom line (EBP sandwich format)
were given. They clearly defined their physical therapy regimen,but their criteria for patients to
be eligible for the study was different from the criteria used in studies that researched the
 Answer to your question (meat): There may be some therapeutic benefit to utilizing non-operative
outcomes from surgery.
treatments for patients with femoroacetabular impingement, rather than surgery, although there is
According to Jager et al, all 17 patients treated nonoperatively were still experiencing pain and
limited evidence available to support it.
hip dysfunction after an average 16 month follow up, while the 8 patients who underwent
 Magnitude of effect: This review failed to report any numerical values, stating only that the included
surgery graded their results as “excellent” or “good” after an average 26 month follow up.
studies showed some benefit for symptom relief in FAI patients, but it is inconclusive. It is therefore
The study by Feeley et al showed that the 8 NFL players with FAI treated with physical therapy
not possible to determine statistical significance for this review. Clinical significance is low due to
returned to playing in the NFL, though the follow up was not defined. Of the 5 players treated
the lack of statistics or confidence intervals, there is not enough evidence to base a clinical
with FAI surgery, 4 of them played in the NFL within 6 months of surgery. 1 surgical patient
decision on.
played 1 year in the NFL, then had to retire.
 Strength of evidence: While this review is fairly well done, the actual evidence for using nonThere were no numerical results from the review/discussion articles, but 48% of them
operative therapy for FAI patients is of low quality. There are only 5 primary research studies, and
promoted physical therapy-led care for FAI, and >50% promoted a trial of conservative care,
of those, only 2 had quantitative outcome measures. There are no published RCTs regarding this
including activity modifications and OTC NSAIDs. However, without any published RCTs it is
hard to determine whether this information can be applied to patient care.
subject, further lowering the quality of evidence.
2) Notable weaknesses or strengths?
9) Methods summary:
 The authors listed strengths of their review, including using a registered review protocol, a
 Question (hypothesis): Have any nonsurgical treatments been effective for reducing symptoms of
reproducible search strategy, application of PRISMA statement, and using quality assessment
FAI?
tools to grade the quality of evidence. I would agree that these are all strengths of this particular
 Population studied and setting: Patients with hip injuries
review. However there are several weaknesses, including the lack of results reported (no
 Number of subjects in study groups: 310, 17, 678, 37, 52 (Five primary research studies)
numerical values, no p values or confidence intervals), the limited amount of studies (though this is
 Assessment outcomes/ measures used: Primary outcome measure not defined in all 5 primary
not necessarily the fault of the reviewers – there is limited evidence available), and the
research articles. The two with higher evidence ratings included Non Arthritic Hip Score (NAHS)
broad/vague inclusion and exclusion criteria used. The authors also listed biases within the
and Harris Hip Score (HHS). The three with very low evidence ratings did not include quantitative
individual studies that caused them to be rated as low or very low evidence. The main one was
measures.
lack of a defined primary outcome marker, which was a problem in all 5 research studies. There
was also no evidence of sample size calculation in 4 of the 5, and the three studies rated “very low”
o Data bases searched: PubMed, Ovid Medline, Excerpta Medica Database, CINAHL, Allied and
showed no outcome of homogeneity testing. It was also mentioned that both the patient
Complimentary Medicine Database, Cochrane Library databases. Additionally they searched
characteristics and the type and definition of FAI used varied significantly amongst the studies,
the International Standard Randomised Controlled Trial Number Register and MetaRegister of
making it difficult to compare results accurately.
Controlled Trials for reports of ongoing and unpublished trials.
3) Applicable/generalizable:
o Search terms: femoroacetabular impingement, femoro-acetabular impingement, and hip
 Yes
No
Not sure
impingement.
 Why: More research needs to be done, preferably RCTs, in order to determine if non-operative
o Number of studies and patients: There were 5 articles that discussed primary experimental
treatment is statistically and clinically significant in reducing the symptoms of FAI. The limited
evidence (3 prospective case studies, one retrospective study, and one descriptive
research that has already been done in this area suggests that it can be beneficial, but I am
epidemiologic study), and 48 additional included studies that were classified as review or
hesitant to apply this to my patients without more solid evidence.
5) Reference: Wall PDH, Fernandez M, Griffin DR, Foster NE. Nonoperative treatment for
femoroacetabular impingement: a systematic review of the literature. PM R 2013: 5, 418-426.
6) Study Category:
Diagnosis
Therapy
Harm
Prognosis
Other
o
discussion articles about FAI.
Studies quality rated: Yes
No
Method used GRADE tool (for primary research
studies), quality assessment tool specifically designed for case series (for primary research
NECESSARY INGREDIENTS
WILLINGNESS
RELEVANCE
ABILITY
TOOLS & OPPORTUNITY
“Really helping clinicians - we used this as a way
to get interns into literature, and bring the
clinicians along in a way. But the clinicians need
to be expert, and we need more time for clinicians
and interns to really appraise together and
incorporate together into treatment plans.”
First year: 10 foundational workshops
Second year: 5 journal club meetings
Follow up: One 60-90 hour workshop each
quarter





Lit search workshops
Reading results workshops
Appraisal of therapy study workshops
Patient application workshop
Probability & diagnosis
EIP
Cue Cards
John Stites, DC DACBR
Friday
8:30-10


Research evidence that someone else has read
and summarized and/or pre-appraised.
Often the starting point for the busy
practitioner.
1.
2.
3.
4.
What is it most relevant for? (Oriental
medicine? Midwifery? Massage? Chiropractic?
PC, NMS? Manual Therapy? Nutrition?)
Who is doing the filtering/appraising?
How often is it updated?
What is the quality & transparency of the
appraisals?
5.
How thoroughly does it scan the literature?
6.
How easy is it to use?
7.
How expensive?
Criteria
Dynamed
1. What is it most relevant for?
Primary care (PC), neuromusculoskeletal
care (NMS))—weak in manual therapy.
2. Who is doing the
filtering/appraising?
Medical physicians in various specialties
3. How often is it updated?
Daily
4. What is the quality &
transparency of the appraisals?
Transparent (offers criteria, assigns quality
levels)
Quality of appraisal: ?
5. How thoroughly does it scan the
literature?
Thorough. Many journals cover to cover, plus
6. How easy is it to use?
Easy
7. How expensive?
$300+ for a practitioner
(NMS is done by Rheumatologists, not physical
therapists, orthopedists, or physiatrists)
regularly searches for new guidelines ,
systematic reviews; additional journals that are
checked include JMPT and Manual Therapy.
2008 systematic review based on 9 trials


3 RCTs (n=128) and 6 non randomized trials (n=356).
One study had adequate methodological quality (6 out
of 9 quality scale).
 Complement the Cochrane Reviews
Criteria
PEDro
1. What is it most relevant for?
Neuromusculoskeletal care (NMS),
manual therapy (treatment only)
2. Who is doing the
filtering/appraising?
Australian editors; physical therapists
3. How often is it updated?
Updates are done once a month, usually
the first Monday (e.g., 5/6/13: 32 in MS, 4
ortho, 10 sports, 1 whiplash)
New updates can be pushed to you.
4. What is the quality &
transparency of the appraisals?
Transparent (lists specific criteria, assigns
quality score for RCTs)
Quality of appraisal: ?
5. How thoroughly does it scan the
literature?
Unknown
6. How easy is it to use?
Moderately easy. Although search engine
is not very sophisticated, hard to narrow to
relevant topic.
7. How expensive?
Free
www.coffeebreaku.com



PURLs
Clinical Inquires
Applied Evidence
http://www.wfcsuggestedreadinglist.com
Librarians as Resources



At the Birmigham Women’s Hospital, the Trust
librarian has been involved in the journal club for
the last four years with remarkable success.
Auditing of the critical appraisal topics revealed
that in the years before the involvement of the
librarian, for 22% of the topics the most relevant
articles were missed, compared with 3% in the
years afterwards.
Coomarasamy A. Latthe P, et al. Critical Appraisal
in clinical practice: sometimes irrelevant,
occasionally invalid. JR Soc Med 2001:94:573-7.
Friday
10:30-12
Different Levels
of Expertise
Create content experts and enhance training of
other faculty
 Intensive workshops
Oxford
 McMaster’s
 Duke
 Tufts
 CEIPE faculty training workshops


Advanced degrees: (e..g, UWS grant funded 1 core EIP
instructor in master’s clinical research)
Preappraised
lit
Search
skills
& push
services
Reading
results
Assessing
quality
Preappraised
lit
Search
skills
& push
services
Reading
results
Assessing
quality
P value
Exercises in general combining all of the studies

RR 0.63 (95% CI 0.53 - 0.75, I2 = 70%)
Strength Training

RR 0.31 (95% CI 0.20-0.48, I2 = 0%)
Stretching

RR 0.96 (95% CI 0.84-1.09, I2 = 0%)
+LR = 3.4 for
acute appendicitis
McGee S. Evidence-Based Physical Diagnosis Elsevier 3rd edition 2007
Preappraised
lit
Search
skills
& push
services
Reading
results
Assessing
quality
ABCDFIX
GRADE
Strategy 1:
Train the trainers

SCUHS
experience
http://www.csh.umn.edu/evidenceinformedpracticemodules/
SCU EIP Plan
PIE
7/8/15
• Faculty training
• Curricula planning
• Student training
1.
2.
3.
4.
Overview
Types of Research
Using Evidence in Practice
Understanding Research & Statistics
• How can topic be incorporated into
department courses?
• Capstone project?
• Final session – curricula maps
Students completed same modules in
Research Methods Class
Still to be Accomplished
• Follow-up of planned curricula
changes
• Capstone project planning
committee
Strategy 3: The Hybrid
model
All Faculty


10 two-hour modules (didactic & small group, met
monthly—leading to certificate of completion)
a self paced learning module on information literacy
Classroom Faculty



5 one-hour presentations on how to bring EBP into
the classroom (spread over 2 quarters)
4 one-hour departmental workshops on information
mastery
Starting a year ago, 4 60-90 workshops per year
Clinicians


7 one-hour journal club meetings (spread over one
year)
8 two-hour workshops on EBP and information
mastery (spread over two years)
New Faculty
7 one-hour Moodle-based Information Mastery
modules
 10 2-hour EBP modules
 Starting 2 years ago, 4 training workshops a year







Keep a clinical perspective
Relevant examples/articles
Space out the training
Many small bites at the apple
Redundancy and review
Apply the skills (journal club)
Active incentives (which ones can you muster?)
 The Administrative bully pulpit
 P&E
 CE hours
 Certificate
 Food
Passive incentives (critical mass)
 Peer pressure
 Student pressure (the Berlin principle)
Brain storming & Planning
Revisited…



Part of lecture (by faculty)
Part of a CAT or checklist assessment (by a
student)
In the clinic (by an intern)
Lecture
with
hand
waving
Overheads
Power
point
Power point Power point
with evidence with EIP
sandwich
Patients traveling > 5 hours by
air should wear compression
stockings to lower their risk of
DVT.

Patients traveling 5 hours or more by air
should wear compression stockings to lower
their risk of DVT.
NNT = 8
 Level
on the
pyramid
 Quality of the
study
(strengths &
weaknesses)


Patients traveling 5 hours or more by air
should wear compression stockings to lower
their risk of DVT.
NNT = 8
(small moderate quality RCT, Schurr 2001)
Systematic review of high quality RCTs (how
many RCTs?)
 Systematic review of mixed quality RCTs (how
many RCTs?)
 Individual RCTs?
 Individual cohort or case
control studies?
 Case series?








Allocation concealed and randomization
proper?
Blinding adequate?
Comparable groups, comparably managed?
Drop outs acceptable?
Follow up long enough?
Intention to treat?
X factor: any other major weakness or bias?

If you read a Guideline, what was the strength
of their recommendation and what level of
evidence was cited.



Strength of recommendation: I*
(AHA/ACC)
Level of evidence: A (AHA/ACC**)
Treatment effect: decrease 5-6/3
mmHg
*A recommendation of I indicates that the benefits greatly outweigh the risks and the guidelines panel
indicates that the treatment SHOULD be followed.
**American College of Cardiology/American Heart Association Task Force
Results
precise?
Manual acupuncture was an effective and safe
treatment for short-term relief of frequent migraine
in adults. In a small RCT, acupuncture was more
effective than sham acupuncture reducing the
number of migraine days per month by a mean of
4.4 (CI 95% -7.2,-1.4, p = 0.005).

Spinal manipulative therapy reduced migraine
severity and frequency compared to the
placebo. It reported statistically significant
improvements in migraine frequency (p< .005)
and medication use (p<.001) when compared to
the control group. This RCT is of moderate
quality but not high due to a limited sample
size and incomplete blinding of the practitioner
and outcome assessors.

Focused extracorporeal shock wave therapy
(ESWT) is useful in treating painful heel
syndrome. Therapy showed a 73.2% reduction
in composite heel pain. Unfortunately, the
difference in achieved results with this small
group is not statistically significant. This study
was of high quality because of strict screening
criteria, patient directed treatment without
anesthesia, and being a randomized, doubleblind study.

After 20 weeks of myofascial therapy, the
experimental group showed a significant
improvement (p < 0.05) in painful tender
points, McGill Pain Score (p < 0.032), physical
function (p < 0.029), and clinical severity (p <
0.039). The results suggest that myofascial
release techniques can be a complementary
therapy for pain symptoms, physical function
and clinical severity. The PEDro score was 6
(lacking concealed allocation and an intention
to treat analysis).
Saturday
8-9:30
Faculty &
Classroom
Students
Clinicians &
Clinics
Quarter
Course
Assignment
When
Time
1
Phil & Prin 1
Search
Wk 4-5
1 hr
2
Clin Topics I
Moodle, CAT
?
3
Gross Anat III
Phil & Prin III
Clin Topics II
Search/work sheet
Lecture
CAT
Janet – lecture
Access Pre-Appraised @ Primary Study
Intro to CAT, Pub Med
Lit search and a paper
CAT
Search/Analyze
Wk 2
Wk 8
Wk 6
N/A
Wk 6
Wk 4 & 7
Wk 6
?
1 hr
Wk 8
Wk 2 & 4
1 hr
2nd assignmt = 1530 min
Wk 2
1 hr
Wk 9
2-6 hrs
Wk 3-9
?
in lab
?
4
5
EBP I
Microbiology
Pathology
Clin Micro
EBP II
Prin IV
Nutrition
6
7
8
8/9
9
10/11
2 hrs
N/A
?
NMS I
PT II
Clin Phase I
Clin Phase 1 lab
Clin Phase II
CAT
Search/Critique 3 articles
RCT/observe study/system review
Lit search & paper
Lit search and paper
Clin question/CAT
Search Cochrane library
CAT/Clin question
Clin Phase 1 lab
Search Sports Discus
in lab
Clin Nutr/Bot I&II
4 research assignment using database
½ hr each
Clin Phase III
Answer to question using pre-appraised sources
(TRIP/Dynamed)
Critically review 3 papers, journal club format
Wk 4-7
Check list assessment of 4 types of studies (all
interns); CATs (10s only)
Wk 1-10
Prin & Phil V
EBP III & EBP IV
Wk 3-9
?
?
?
Posing a Question
Courses
PICO review
Students pose a PICO
question
Search term review
Assess question
Assess search terms
Phil & Prin 1
Clin Topics I
Gross Anat III
Phil & Prin III
Clin Topics II
EBP I
Microbiology
Pathology I
Clin Micro/Public Health
EBP II
Prin IV
Nutrition
NMS I
Clin Phase I
EBP III & EBP IV
no
yes
no
no
yes
yes
yes
no
no
no
no
no
no
brief
yes/no
no
yes
no
no
yes
yes
yes
no
yes
no
no
no
no
yes
yes
yes
no
yes
no
no
no
yes
no
no
yes
yes
no
yes
no
no
yes
no
no
yes
no
no
no
no
no
no
no
no
yes
yes/no
yes
no
no
no
no
no
no
no
no
no
no
no
no
no
159





Formulating a question (PICO)
Composing a search string
Choosing data bases
Searching for a study that answers the clinical
question
Searching for the best available evidence to
answer a clinical question
ATTITUDE
SKILLS
BEHAVIOR
COMPLETE ASSESSMENT
Effects of the UWS R25 EBP Curriculum on
Knowledge, Attitudes, Skills, and Behavior
Mitchell Haas, DC, MA
Associate VP of Research, UWS
Funding: NCCAM / NIH (R25 AT002880)
CAM Practitioner Research Education Project Grant Partnership PAR-04-097
EBP Questionnaires
 Skills self-appraisal (4 items on 7-point Likert Scales):
 Anchors: 1 = not at all competent to 7 = very competent.
 Scored: 4 to 28.
 Ability to:
•
•
•
•
Retrieve relevant research.
Critically evaluate if study well done.
Integrate into clinical practice.
Understand basic statistical concepts.
 Behavior self-appraisal (3 items, ordinal responses).
• Time spent reading original research each month.
Use of EBP methods in clinic (% patients).
• Time spent accessing PubMed each month.
UWS EIP Questionnaires
 Knowledge Questionnaire:
 20 multiple-choice questions from a pool of 115.
 Scored: 0 to 20.
ATTITUDE
ASSESSMENT of ATTITUDES


Follow the leaders!
“In the initial planning phase, nearly all
grantees (n = 7) conducted faculty surveys…to
assess attitudes, perceived skills, and behaviors
related to research literacy and EBP.” (p.565)
Long, C. R., Ackerman, D. L., Hammerschlag, R., Delagran, L., Peterson, D. H., Berlin, M., & Evans, R. L. (2014).
Faculty development initiatives to advance research literacy and evidence-based practice at CAM academic
institutions. J Altern Complement Med, 20(7), 563-570. doi: 10.1089/acm.2013.0385






Demographics
Attitude/opinion of EIP & research literacy
Training and self-assessed skills
Use of EIP & research literacy in
classroom/clinic supervision
Perceived barriers to EIP & research literacy
Experience conducting research/authoring
publications




No need to reinvent the wheel!
Bastyr University (2010, June 6). Evidence Informed Practice
(EIP) Survey. Retrieved
fromhttp://optimalintegration.org/pdfs/perl/EIP-SurveyTool.pdf
LeFebvre, R. (2011, May 27). Faculty Survey for Evidence Based
Practice (EBP). Retrieved from
http://optimalintegration.org/pdfs/perl/Faculty-survey-forEBP.pdf
Evidence-Based Practice Attitude and Utilization SurvEy
(EBASE) (Leach & Gillham, 2008)
Leach, M. J., & Gillham, D. (2008). Evaluation of the Evidence-Based practice Attitude and utilization SurvEy for
complementary and alternative medicine practitioners. J Eval Clin Pract, 14(5), 792-798. doi: 10.1111/j.13652753.2008.01046.x



Respondents have a positive attitude towards EIP
29% of respondents have no formal training in EIP
Self-assessed research literacy





Moderate overall skills
Relative weakness in interpreting statistics
Agree that students should be research literate; skillbuilding infrequently incorporated into courses
Lack of time and resources are primary barriers
56% of respondents have not authored a peerreviewed publication

Faculty “buy-in” may not be a major concern

Need for basic skill development


Need to integrate EIP/RL across the
curriculum
Need for writing groups to support lessexperienced faculty members
Knowledge & Understanding
Micro-skills
UWS EIP Questionnaires
1.
2.
3.
4.
5.
EBP Overview / questions.
Finding evidence.
Diagnostic studies.
Prognostic studies.
Therapy studies.
6. Harm studies .
7. Applying evidence in practice.
8. Preventive care.
9. Systematic reviews / guidelines.
10.Quality assurance.

Please list 3 areas for improvement relative to
your ability to rapidly find the best available
research evidence, judge if it has adequate
quality, and decide if it is relevant to your
patient care.
Skill Application
A “capstone” assessment
Criteria
Exceeds expectations
(15)
Meets expectations
(10)
Developing
(5)
Does not meet
expectations
(0)
Case history and
exam
Covers all pertinent aspects of
history and exam succinctly.
Discusses key findings in
history or exam and
implications.
Discusses interesting nuances
and areas of uncertainty.
Covers all pertinent aspects of
history and exam succinctly.
Discusses key findings in history or
exam and implications.
Incomplete discussion of
history and exam OR
no discussion of the key
findings
Incomplete discussion of
history and exam AND
no discussion of key
findings
Diagnosis
(tools/techniques,
imaging labs if
available)
Answers the question—how
was the diagnosis made?
Presents results of exam
findings and diagnostic tests
and discusses implications.
Discusses interesting nuances in
results and areas of uncertainty.
Answers the question—how was
the diagnosis made? Presents
results of exam findings and
diagnostic tests and discusses
implications.
No clear discussion of
clinical thought process
leading to diagnosis
OR
no discussion of result of
exams or diagnostic tests.
No clear discussion of
clinical thought process
leading to diagnosis
AND
no discussion of result of
exams or diagnostic tests.
Discussion of EIP
model
Addresses how patient
preferences, clinician
experience, and research were
used – or not used.
Discussion of the limitations of
each aspect in this situation.
Addresses how patient preferences,
clinician experience, and research
were used – or not used in
decisions surrounding patient care.
Missing one of the
elements of the EIP
model.
Missing two or more
elements of the EIP
model.
Discussion of
patient outcome
Discusses outcome of the case,
describes how patient outcomes
were assessed, which outcome
measurement tools were used.
Discusses any uncertainty.
Discuss outcome of the case,
describe how patient outcomes
were assessed, which outcome
measurement tools were used.
Incomplete discussion of
outcome of case OR
no mention of outcome
measures
Incomplete discussion of
outcome of case AND
no mention of outcome
measures






Occurs in EIP IV (11th quarter)
2 hour real time assessment of an RCT
Must do a quality assessment using the ABCDFIX
mnemonic
Must summarize the study in an EIP sandwich
Must answer two general questions about EIP
micro-skills
Must read a results table from the study and pick
out inter and intra group results and judge
whether they were clinically significant,
statistically significant and precise.
UWS EIP STUDENT ASSSSMENT
“HIGH STAKES” EXAMS
EIP
Clin 1
Clin 2
1
2
CAT
CAT
Q4
Q5
Q7
Q8
Info
Lit
EIP
Q 1
CSA 1
Q9
Clinic
CAT
Q9
EIP 3
CAT
10
EIP IV
Capstone
Skills
In the classroom
UWS Faculty Survey Domains
Questions
Notes and power point slides
11 questions
Life long learning/information literacy
skills
6 questions
Current EBP terms/concepts in
diagnosis, treatment, and health risks
2 questions
Quality of evidence
3 questions
Applying EBP course instruction to
clinical practice
3 questions
Self assessment question regarding
overall impact of EBP training.



13 (45%) give a search assignment relevant to
their course material.
4 (16%) require formulation of a search
question (as opposed to a topic or condition).
4 (16%) give feedback on the quality of the
searches.
•
You explicitly remind students how to
interpret quantitative expressions if they
come up in lecture (e.g., sensitivity,
likelihood ratio, NNT, RR).
Never
1-3x/term
>3 x/term
7
12
7


My clinician encourages me to access and
critique the published literature and apply
findings in an evidenced-informed approach to
patient care.
My clinician fosters the application of balance
between published evidence, clinical
experience, and patient values in the process of
evidenced-based practice.
SNAPPs and….?
CEIPE
 Question
Bank
 Journal Club checklists
 EIP course syllabi
 Mentors
 Stocking the pond?
• EIP related topics
• Host institution pays for travel and expenses
• Speaker donates time to CEIPE member
institutions for faculty training (not postgraduate courses)





Keep it simple. Content is some aspect of EBP (a
concept or micro skill) written in a simple
understandable manner.
Keep it short. Preferably single page , two sides.
Longer pieces can be broken down into multiple
bulletins.
Remember your audience. The primary audience
are chiropractic educators (not, EBP specialists)
Small bites, repeated exposures. Periodic (once a
month?)
Teach how to teach. When appropriate, includes
Teaching Tips
Teaching Tip is at the end.
It can be modified to make it college-specific or deleted to
make it flexible enough to disseminate to field
practitioners or students.