e-Learning
Evidence-based Medicine
Katrien Oude Rengerink
Academic Medical Center
Dept. of Obstetrics and Gynaecology
The Netherlands
EBayesMet conference October 2011
Contents




What is Evidence-based Medicine?
Why e-learning modules EBM?
e-learning basic principles EBM
e-learning modules Teach-the-Teacher EBM
What is EBM?
Dave Sackett BMJ 1996: “Good doctors use individual
clinical expertise, the best available external evidence as
well as patient preferences, and neither alone is
enough.”
Reduction Number Needed to Read
 120+ journals scanned
• 50,000 articles
 Is it valid? (<5%)
• Intervention: RCT
• Prognosis: inception cohort
• Etc
 Is it relevant?
• 6-12 GPs & specialists asked:
Relevant? Newsworthy?
 < 0.5% selected
www.evidence-basedmedicine.com
Number Needed to Read
is 200+
Why EBM?
 Exponential growth knowledge
 Integration research into clinical practice slow
 evidence summaries important
 skills for knowledge translation important
5 steps of EBM
Theory of the Stork vs
Theory of Sexual Reproduction
R2 0.49, p<0.05
Evidence supporting Theory of Stork





Storks exist
Unexplained features fetal development
Sexual intercourse without delivery
Correlation birth rate - stork population
Storks not seen in hospitals: apparently medical
interventions may replace storks
 EBM: hierarchy in evidence sources
Organisation of evidence
Parachute use to prevent death:
systematic review of RCTs
 Parachutes used to prevent death and major injury due
to gravitational challenge
 Free fall studies: no 100% mortality
 No RCT: basis use purely observational. Apparent
efficacy potentially explained by a ‘healthy’ cohort effect.
 No evidence  do not use intervention
 Some effects not need RCT
Why e-learning EBM?
 Clinical professionals value EBM important, but slow
integration and application in practice
 Huge variation EBM content and education Europe
 Interactive, clinically integrated education best way
learning and applying EBM
 Current available courses useful, but:
- minimum integration clinical practice
- low flexibility
Leonardo I
An European e-learning course in
Evidence Based Medicine
Partners
 UK







• University of Birmingham/Birmingham Womens’ Hospital
• CASP International
Germany - Agency for Quality in Medicine
Poland - CASPolska
Spain - CASPe
Hungary - TUDOR
Italy - Universita Cattolica del Sacro Cuore
Switserland - Basel Institute for Clinical Epidemiology
The Netherlands - AMC
Independent steering commitee
Aim
 Familiarise participants with EBM basics to help
incorporate evidence from systematic reviews on
therapeutic interventions into daily clinical practice
 Harmonise EBM teaching across Europe
 Short, individual e-learning sessions
 On-the-job training
Course content
1. Defining clinical questions
2. Searching the evidence
3. Appraisal of the systematic review
a. Validity of RCTs
b. relative risk and 2x2 table
4. Applying evidence to patient
5. Implementation of evidence into practice
Instruments for evaluating education in
EBM
4 EBP domains:
1. Knowledge
2. Skills
3. Attitude
4. Behavior
•
•
•
enacting EBP steps in practice
permorning evidence-based clinical manoevers
affecting patient outcomes
Course evaluation
 101 participants from Switserland, Germany,
Hungary, Spain, UK
 Pre- and postcourse assessment
• Knowledge/skills: modelled Berlin Questionnaire
• Attitudes: questionnaire by Taylor L. et al.
Results
 Across all countries, knowledge scores improved from
pre- to post-course for all five modules (p<0.001) (paired
Wilcoxon signed ranks test)
 After completing course participants felt more confident
that they can assess research evidence
 E-learning at least equally effective as regular
teaching
Leonardo II
Evidence Based Medicine: Training the
Trainers across the healthcare sector
Background
 During Leonardo I & Sicily EBM conference:
need support teaching EBM in practice
• low level EBM teaching in clinical practice possibly
due to lack of confidence to teach EBM in workplace
 Complementary to existing EBM programmes:
exploit learning opportunities in clinical setting
 Leonardo II
Partners
 England (lead):
• University of Birmingham/Birmingham Women’s Hospital
• J&B Associates






Italy – Universita Cattolica del Sacro Cuore
Poland - CASPolska
Hungary - TUDOR
Switserland - Basel institute of clinical epidemiology
Netherlands – AMC
Germany – AzQ
Aim
 Train trainers to teach EBM through on-the-job training,
by demonstration of applied EBM, to encourage clinically
relevant teaching of EBM in post-graduate education
 Develop European standard in EBM education
 Target: clinical teachers who teach EBM in clinical
practice to postgraduates
Course contents
 Based on teaching opportunities modules
developed for 6 clinical settings:
• Learner-centered (e-learning)
• Problem-based
• Video clip incorporated
• Applicable in Europe
Teaching opportunities in clinical
practice
5 EBM steps
Formulating
questions
Searching
for
evidence
Critical
appraisal
Integrating
evidence
with clinical
scenario
Bringing
change to
practice
Ward round
+++
(+)
+
++
+
Journal club
+
+
+++
+
+
Clinical teaching and
assessment
++
(+)
++
+++
+
Outpatient clinic
+++
(+)
++
++
+
Formal clinical
meeting
+++
(+)
+
++
+
Audit
++
(+)
++
+
+++
Learning
Opportunities
Example: Teaching and learning EBM during
ward rounds
Outline:
 Learning objectives
 Pre-requisites to undertake the module
 Learning opportunities
 EBM steps
 Example
 Video clip (3 min)
 Teaching and learning lesson
 Conclusion
Course evaluation
 5 countries: about 10 clinical teachers who
teach EBM to postgraduates per country piloted
 Pre- and postcourse assessment using
‘Basel questionnaire’
 e-learning course on how to teach EBM in
various settings is feasible and effective.
Barriers EBM teaching
 120 clinical EBM teachers from 11 countries completed
questionnaire regarding barriers for EBM teaching
 Barriers:
• lack of time in busy practice
• lack of requirements for EBM skills
• pyramid hierarchy in health care management structure.
 Teaching EBM in clinical settings is most successful where EBM
principles are culturally embedded and form part and parcel of
everyday clinical decisions and medical practice.
Availability
Free available in:




English
German
French
Polish
 Hungarian
Accessible through: www.ebm-unity.org
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Sackett, BMJ 1996 EBM what is it and what isn’t it?
Höfer T, Przyrembel H, Verleger S. New evidence for the Theory of the Stork. Paediatric and perinatal
epidemiology. 2004(18):88-92.
Smith GCS, Pell JP. Parachute use to prevent dath and major trauma related to gravitational challenge:
systematic review of randomised controlled trials. BMJ 2003.
Khan AT, Mehr MN, Gaynor AM, Bowcock M, Khan KS. Is general inpatient obstetrics and gynaecology
evidence-based? A survey of practice with critical review of methodological issues. BMC Women’s Health
2006(6):5.
Khan KS, Coomarasamy A. A hierarchy of effective teaching and learning to acquire competence in evidencebased medicine. BMC Med Educ 2006:15(6):59.
Coppus SFPJ, Emperanza JI, hadley J et al. A clinically integrated curriculum in Evidence-based medicine for
just-in-time learning through on-the-job training: The EU-EBM project. BMC Medical Education 2007(7):46.
Shaneyfelt T, Baum KD. Instruments for evaluating education in evidence-based practice: a systematic review.
JAMA 2006;296(9):1116-1127.
Kulier R, Hadley J, Weinbrenner S et al. Harmonising Evidence-based medicine teaching: a study of the
outomes of e-learning in five European countries. BMC Medical Education 2008(8):27.
Taranganitam S, Barnfield G et al. Teaching trainers to incorporate evidencebased medicine (EBM) teaching in clinical practice: the EU-EBM project.
Oude Rengerink K, Thangaratinam S. How can we teach EBM in clinical practice? An analysis of barriers to
implementation of on-the-job EBM teaching and learning.
Thanks to
Leonardo I
Regina Kulier
Julie Hadley
Susanne Weinbrenner
Berrit Meyerrose
Tamas Decsi
Andrea Horvath
Eva Nagy
Jose Emparanza
Sjors Coppus
Theodoris Arvantis
Amanda Burls
Juan Cabello
Marcin Kaczor
Gianni Zanrei
Karen Pierer
Katarzyna Stawiarz
Regina Kunz
Ben Willem Mol
Khalid Khan
Steering committee
Leonardo II
Shakila Thangaratinam
Gemma Barnfield
Theodoros Arvanitis
Susanne Weinbrenner
Khalid Khan
Berit Meyerrose
Andrea Horvath
Gianni Zanrei
Regina Kunz
Katja Suter
Jacek Walczak
Anna Kaleta
Javier Zamora
Harry Gee
Ben Willem Mol
Maciej Nogas
Bernard Burnand
Chantal Arditi
Katrien Oude Rengerink:
K.OudeRengerink@amc.uva.nl
www.ebm-unity.org