TEXAS TECH UNIVERSITY
HEALTH SCIENCES CENTER
Libraries of The Health Sciences
Richard C. Wood, MLS, Executive Director of Libraries
Amarillo: Jan Rice, MLS, AHIP
El Paso: Lillian Carl, MSLS, Milagros de Jesus Rivera, MLS, Rebecca Ruddock, MLS, AHIP, and Linda Schaake, MLS
Lubbock: Peggy Edwards, AMLS, Dawn Kruse, MIS, Judy Orr, MLS, Stephanie Shippey, MLS, Margaret Vugrin, MSLS, AHIP, Robert Neilson, MD and John Griswold, MD
A multi–faceted approach to EBM OSCE instruction
A Collaborative Project with Librarians from the TTUHSC Libraries
and the TTUHSC School of Medicine
Amarillo, El Paso, Lubbock
Te x a s
T h e T T U H S C L ibraries have been collaborative partners with the Departmen t s
o f S u r g e r y a n d Internal Medicine in teaching EBM research skills at three o f t h e
f o u r c a m p u s l o cations (Lubbock, Amarillo, El Paso). Librarians have worked w i t h
t h e f a c u l t y a n d students during their clerkship rotations for the last four ye a r s .
T h r e e d i f f e r e n t complementary approaches have evolved and will be presen t e d .
Lubbock
Amarillo
El Paso
The Amarillo Experience
Library involvement in teaching evidence–based
medicine skills within the required third year
rotations was first incorporated into the MSIII
Surgery rotation in 2004 and it has been evolving
ever since. An EBM station with standardized
patient was introduced into the Surgery OSCEs in
December, 2004. The Internal Medicine Department
introduced an EBM station into the OSCE in 2006.
In order to help prepare students to incorporate
evidence–based practice into their clinical year,
early in the third year all medical students in
Amarillo are instructed by the library faculty in the
basic tenets of evidence-based medicine (EBM) and
in appropriate database searching techniques. The
Surgery rotation builds on these basic skills with
five additional EBM components:
1. As a review of epidemiology and statistics, the
students are required to complete the tutorial
available through Vanderbilt ‘s Eskind Biomedical
Library Evidence-Based Knowledge Portal
(http://www.mc.vanderbilt.edu/biolib/
ebmportal/)
2. A critical appraisal session is conducted early in
each rotation, with selected journal articles based
on the American College of Surgeons Evidence–
Based Reviews in Surgery monthly reading list.
3. Two “EBM Trauma Rounds” are held each
rotation in the SICU of the teaching hospital. This
gives the students an opportunity to take EBM “to
the bedside.” The group discusses the specifics
of a current case and, using PDA resources and a
computer to access library resources, determines
evidence-based responses to management and
treatment questions posed by the Clerkship
Director. The faculty librarian is present during
these sessions to help the students define the
question, determine appropriate resources and
introduce them to additional web resources.
4. Surgery students are required to prepare a forty–
five minute evidence-based case presentation.
The faculty librarian is involved in evaluating
Amarillo Contact Information:
Jan Rice, MLS, AHIP
jan.rice@ttuhsc.edu
the adequacy of each student’s clinical question,
PICO statement and search strategy and by
giving final approval on the Critically Appraised
Topic (CAT) summary for each presentation.
5. EBM skills are evaluated through an EBM
OSCE station during end-of-rotation exams. By
utilizing a standardized patient as a component
of this station, communication and interpersonal
skills are evaluated in addition to the “4 A’s” of
evidence-based medicine: Ask, Acquire, Appraise
and Apply. The standardized patient (SP) comes
to the office seeking a second opinion from the
student doctor; the student interviews the SP
to determine the SP’s questions and concerns;
the student leaves the office to conduct a brief
(fifteen minute) literature search to determine
the best evidence to answer the questions; the
student returns to explain the evidence to the
SP and to suggest an appropriate treatment plan
based on the best evidence. The standardized
patient evaluates the student doctors on their
communication and interpersonal skills; the
faculty librarian evaluates the student doctors on
their communication and interpersonal skills and
also on the EBM skills demonstrated during the
exercise.
The third year Internal Medicine EBM OSCE
measures the same EBM skills (ask, acquire,
appraise and apply) although a standardized
patient is not used during the examination.
Students are given a written clinical scenario
with specific questions to be answered (often a
medications question with patient education and
guidelines/’latest best evidence’ components).
The student has twenty minutes to search
appropriate databases to answer the questions.
The faculty librarian interviews the student to
assess the answers to the questions, the student’s
search process and the appropriateness of the
evidence used to answer the questions. The
faculty librarian evaluates the student on the
EBM skills demonstrated during the exercise.
Harrington Library of the Health Sciences
Texas Tech University Health Sciences Center
1400 Wallace Blvd.
Amarillo, Texas 79106
806.354.5450
The El Paso Experience
Four Core Purposes of EBM searching are:
•
•
•
•
To construct an appropriate clinical question.
To retrieve the current best evidence that answers the question.
To critically appraise the evidence for validity as it is applied to the patient in question.
To apply the evidence using the appropriate clinical filter (diagnosis, treatment, prognosis, etc.) for
the patient.
Teaching EBM – Searching Orientation
• Medical Student Coordinator calls the first week of each period to schedule 1-hour orientation for
new rotation class MS III’s.
• There are usually about 10 students in each rotation.
• The Librarian: explains EBM and how to research EBM databases, distributes handouts, and
reviews the routine for scheduling the practice search session in the library.
Internal Medicine EBM Routine
• Associate Professor (Internal Medicine) assigns each student (usually 1 or 2 students per week) a
patient care topic.
• The student comes to the library to schedule an EBM search session with a librarian.
• The student is instructed to bring the completed PICO question to the session.
• The Librarian: reviews the student’s PICO statement for appropriateness.
• The Librarian: After session, scores practice OSCE.
OSCE Itself
•
•
•
•
A half-day in duration.
The students are split into two groups.
6 stations: 2-chest x-ray, 2 patient exam, and 2 EBM stations.
Usually held in an Internal Medicine Clinic (Patient Examination Room that has a computer with
Internet access).
• Outcomes—pass—fail; EBM is only 25% of total OSCE grade.
• 30 minutes total test – 5 minutes PICO, 15 minutes searching, 5 minutes interpretation &
applicable questions; 5 minutes recording and scoring evaluation form.
El Paso Contact Information:
Delia Montes-Gallo Library of the Health Sciences
Texas Tech University Health Sciences Center
4800 Alberta Ave
Milagros de Jesus Rivera, MLS milagros.de@ttuhsc.edu
El Paso, TX 79905
Rebecca Ruddock, MLS, AHIP rebecca.ruddock@ttuhsc.edu
915.545.6652
Lillian Carl, MSLS lillian.carl@ttuhsc.edu
The Lubbock Experience
Evidence–Based Medicine (EBM) training for the 3rd–year Internal Medicine
Objective Structured Clinical Exam (OSCE)
During the 8–week Internal Medicine rotation,
third–year medical students learn evidence–based
medicine literature searching skills that are
evaluated in an EBM station during the OSCE. A
librarian introduces the objectives during the first
day of the Internal Medicine clerkship orientation.
The objectives state that as a result of this course, participants will:
•Create a focused, well-articulated question
using the PICO model;
•Distinguish between the various point–of–care tools, the types of data retrieved from each tool, and incorporate their use into daily practice;
•Search PubMed effectively;
•Utilize various EBM tools to answer a clinical
test question as a part of the OSCE;
•Discuss search skills and assess the EBM process used in the EBM OSCE station with a library faculty member;
•Integrate the evidence with their clinical
expertise and the patient values.*
Reference librarians meet one–on–one for 2 or
3 hours with each student to teach and practice
the following resources: ACP PIER, DynaMed,
Essential Evidence Plus, First CONSULT, Gold
Rush, National Guideline Clearinghouse, and
PubMed. In preparation for a case presentation
to the Clerkship Director, the student learns
to search the databases by researching a PICO
formulated clinical question they developed
earlier. The EBM station is one of four stations
and each station is 30 minutes long. The EBM
OSCE takes place in an Internal Medicine
Clinic exam room that has a computer with
Internet access. The test question is developed
by the Internal Medicine Clerkship Directors.
The student has 20 minutes to research the
question, then the librarian joins the student to
review search strategies. Using an evaluation
form, jointly developed by Internal Medicine
and library faculty, answers are discussed and
the student’s research skills are graded.
During the third–year Surgery clerkship, the
library collaborates with the Chair of Surgery
teaching EBM searching and critical appraisal
skills of the medical literature in a group
setting. However, this training is not geared
toward OSCE testing.
After 10 Internal Medicine rotations with
approximately ninety students, the librarians
are pleased that the students are doing very
well. They are reasonably comfortable using
the databases presented in the one–on–one
teaching sessions. Student research confidence
levels have increased dramatically with their
newly acquired searching skills. In addition, the
Internal Medicine faculty is satisfied with the
collaborative efforts between their department
and the library.
*Glasziou, Paul. Centre for Evidence-Based Medicine, Downloads. Hp. July, 2003. Powerpoint: Introduction to EBM. University of Oxford and Queensland. Available: http://www.
cebm.net/learning_ebm.asp. 10 Apr. 2005.
Lubbock Contact Information:
Peggy Edwards, AMLS peggy.edwards@ttuhsc.edu
Dawn Kruse, MIS dawn.kruse@ttuhsc.edu
Stephanie Shippey MLS
stephanie.shippey@ttuhsc.edu
Margaret Vugrin, MSLS, AHIP margaret.vugrin@ttuhsc.edu
Preston Smith Library of the Health Sciences
Texas Tech University Health Sciences Center
3601 4th Street, MS7781
Lubbock, Texas 79430
806.743.2200
TEXAS TECH UNIVERSITY
HEALTH SCIENCES CENTER
Libraries of The Health Sciences
A multi–faceted approach to EBM OSCE instruction
A Collaborative Project with Librarians from the TTUHSC Libraries
and the TTUHSC School of Medicine
Amarillo, El Paso, Lubbock
Te x a s
This annotated historical review of research articles
documents librarians’ involvement with both Evidence–Based Medicine (EBM) training
and the Objective Structured Clinical Examination (OSCE).
Dawn Kruse, MSIS, Jan Rice, MLS, AHIP, Stephanie Shippey, MLS, AHIP
October 2008
A multi-faceted approach to EBM OSCE instruction.
This historical review of research articles documents librarians’ involvement with both Evidence-Based
Medicine (EBM) training and the Objective Structured Clinical Examination (OSCE). Jan Rice, MLS, AHIP,
Stephanie Shippey, MLS, AHIP and Dawn Kruse, MSIS collected the research articles. Databases searched were
PubMed, Cinahl Plus with Full Text, Library and Technology Abstracts and Proceedings – Conference, Academic
Search Premier and Google. The search terms used were OSCE, objective structured clinical examination,
librar*, EBM, evidence-based medicine, (education, medical, undergraduate), clinical competence and
educational measurement. A brief annotation follows each citation.
Dawn Kruse, MSIS
(1979). Harden, RM et al. Assessment of clinical competence using an objective structured clinical
examination. Med Educ.;13:41-54.
PMID: 763183
This article describes the OSCE, its usefulness, pros and cons, reliability, practicability, etc.
EBM was not discussed.
Librarians were not involved.
A shorter variation of this article appeared in British Medical Journal:
(1975 Feb). Harden, RM et al. Assessment of clinical competence using an objective structured clinical examination. British Medical Journal;1:447-451.
About OSCE being given to surgical interns.
No EBM discussed or searches done.
No librarian involvement.
(1993). Sloan, DA et al. Use of an objective structured clinical examination (OSCE) to measure improvement in clinical competence during the surgical internship. Surgery.;114:343-351.
PMID: 8342135
(1996 Jan). Woods, SE et al. MEDLINE as a component of the objective structured
clinical examination: the next step in curriculum integration. Bull Med Libr Assoc.;84(1):108-109.
PMID: 8938339
This became a research project to establish a baseline for further educational
instruction in searching techniques for the medical student curriculum.
Over 8 days approximately 120 - 4th year medical students in 15 small groups were tested. The
exam lasted for at least 4 hours, during which time a librarian was assigned to the MEDLINE
station. (A total of 7 librarians participated.) This station was not used as a grade but to gather
statistics on how well the students knew how to find information that they would need to answer
questions in the future. It was determined that one library instruction session is not enough to
equip students with the needed information-retrieval skills.
EBM was discussed.
(1999 Oct). Burrows, SC et al. Evaluating medical student searches of MEDLINE for evidence-based
information: process and application of results. Bull Med Libr Assoc.;87(4):471-6.
PMID: 10550033
Discussed EBM.
Librarians taught a 2 hr. course in the first year of medical school. The students were required by the medical school to do 6 searches after that course, but the searches were not graded by anyone.
In their 3rd year, they did an OSCE and one of the stations was searching MEDLINE. The librarians graded
the results. It was found that the 2 hr. course was not sufficient so they have expanded the curriculum to
include more time with the librarians and more practice searches are now graded by the librarians.
(2000 May). Dunn, K et al. A dissemination model for teaching evidence-based
medicine. Acad Med.;75(5):525-526.
PMID: 10824800
EBM program taught at the New York Chapter of the American College of Physicians.
Program began in 1996 .
Taught to nominated teaching faculty (physicians) and librarians.
EBM resource center at ACP started as a result of these courses.
Gives OSCE definition, use, psychometric qualities and feasibility/practicality.
Librarians taught MEDLINE searching skills (EBM) to residents on a one on one basis. Pre-, post- and 6-month post-surveys were given to see how the residents felt about their abilities.
OSCE was not discussed.
Included in the paper is a Search strategy analysis tool.
Tested EBM and critical appraisal skills through an OSCE.
Did not teach EBM skills before hand; after testing, decided to add it to the curriculum.
Tested 3rd year students.
Does not mention the library or librarians.
(2000 Sep). Toolbox of Assessment Methods. Accreditation Council for Graduate Medical Education
(ACGME), and American Board of Medical Specialties (ABMS). Version 1.1. A product of the Joint
Initiatve.:pg. 7http://www.acgme.org/Outcome/assess/Toolbox.pdf
(2002 Apr). Bradley, DR et al. Real-time, evidence-based medicine instruction: a randomized controlled
trial in a neonatal intensive care unit. J Med Libr Assoc.;90(2):194-201.
PMID: 11999177
(2002 Nov). Fliegel, JE et al. A computer-based OSCE station to measure competence in evidence-based
medicine skills in medical students. Acad Med.;77(11):
1157-1158.
PMID: 12431934
(2004 Mar). Davidson, RA et al. Evaluating Evidence-Based Medicine Skills during a Performance-Based
Examination. Academic Medicine.;79(3):272-275.
PMID: 14985203
PBE – Performance-Based Examination
30-hr. EBM course was taught including, MEDLINE searching. Teacher is unidentified.
At one station the students were asked a clinical EBM question relating to their disease process. Students were asked to develop an appropriate clinical question, perform a Medline search, critically appraise a complete selected article, reach a conclusion to their question, and transmit the information to the patient. Each student’s clinical question, search terms, selected articles, and rationale
were evaluated by faculty question-writers, clinical librarians,and the EBM course director, using a fivepoint Likert scale.
(2004 Summer). Linton, AM et al. Evaluation of evidence-based medicine search skills in the clinical years.
Med Ref Serv Q.;23(2):21-31.
PMID: 15148016
Librarians and physicians taught classes during the first 2 years of medical school. In the 3rd year they decided to offer a workshop to see how well the students had learned what they were taught the first two years. Weaknesses in Boolean and MeSH searching were identified.
OSCE was not discussed.
EBM discussed.
OSCE included no discussion of EBM or Library assistance.
OSCE was not discussed.
Librarians were not used.
Discusses proficiency at critically appraising an article. Found that it can be done well but the information is not transferred to the patient level.
EBM discussed
A systematic review of EBM being taught different ways and whether it improved knowledge, skills, attitude and behavior.
OSCE was not discussed.
No librarians used.
OSCE was not given.
Had formal presentations.
Librarians did teach two sessions on EBM.
For Internal Medicine students, but taught similarly to the surgery clerkship at TTUHSC in Lubbock.
OSCE was not given.
EBM curriculum based on an individual block rotation and designed to integrate and reinforce EBM skills throughout the residency program.
(2004 Jul). Karani, R et al. An unfolding case with a linked Objective Structured Clinical Examination
(OSCE): a curriculum in inpatient geriatric medicine. J Am Geriatr Soc.;52(7):1191-1198.
PMID: 15209661
(2004 Oct). Bergus, G et al. Appraising and applying evidence about a diagnostic test during a
performance-based assessment. BMC Med Educ.;4:20
PMID: 15482600
(2004 Oct). Coomarasamy, A et al. What is the evidence that postgraduate teaching in evidence based
medicine changes anything? A systematic review. BMJ.;329(7473):1017
PMID: 15514348
(2004 Oct). Dorsch, JL et al. Impact of an evidence-based medicine curriculum on
medical students’ attitudes and skills. J Med Libr Assoc.; 92(4):397-406.
PMID: 15494754
(2004 Oct). Thom, D et al. Description and evaluation of an EBM curriculum using a block rotation. BMC
Med Educ.;4:19
PMID: 15476556
2-week, individual EBM rotation for interns.
Taught by Faculty Director and the medical librarian (co-director).
Question obtained from inpatient service rounds was used by the librarian as material for a tutorial on developing search strategies and using high quality web-based EBM resources. Librarian provided input into alternative search strategies and information resources. One page response is developed in the form of a Critically Appraised Topic or CAT. Results are presented to and discussed with
the inpatient team. They estimated that training 13 interns per year requires approximately 70 hours of
librarian time and 200 hours of the faculty Director’s time.
Similar to surgery clerkship done by TTUHSC in Lubbock.
(2005 Jan). Dawes, M et al. Sicily statement on evidence-based practice. BMC Med Educ.;5(1):1
PMID: 15634359
Historical perspective on where EBM and EBP (Evidence Based Practice) originated, what it means, and how it is supposed to be used, and how it benefits the patients.
(2005 Dec). McCluskey, A et al. Providing education on evidence-based practice improved knowledge but
did not change behavior: a before and after study. BMC Med Educ.;5:40
PMID: 16364181
Pre-test before and Post and 8 month surveys after a 2 day workshop run by physicians with
health librarians assisting. The occupational therapists were asked to keep a diary of their searching for
EBM, reading and appraising articles. They also had an assignment that they had to finish after the workshop.
OSCE was not discussed.
Librarians only used during the workshop.
(2006 Jan). Dinkevich, E et al. Effect of a brief intervention on evidence-based medicine skills of pediatric
residents. BMC Med Educ.;6:1
PMID: 16403214
Residents attended 4 – 2hr. sessions over 4 weeks on EBM.
OSCE was not mentioned.
No librarians used.
Overview of two programs that began teaching professionals (physicians and librarians) how to teach EBM and why it is important.
Other skills (critical appraisal and statistical concepts) were also taught.
One program began in 1996 and the other 1999.
OSCE was not mentioned.
Survey of librarians was administered after they returned to their respective campuses to determine if they used what they had learned.
Critical appraisal and research of EBM.
4th year students tested on understanding of EBM through survey.
(2006 Apr). Scherrer, CS et al. An evaluation of a collaborative model for preparing
evidence-based medicine teachers. J Med Libr Assoc.;94(2):159-165.
(2006 Jun). Caspi, O et al. Evidence-based medicine: discrepancy between perceived competence and
actual performance among graduating medical students. Med Teach.;28(4):318-25.
PMID: 16807169
10
OSCE was not mentioned.
No Librarians used.
4th year medical students.
OSCE was given.
EBM searching station.
Searching algorithm for scoring.
3 hr. of trained librarian time but no indication if the librarian taught or advised on setting up the OSCE station.
Librarian didn’t score OSCE.
(2006 Summer). Frohna, JG et al. Development of an evaluation of medical student competence in
evidence-based medicine using a computer-based OSCE station. Teach Learn Med.;18(3):267-272.
PMID: 16776616
(2006 Aug). Green, ML. Evaluating evidence-based practice performance. Evid Based Med.;11(4):99-101.
PMID: 17213114
Discusses problems teaching and evaluating EBM performances.
OSCE was not discussed.
No Librarians mentioned.
Discusses Miller’s pyramid of clinical competence in the article below:
(2000 Nov). Vleuten,C. Validity of final examinations in undergraduate medical training. BMJ.;321:1217-1219.
(2006 Sep). Frank, C. Evidence based checklists for objective structured clinical examinations.
BMJ.;333(7567):546-548.
PMID: 16960213
Discusses OSCE and the checklists used to evaluate the student’s performances.
Does not include searching databases or EBM.
Librarians taught PubMed searching using MeSH tutorial links and PowerPoints. Skills included keyword and MeSH searching.
Taught in small groups until the class size became so large that they had to teach an entire class year at one time.
Determine that small groups retained the skills better over time.
Chose OSCE to evaluate PubMed searching skills because of bandwidth problems.
3rd and 4th year medical students are given an OSCE twice a year to include PubMed searching.
The students were identified by a student number on their OSCE answer sheets.
Librarians then graded the PubMed searching on those answer sheets.
Did not have face-to-face evaluation or discussion with the students.
(2007 May). Myers, G et al. Comparison of training interventions for PubMed search skills amongst 3rd
and 4th year medical students. Presented at 4th International Evidence-Based Library and Information
Practice Conference in Durham, NC.
http://www.eblip4.unc.edu/papers/Myers.pdf
(2007 Jul). Ryce, A et al. A partnership in teaching evidence-based medicine to interns at the University of
Washington Medical Center. J Med Libr Assoc.;95(3):283-286.
11
OSCE was not discussed.
8-week EBM course taught for a total of 24 hours of instruction.
Taught by faculty, chief residents, and librarians in the Library’s computer lab.
Each session was 3 hours long.
Librarians also created a website for accessing all of the course materials.
OSSE – Objective Structured Searching Evaluation
Physicians and one librarian taught the courses.
Two physicians grade the OSSE.
Interesting ideas about how to grade OSSE are discussed.
EBM discussed.
(2007 Dec). Stark,R et al. Real-time EBM: from bed board to keyboard and back. J Gen Intern
Med.;22(12):1656-60.
PMID: 17922170
(2008 Feb). Franzese, CB. Pilot study of an Objective Structured Clinical Examination (“the Six Pack”) for
evaluating clinical competencies. Otolarynogol Head Neck Surg.;138(2):143-148.
PMID: 18241705
Discusses OSCE and how performed.
Discusses EBM being tested, which was modeled after the Fresno EBM test. It does not tell how the students were taught EBM, only that they didn’t receive adequate instruction. It does not include the
library or librarians in any of the discussion.
If you are interested in the validation of the Fresno test, see:
Ramos, KD et al. (2003 Feb). Validation of the Fresno test of competence in evidence based
medicine. BMJ.;326:319-321.
PMID: 12574047
If you would like to see the Fresno test rubric visit this site: www.fammed.washington.edu/ebp/
media/GradingRubricTestA4_23_.doc
(2008 Fall). Kaneshiro, KN et all. Use of an audience response system in an evidence-based minicurriculum. Med Ref Serv Q.;27(3):284-301.
EBM workshops taught by physicians and librarians together.
OSCE was not discussed.
Administered EBM practicum final exam.
Use of Audience Response System (ARS) during workshop.
The ARS helps the teachers to collect data, promote active learning and to have real time feedback.
Librarians working along side physicians.
Librarians producing library portals for each of the medical rotations that showcase different resources.
Mentions EBM but not who teaches it.
No OSCE, but they have Five Minute Challenges (5MCs).
Level of evidence is taught in the 1st and 2nd years because of how the levels were misunderstood in the 3rd and 4th years.
(2008 Fall). Skhal, KJ. A full revolution: Offering 360 degree library services to clinical clerkship students.
Med Ref Serv Q.;27(3):249-259.
12
Texas Tech University Health Sciences Center
Preston Smith Library of the Health Sciences
3601 4th Street, MS7781
Lubbock, Texas 79430
806.743.2200
Lubbock Contacts:
Dawn Kruse, MIS dawn.kruse@ttuhsc.edu
Stephanie Shippey MLS
stephanie.shippey@ttuhsc.edu
Harrington Library of the Health Sciences
1400 Wallace Blvd.
Amarillo, Texas 79106
806.354.5450
Amarillo Contact:
Jan Rice, MLS, AHIP
jan.rice@ttuhsc.edu
13
TEXAS TECH UNIVERSITY
HEALTH SCIENCES CENTER
Libraries of The Health Sciences
Richard C. Wood, MLS, Executive Director of Libraries
Amarillo:
Jan Rice, MLS, AHIP
14
El Paso:
Lillian Carl, MSLS, Milagros de Jesus Rivera, MLS, Linda Schaake, MLS, Rebecca Ruddock, MLS, AHIP
A multi–faceted approach to EBM OSCE instruction
A Collaborative Project with Librarians from the TTHUSC Libraries and the TTUHSC School of Medicine
Amarillo, El Paso, Lubbock
Te x a s
Lubbock:
Peggy Edwards, AMLS, Dawn Kruse, MSIS, Judy Orr, MLS, Stephanie Shippey, MLS, Margaret Vugrin, MSLS, Robert Neilson, MD & John Griswold, MD
The TTUHSC Survey
SURVEY: In order to determine actual numbers of libraries teaching EBM and then grading OSCE EBM
performance, the Lubbock Library Faculty sent out a survey to three Library listservs (MEDLIB-L, JISCmail
and NNLMSCR) for one month during the summer of 2008.
RESULTS: The survey found that 18 of the 33 respondents
were involved in teaching EBM skills to medical students.
In most contexts (13 respondents), the skills were taught as
part of an integrated departmental rotation. Only one library
respondent was involved in evaluating the EBM skills of the
student through an Objective Structured Clinical Exam (OSCE).
CONCLUSIONS: We determined, based upon the small number
of responses, that few libraries are teaching EBM skills to Medical
Students; only two institutions, TTUHSC Libraries and Indiana
University Medical Libraries, are involved in teaching and evaluating
EBM skills for the OSCE.*
Which of the following EBM skills
are evaluated?
Formulating a clinical question
Defining a PICO statement
Selecting an appropriate database/resource
Developing an adequate search strategy
Choosing appropriate info to answer the clinical question
Appraising resource using critical appraisal techniques
Applying the evidence to the patient
Communicating with a standardized patient
Other
# of
Responses
6
4
5
5
4
2
2
0
1
s:
stion
g que ased r
in
w
ollo
ce-B t you
viden nts a
the f
de
gE
nd to
achin al stu
respo volved dinyteear medic
e
s
a
in
Ple
thir
ary
ical
the
Libr
med
2)
your ills to
tion
year
1. Isdicine sk?
ques
third
o
t
Me tution , proceed to
t
taugh
insties (If yes
EBM
a. Y
xt is ?
contsetitutionn
d in
b. No
t
a
h
tion
w our in ntatio e
istertemental
l rota
e
rs
ithin
admin
par
d
E
e
e
C
2. Wdents at gyeneral olerictive coeupartmenta
d
s
S
r
a
ce-b ed
BM O othe
stu s part of -alone e rt of a d
viden uctur
the E with
a. A a stand rated pa
6. Is junctions?
ting e ve Str
b. As an integ
valua(Objectitations?
on E exam
e
c
s
in
A
E ol ro
c.
by a
OSCes
er
olved OSC
rved
d. Oth
y inv h an l scho
a. Y
obse
ibrar roug dica
b. No
the L ills th g me n 4)
OSCE
M
:
B
E
3. Isdicine skm) durinto questio
E
C
S
the
he O
me ical Exa, proceed
7. Isarian?
the
s to t
Clines (If yes
libr
ce in
pplie
ing asent
a. Y
rmanbrarian?
all a. Yes
w
k
o
r
o
ll
a
re
perfo
fo nt is p sent
a li
(m
No
b. N
’s
e
.
y
t
?
h
b
b
n
t
d
re
e
e
e
of ed pati ent is p
tud
ated
aluat
hich
the s valu
re ev
4. Wstandardizrdized pati
8. Is OSCE e
kills a
the
a. A standa
EBMes
BM s
d for E?
E
o
c.
N
g
.
b
llowe OSC
a. Y
er:
owin
e is athe EBMutes ome
c. Oth
e foll
. No
m
b
h
n
ti
t
o
f
in
to
h
e
ti
uc mplet b. 30 m a take-h
on
ho
rce
ues
ti
u
q
ic
m
a
o
l
h
s
a
w
rm
o
is is
5. W apply) g a clinic tement base/re gy
9. Hdoent to cs
d info
d. Th
l
thatormulatin PICO stapriate dataarch strater foregroun
raisa
stu minute
a
l app
e
F
o
a.
und
a. 15 inutes
appro uate s
ritica
fining
ate c
m
b. Delecting an an adeq te backgro
ropri
tient 60 signment
c. Se veloping ppropria estion d on app
ed pa
as
iz
e
rd
e
a
u
r:
s
a
D
d. lecting linical q rce ba
nd
ta
Othe
s
.
e
a
u
t
e. Se er the c the reso
with
atien
the p municate
answpraising
to
e
c
f. Ap niques e eviden ity to com
tech plying th ent’s abil
g. Ap ical stud
d
e
h. M er:
i. Oth
Within what context is EBM taught to
medical students at your institution?
# of
Responses
As part of general orientation
As a stand-alone elective course
As an integrated part of a departmental rotation
Other
1
3
13
5
*The large graphs are answers to selected
survey questions listed to the left.
15
The TTUHSC
Amarillo Experience
min.
Use of Standardized Patient in the EBM MSIII Surgery OSCE 30
station
Librarian
Student
supplements training of
Standardized Patient (SP)
for the EBM OSCE ensuring that
SP understands clinical scenario
is present as observer/evaluator
reads clinical scenario
meets with SP, who has come for
a 2nd opinion
interviews SP to determine concerns
leaves to formulate clinical question
based on PICO format
researches & locates evidence–based
articles for SP’s
question
returns and
explains evidence
to SP
leaves consultation room
interviews student
using EBM evaluation form
provides student feedback
on d performance
16
librarian and SP complete
“Communications Skills Checklist”
leaves consultation room
The TTUHSC
El Paso & Lubbock Experience
ACQUIRE
ASK
The Literature Search
• Select the resource(s)
• Formulate the strategy
• Conduct the research
The Well–Built Clinical Question
PICO
• Patient
• Intervention
• Comparison
• Outcome
Therapy, Diagnosis, Etiology
Prognosis, Costs / Economics
PATIENT
APPRAISE
Review the Search Results
Evaluate Evidence
• Validity
• Currency
• Applicability
• Useful / usable
• Level of Evidence
ASSESS
RESULTS
APPLY RESULTS
• Integrate the evidence
with Clinical Expertise
• Discuss the evidence
with the Patient
17
SCHOOL OF MEDICINE
Department of Surgery - Amarillo
OBJECTIVE STRUCTURED CLINICAL EXAMINATION (OSCE)
Amarillo Case #27
(Modified 03/06)
Patient Evaluation of Student Performance
Student Name: _______________________________
Date: _______________________________________
1
2
3
4
5
6
7
8
9
10
The Student:
Introduced self in a friendly manner.
Kept good eye contact (note taking is o.k.).
Used speech and language which you understood.
Asked open ended questions in a logical sequence.
Listened well and restated your comments to ensure
understanding.
Was respectful to you, your gender and culture.
Made you feel comfortable with both their verbal
and nonverbal communication.
Allowed you to ask questions.
Responded to your questions respectfully and
adequately.
Provided you with a definite opinion at the end of the
visit.
Overall Communication and Interaction (check one box)
Poor >
>
>
>
>
>
>
0
1/2
1
2
Did not
perform
(0)
Partially
perform
(1/2)
Performs
well
(1)
Superior
Adapted from: UIC-COM Basic Interpersonal and Communication Skills Rating Scale.
The Clinical Performance Center. Dept. of Medical Education. University of Illinois at Chicago, College of Medicine.
18
19
5 pts.
5 pts.
10 pt.
15 pts.
18 pts.
20 pts.
Student’s search strategy:
Adapted from: Ramos, KD, Schafer S and Tracz SM. “Validation of the Fresno test of competence in evidence based medicine.” BMJ 2003; 326(7384):319-21.
Detail the search strategy used:
1. 3 expanded concepts
EX: inguinal hernia AND (surgery OR surgical)
plus correct Boolean
AND (non-surgical OR nonoperative OR
“watchful waiting” OR observation)
2. Appropriate strategy
Inguinal hernia / groin hernia
(DynaMed/FirstConsult)
2. 3 simple concepts plus
EX: inguinal hernia AND surgery AND
correct Boolean
“watchful waiting”
3. 2 concepts plus correct
EX: inguinal hernia AND “watchful waiting”
Boolean
4. 2 or 3 concepts, no
EX: inguinal hernia watchful waiting
Boolean
5. Correct use of limits
RCT/meta-analysis/Trials
Ovid EBMR
PubMed
DynaMed, FirstConsult
MD Consult
Book
Google
Yes
No
Student formulated a PICO statement before starting the literature search.
PICO elements:
ACQUIRE
Which database was used?
Yes
No
ASK
Student formulated a Clinical Question before starting the literature search.
Clinical Question:
October, 2008
TTUHSC Department of Surgery, Amarillo
Objective Structured Clinical Examination (OSCE)
EBM Evaluation
10 pts.
10 pts.
8 pts.
6 pts.
4 pts
0 pts.
10 pts.
0 pts.
10pts.
0 pts.
Student: ________________________
Points earned: _____________
PICO Worksheet
Name: _______________________________ Date: ________
Clinical Question:
Study category: therapy diagnosis
harm prognosis
PICO
Patient, population, or problem being addressed
What are the characteristics of the patient or population?
What is the condition or disease?
Intervention being considered which could include:
exposure, diagnostic test, prognostic factor, therapy, patient perception
20
What do you want to do with this patient? Treat, diagnose, observe?
Comparison intervention or exposure
Relevant most often when looking at therapy questions.
What is the alternative to the intervention? Placebo, different drug, surgery?
Outcomes of interest
What are the relevant clinical outcomes of interest to you and your patient?
Morbidity, death, complications?
Focused, Well-Built Question:
Reformat by: Peggy Edwards, TTUHSC Preston Smith Library, Lubbock, Texas, May, 2007
University of Washington Health Sciences Libraries, (1994-2008). Construct well-built clinical questions using PICO. Retrieved May, 2007, from
HealthLinks, University of Washington; Toolkits; Public Health; EBP: Evidence-Base Practice; Learn About EBP. Web site: http://healthlinks.washington.edu/ebp/pico.html Sackett, David L, Richardson, W. Scott, Rosenberg, W., & Haynes, R. Brian (1997). Evidence-based medicine: how to practice and teach EBM.
Edinburgh: Churchill Livingstone.
21
SCHOOL OF MEDICINE
Department of Surgery - Amarillo
OBJECTIVE STRUCTURED CLINICAL EXAMINATION (OSCE)
Amarillo Case #27
(Modified 03/06)
Patient Evaluation of Student Performance
Student Name: _______________________________
Date: _______________________________________
1
2
3
4
5
6
7
8
9
10
The Student:
Introduced self in a friendly manner.
Kept good eye contact (note taking is o.k.).
Used speech and language which you understood.
Asked open ended questions in a logical sequence.
Listened well and restated your comments to ensure
understanding.
Was respectful to you, your gender and culture.
Made you feel comfortable with both their verbal
and nonverbal communication.
Allowed you to ask questions.
Responded to your questions respectfully and
adequately.
Provided you with a definite opinion at the end of the
visit.
Overall Communication and Interaction (check one box)
Poor >
>
>
>
>
>
>
0
1/2
1
2
Did not
perform
(0)
Partially
perform
(1/2)
Performs
well
(1)
Superior
Adapted from: UIC-COM Basic Interpersonal and Communication Skills Rating Scale.
The Clinical Performance Center. Dept. of Medical Education. University of Illinois at Chicago, College of Medicine.
22
A Collaborative Project with
Librarians from the TTUHSC Libraries
and
faculty in the
TTUHSC School of Medicine.
Amarillo, El Paso, Lubbock
Texas
TEXAS TECH UNIVERSITY
HEALTH SCIENCES CENTER
Libraries of The Health Sciences
TEXAS TECH UNIVERSITY
HEALTH SCIENCES CENTER
Libraries of The Health Sciences
HEALTH SCIENCES CENTER
Libraries of The Health Sciences
TEXAS TECH UNIVERSITY
HEALTH SCIENCES CENTER
Libraries of The Health Sciences
TEXAS TECH UNIVERSITY
Margaret Vugrin @2008
EBM OSCE Instruction
HEALTH SCIENCES CENTER
Libraries of The Health Sciences
EBM OSCE Instruction
EBM OSCE Instruction
EBM OSCE Instruction
EBM OSCE Instruction
TEXAS TECH UNIVERSITY
HEALTH SCIENCES CENTER
Libraries of The Health Sciences
HEALTH SCIENCES CENTER
Libraries of The Health Sciences
TEXAS TECH UNIVERSITY
TEXAS TECH UNIVERSITY
EBM OSCE Instruction
HEALTH SCIENCES CENTER
Libraries of The Health Sciences
EBM OSCE Instruction
EBM OSCE Instruction
TEXAS TECH UNIVERSITY
23
Dawn Kruse@2008
24