EM Milestones and
Resource Development
CORD 2013
EM Milestones and
Resource Development
• Putting Competency Based Education into
Practice – Doug
• Intro to the EM Milestones Wiki – Joint
Milestones Task Force
• When you get home, get on the Wiki and see
what works for you
• Not a finished product but a starting point
Douglas Char, MD FACEP FAAEM
CORD Academic Assembly
March 2013
 We are at a tipping point in competency-based
medical education (CBME) — and it’s only taken 40
years since the competency conversation first
appeared in the medical literature!
 In case you were not aboard the CBME train as it left
the station, this concept is an integrated framework
for education, in which specific behavioral outcomes
(competencies) drive both medical school curricula
and individual advancement, rather than the current
driving forces of time (four years of medical school)
and process (clerkships of specific length).
 Carol Aschenbrener – Chief Medical Officer, AAMC
http://wingofzock.org/2012/09/25/competency-based-medical-education-the-time-is-now/
 Traditional medical education presumes that all students are
ready to graduate once they have completed a set number of
years of study and passed the required assessments,
 There is a growing interest in tailoring the length as well as
the content of medical education to individual aptitudes.
 “People learn in different ways and at different speeds,”
 “As early as 1932, reports emerged saying that it is not
enough to stuff students’ heads with information
 Stakeholder no longer accepting residents as independent
actors, they expect physicians to function as leaders and
participants in team-oriented care.
https://www.aamc.org/newsroom/reporter/april11/184286/competency-based_medical_education.html
 It is a curricular concept designed to provide the skills
physicians need, rather than solely a large, prefabricated
collection of knowledge.
 AThe
medical
school or residency
program
using scientific
competency-based
approach
still includes
competency-based
medical
education
defines
a set of
knowledge, but in the
broader
context of
a physician’s
skills or competencies
based
societal and patient
tasks
as aon
healer.
needs, such as medical knowledge, patient care, or
communications approaches, and then develops ways to
teach that content across a range of courses and settings.
https://www.aamc.org/newsroom/reporter/april11/184286/competency-based_medical_education.html
 We are wrestling with it just like everyone else. The
challenge is not so much accepting the concept,
which we think is great, but figuring out how to
make it work. Where do we teach? How do we
evaluate performance? How do we remediate
students who have not met requirements?” Thomas
Pellegrino - EVMS
 How to define competencies, and how to assess
performance are perhaps the two most significant
concerns about competency-based medical
education. Peter Katsufrakis – NBME
 “we’ve been wrestling with this question for
decades,” M. Brownell Anderson - AAMC
 Implementing competency-based training in
postgraduate medical education poses many
challenges. Making this transition requires change
at virtually all levels of postgraduate training.
 Key components of this change include;
 Development of valid and reliable assessment tools such




as work-based assessment using direct observation,
Frequent formative feedback
Learner self-directed assessment;
Active involvement of the learner in the educational
process;
Intensive faculty development that addresses curricular
design and the assessment of competency
Iobst. Teach Med 2010; 32: 651–656
Bloom’s Taxonomy 1956
Anderson’s revision 2000
Cognitive (Knowledge)
Affective (Attitude)
Psychomotor (Skills)
Remembering: can the student
define, duplicate, list, memorize, recall, repeat,
recall or remember the information? reproduce state
Understanding: can the student
explain ideas or concepts?
classify, describe, discuss, explain, identify, locate,
recognize, report, select, translate, paraphrase
Applying: can the student use the
information in a new way?
choose, demonstrate, dramatize, employ, illustrate,
interpret, operate, schedule, sketch, solve, use, write.
Analyzing: can the student
appraise, compare, contrast, criticize, differentiate,
distinguish between the different
discriminate, distinguish, examine, experiment,
parts?
question, test.
Evaluating: can the student justify a appraise, argue, defend, judge, select, support, value,
stand or decision?
evaluate
Creating: can the student create new assemble, construct, create, design, develop, formulate,
product or point of view?
write
• Final milestones will provide meaningful
data on the performance that graduates
must achieve before entering unsupervised
practice (graduate)
• Initial milestones for entering residents will add a
performance- based vocabulary to conversations with
medical schools about graduates’ preparedness for
supervised practice (residency)
Norcini BMJ 2003:326(5):753-755
Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990:S63-7.
 Many people argue that this statement is incorrect and
that the curriculum is the key in any clinical course.
 In reality, students feel overloaded by work and respond
by studying only for the parts of the course that are
assessed.
 To promote learning, assessment should be educational
and formative—students should learn from tests and
receive feedback on which to build their knowledge and
skills
 Pragmatically, assessment is the most appropriate
engine on which to harness the curriculum.
V Wass. Lancet 2001; 357: 945–49
 Trained Observers
 Common understanding of expectations
 Sensitive “eye” to key elements
 Consistent evaluation of a given level of performance
 Minimum number of quality observations
 Assessment based on 7-9 observations felt to be valid
and reliable
 Interpreter/Synthesizer Experts
 Clinical Competency Committee
 Numbers produce range restriction
 Narratives are easily understood by faculty and
produce data without range restriction
 Natural to how we teach and provide feedback
 Goalis to create verbal pictures
 4 cm laceration right arm
vs
 Danger here is that rather than engaging a total
practicum to which other forms of learning
discourse bring their insights, a limited
professional education is based upon an
inappropriate epistemology of competency
 Tendency to limit the reflection, intuition,
experience and higher order competence necessary
for expert, holistic or well developed practice
 Martin Talbot, Med Educ 2004; 38: 587–592
 If your are feeling overwhelmed and confused by all
this new jargon – you are not alone
 Your faculty are looking to you for answers!
 Nobody has all the answers so stop waiting for the
Holy Grail?
 Better to join the legion of PDs working to define it
 There is no way to sort out the milestones without
getting “dirty” – expect to make mistakes
 Assessment is suppose to drive curriculum (this is a
game changer)
 Resistance if futile, give in and drink the kool aid
 Reduce your stress, it’s going to happen!
Intro to EM Milestones Wiki
Kevin Biese, MD, MAT
EM Milestones Wiki
JMTF – Work so far
•
•
•
•
•
Intro to the Wiki – Christina
Care Based Milestones – Moshe
Systems Based Milestones – Mary Jo
Procedural Milestones – Jenna
Milestones Workbook - Rodney
Intro to Wiki Resources
Christina Shenvi, MD, PhD
WiiFM
But ultimately, check back to…
Accessing the wiki
• Emmilestones.pbworks.org
• You do not need a login
• We hope this will be a helpful resource
JMTF – Care-Based Milestones
Moshe Weizberg, MD, FACEP
THANK YOU
• Committee
• Nestor Rodriguez
• Jason Seamon
EVALUATIONS
• How do you want to divide your milestones
• What do you want your evaluation questions
to look like
• How do you label each level
• Where do you want your comment boxes
HOW DO YOU WANT TO DIVIDE YOUR
MILESTONES
•
•
•
•
All milestones on every evaluation
Divide among various rotations
Divide among faculty
Divide by day of the month
• Hit all core competencies
• CCCs evaluate all milestones based on evals
PRO’S AND CON’S
PRO’s
CON’s
All milestones on every
evaluation
Lots of data
Most information
Very long evaluations
Faculty compliance?
Realistic?
Divide among various
rotations
Select milestones that
relate to that rotation
Have to match up the right
milestones to the right
rotations/evaluators
Divide among faculty
Different faculty members
become experts in
evaluating their milestones
Those faculty won’t have
input on other milestones
Divide by day of the month Hit every milestone every
block
Avoid lengthy evals
Logistically challenging
Hit or miss
WHAT DO YOU WANT YOUR EVALS TO
LOOK LIKE
• Yes/No/N/A
• Mix up elements from various milestones
• Mirror the milestone pages
PRO’S AND CON’S
PRO’s
CON’s
Yes/No/N/A
More accurate
Better info on each
element
Many more questions
Harder to feed into CCCs
Mix up elements
from various
milestones
Probably most accurate
Very difficult to set up
Very difficult to analyze
May be confusing for
faculty
Mirror the
milestone pages
Easier
Less questions
Feeds cleanly into CCCs
Resident hits some elements
and not others
Very wordy for faculty to read
?less accurate
LESSONS LEARNED
•
•
•
•
Many faculty gave all residents level 5
Faculty bias based on label of each level (PGY)
Faculty education
Resident education
JMTF - Systems-Based
Milestones
Mary Jo Wagner, MD
JMTF: Procedural Milestones
Jenna Fredette, MD
Christiana Health Care System
jfredette@christianacare.org
Progress So Far
• Early Stages: Emails exchanged, surveys
completed, one conference call  good
position for new members!
• None the less we have created an agenda for
moving forward in 2013
Challenges with Procedural Milestones
• Achieving a Level 4/5 in a Procedural
Milestone does not necessarily prove
procedural proficiency
Challenges with Procedural Milestones
• Milestones addresses the following
–
–
–
–
–
–
General Approach to Procedures
Airway Management
Vascular Access
Focused Ultrasound
Wound Management
Anesthesia/Sedation
• We perform and require proficiency in many
more areas than this
Current Agenda Moving Forward
• Collect institutional checklists and come up
with set of standardized procedural checklists
• Add milestone language to the checklists
when appropriate
• Determine the best way to evaluate each of
the milestones (?procedural shift cards,
SDOTs, SIM labs…)
• Welcome new ideas and input!!
• JMTF Procedural Milestones Wiki
JMTF - Milestone Workbook
Rodney Omron, MD, MPH
Structure: 2011 ABEM Model Content
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
SIGNS, SYMPTOMS, AND PRESENTATIONS
ABDOMINAL AND GASTROINTESTINAL DISORDERS
CARDIOVASCULAR DISORDERS
CUTANEOUS DISORDERS
ENDOCRINE, METABOLIC, AND NUTRITIONAL DISORDERS
ENVIRONMENTAL DISORDERS
HEAD, EAR, EYE, NOSE, THROAT DISORDERS
HEMATOLOGIC DISORDERS
IMMUNE SYSTEM DISORDERS
SYSTEMIC INFECTIOUS DISORDERS
MUSCULOSKELETAL DISORDERS (NONTRAUMATIC)
NERVOUS SYSTEM DISORDERS
OBSTETRICS AND GYNECOLOGY
PSYCHOBEHAVIORAL DISORDERS
RENAL AND UROGENITAL DISORDERS
THORACIC-RESPIRATORY DISORDERS
TOICOLOGIC DISORDERS
TRAUMATIC DISORDERS
Procedures and skills integral to the practice of EMergency medicine
other core competencies of the Practice of Emergency Medicine
Validated Education Tools
Contribute your
validated
educational
research
Thank you!!!!
• Just a start
• Joint Milestones Task Force
– 11:30 – 1:00 Today Spruce Room
– Keep the great work coming