Accreditation Council for Graduate Medical Education
Milestone Project:
Yesterday, Today and
Tomorrow
GME Milestones Retreat
Washington University School of Medicine
February 7, 2014
Laura Edgar EdD, CAE
Executive Director
Disclosures
• Full time employee of ACGME
2
Presentation Topics
•
•
•
•
•
Milestone Background and Purpose
A Broader View
Design
Where Are We Now
What’s Next
3
Milestones
Yorkshire Moors
Via Ignatia
Portadon Ireland
Gemas
Malaysia
Key West, FL
County Cork
Milion of
Constantinople
Boston, MA
Apian Way
Millstones
Milestones: What
• Milestones describe performance levels residents
are expected to demonstrate for skills, knowledge,
and behaviors in the six competency domains.
• Milestones will lay out a framework of observable
behaviors and other attributes associated with
residents’ development as physicians.
• In the next accreditation system , aggregate
resident performance on the milestone level will
be used as one indicator of a residency’s
educational effectiveness.
Nasca, TJ et. Al. The Next Accreditation System. 2012.366:1051-1056
Milestones: In Other Words
Let’s look at the progression of the
wheel through 5 Levels
Milestones: In Other Words
Level 1
Milestones: In Other Words
Level 2
Milestones: In Other Words
Level 3
Milestones: In Other Words
Level 4
Milestones: In Other Words
Level 5
Milestones: In Other Words
Level 5
Progression Over Time
TIME
Level 5
Level 4
Level 3
Level 2
Level 1
PROGRESS
Supervision
Independence
The Continuum of Clinical
Professional Development
Attending
Fellowship
Residency
PGY-1 year
Sub-internship
Clerkship
Physical Diagnosis
Low
“Graded or Progressive
Responsibility”
Authority and Decision Making
© 2012 Accreditation Council for Graduate Medical Education (ACGME)
High
Milestone Template
16
Milestones are progressive over
time. There is no prescribed speed at
which residents must complete a
milestone set.
Level does not refer to
postgraduate year or year in
the program.
Option to select “Not yet rotated”
or Not yet achieved Level 1
Milestones: Why
• Fulfill the promise of the Outcome Project
• Increased use of educational outcome data
in accreditation
• Support the educational process
• ACGME accountability to public
From broad competency frameworks…
Work effectively as a member or
leader of a health care team
Develop and carry out
patient management
plans
Systematically analyze
practice using quality
improvement methods
19
Responsiveness to
patient needs that
supersedes self-interest
Know and apply the
basic and clinically
supportive sciences
which are appropriate
to their discipline
Participate in identifying system errors
and implementing potential systems
solutions.
Move from Numbers to Narratives
• Numerical systems produce range restriction
• Narratives:
• easily discerned by faculty
• shown to produce data without range restriction1
1 Hodges
and others
Most recent reference: Regehr, et al. Using “Standardized Narratives” to
Explore New Ways to Represent Faculty Opinions of Resident Performance.
Academic Medicine. 2012. 87(4); 419-427.
© 2012 Accreditation Council for Graduate Medical Education (ACGME)
Competencies/Milestones
Past Decade
 Competency evaluation stalls at individual programmatic
definitions
 MedPac, IOM, and others question
 the process of accreditation
 preparation of graduates for the “future” health care delivery
system
 House of Representatives codifies “New Physician
Competencies”
 MedPac recommends modulation of IME payments
based on competency outcomes
 Macy issues two reports (2011)
 IOM 2012-2013
Competencies/Milestones
Past Decade
Macy Foundation
MedPAC
COGME
Robert Wood Johnson
Foundation
Milestones: Who
ACGME
Residency Programs
• Accreditation – continuous monitoring of
programs; lengthening of site visit cycles
• Public Accountability – report at a national level
on competency outcomes
• Community of practice for evaluation and
research, with focus on continuous
improvement
• Guide curriculum development
• More explicit expectations of residents
• Support better assessment
• Enhanced opportunities for early
identification of under-performers
Milestones
Certification Boards
Residents
• Potential use – ascertain whether
individuals have demonstrated
qualifications needed to sit for Board exams
• Increased transparency of performance
requirements
• Encourage resident self-assessment and
self-directed learning
• Better feedback to residents
23
Milestones: How
• Joint Initiative of ACGME and ABMS
Boards
• Representatives of the of the RRC, Board,
Program Directors, Residents and
Specialty Organizations
• Face-to-face and Virtual Meetings
• Created narratives for subcompetencies
within each of the six competencies
Development Process
Pilot Test
Develop Draft
Review
Feedback
Implementation
Draft
Pilot Testing
• Pilot testing was done by most specialties
• Some specialties opted for open review by
program director groups
Changes Affected By Pilot Testing
Clearer instructions developed
Milestone levels were changed where
indicated
Some milestones completely rewritten
More examples included
Assessments to aid in milestone evaluation
developed by some specialties
Where are we now?
• All core specialties have milestones posted
• Some milestone sets include examples
• Some milestone sets include assessment tools
• Subspecialty milestones for all but three
core specialties has begun (will begin this
Spring)
• Number of subcompetencies range from
10 - 42
Milestone Framework
• Each specialty and subspecialty made a
choice:
•
•
•
•
•
•
Anatomic
Disease
Subspecialty area
Domain
Procedural
Task
• Program
requirements
• Board exam outline
• Specialty
competency
guidelines
• Other
General Competencies:
ICS, PBLI, PROF, SBP
• Wide variety
• Each group asked to include specific
elements – some did better than others
• Most widely reviewed by outsiders
• Anticipate most changes to happen here in
the future
Why are milestones so different
between specialties
• Specialty created
• Minimal required elements
• Framework vs. content
• Lumpers vs. Splitters
Challenges
• Buy-in from faculty
• Shift in thinking about resident
assessment
• When
• How
• Organization of information
• Training (and recruitment) of CCC
members
Benefits and Burdens
BENEFITS
• A shift in the way we
think about resident
education – no longer
relying on the same old
ways
BURDENS
• A shift in the way we
think about resident
education – cannot rely
on the same old ways
33
What’s Next?
• Within two years, Phase 1 milestones will
start to be evaluated
• Within three years, milestones will be
adjusted to better enhance the experience
for the learners and educators and to
better meet the needs of the public
I think the Milestones will help us
refocus this service/education divide
back toward thoughtful patient
care…Residents will know that they are
being assessed on the depth of their
thinking and the steadfastness of their
work as it relates to their patients.
Accreditation Council for Graduate Medical Education
Questions?
36
Accreditation Council for Graduate Medical Education
For more information or
questions about Milestones:
ledgar@acgme.org
37
Accreditation Council for Graduate Medical Education
Thank you!
38