DEPARTMENT OF PSYCHIATRY
The Milestone Project:
An Update
Carol A. Bernstein, MD
Associate Professor of Psychiatry
Vice Chair for Education and Director of Residency Training
Department of Psychiatry
NYU School of Medicine
Member, Board of Directors, ACGME
Beth Israel Medical Center
January 14, 2013
With Much Appreciation To:
Thomas Nasca, MD, MACP
Timothy Brigham, PhD
ACGME Staff
2
ACGME Mission Statement
“We improve health care by
assessing and advancing the
quality of resident physicians'
education through exemplary
accreditation”
The Social Contract
Compels Educators to assure that graduates:

Maintain the values and virtues of professionalism,
including excellence in clinical practice, and meet
society’s needs

Deliver safe, affordable, quality care in a fashion
that models these values and virtues, and that
meet society’s evolving needs
Thomas J. Nasca, MD, MACP
The Actions of the ACGME must fulfill the
Social Contract
And ensure Sponsors that maintain:

the safety and quality of care of the patients under
the care of residents today

the safety and quality of care of the patients under
the care of our graduates in their future practice

the provision of a humanistic educational
environment
5
Educating Physicians who commit to:
 Being scientifically and clinically competent
 Minimizing self interest in favor of the protection
and promotion of the patient’s health-related
interests (Altruism)
 Maintaining and passing on medicine to future
patients, physicians, and society as a public
trust, not a merchant guild
McCullough, L. The Ethical Concept of Medicine as a Profession.
Advances in Bioethics, Volume 10, 17-27. 2006.
Thomas J. Nasca, MD, MACP, modified by Timothy P. Brigham
Why change now?
7
Brief History
 Late 1990’s: ACGME/ABMS Competencies developed
 ACGME requires:




Competency framework for curriculum
Phased incorporation of competency evaluation
Culminates in “competency based accreditation”
The “field” left to develop “tools” for the “toolbox”
 ABMS organizes individual certification in
Competency framework
 Initial Certification
 Maintenance of Certification
 Tools developed that may be relevant in GME phase
T. Nasca, MD, MACP, 2011
The 2005 ACGME Strategic Plan :
1
At its November 2005 retreat, the ACGME Executive
Committee endorsed four strategic priorities designed to
enable emergence of the new accreditation model:
 Foster innovation and improvement in the learning
environment
 Increase the accreditation emphasis on educational
outcomes
 Increase efficiency and reduce burden in accreditation
 Improve communication and collaboration with key internal
and external stakeholders
1 ACGME
2005 Strategic Plan
9
External Pressures
2009 - House of
Representatives
Codifies “New
Physician
Competencies”
Changing Public
Expectations: IOM,
MedPac, Macy
Reports
CMS encouraged
to modulate IME
payments based
on competency
outcomes
Pressures
on the
GME
System
Recognized need
to move from
“circumstantial” to
“intentional”
practice in resident
education
10
Never be afraid to try something new.
Remember that amateurs built the ark,
and professionals built the Titanic.
Anonymous
11
What Currently Drives the Structure and
Content of our Residency Programs?
12
What Will Drive the Structure and Content
of our Residency Programs in the Near
Future?
13
“One definition of insanity
is doing the same thing
over and over again, but
expecting different
results.”
Rita Mae Brown
Sudden Death, 1983. p. 68
The Conceptual Change From…
The Current Accreditation System
Rules
Corresponding Questions
“Correct or Incorrect Answer”
Citations and Accreditation Decision
15
The Conceptual Change To…
The “Next Accreditation System”
16
Goals of the
“Next Accreditation System”
(NAS)
17
Goals of NAS:

To foster the development of realistic outcomes

To free good programs to innovate

To help weak programs to improve

To reduce the burden of accreditation

To provide accountability for outcomes to the public
18
NAS in a Nutshell

Continuous Accreditation Model – annually updated
 Based on annual data submitted, other data requested, and
program trends

Scheduled Site Visits replaced by 10 year Self Study Visit

Standards revised every 10 years
Organized by:
 Structure
 Resources
 Core Processes
 Detailed Processes
 Outcomes
19
20
Phased Implementation
 Phase 1 Specialties (and Subspecialties)







Internal Medicine
Pediatrics
Emergency Medicine
Diagnostic Radiology
Urology
Orthopedic Surgery
Neurological Surgery
 Phase 2 Specialties (and Subspecialties)



All Other Specialties
Institutional Review
Transitional Year
Implementation Timeline
PHASE 1
PHASE 2
Ceased routine Site
Visits
First round of CLER
visits
(underway)
Will cease routine
Site Visits
PHASE 1
Will begin Continuous Accreditation
9/2012
1/2012
7/2013
(through 6/2014)
7/2012
1/2013
7/2014
PHASE 2
Will enter “preparation year”
PHASE 1
Entered “preparation year”
PHASE 2
New Policies and
Procedures take effect
(7/1/2013)
CLER = Clinical Learning Environment Review
Will begin Continuous Accreditation
Essential Elements
 Continuous (as opposed to Periodic) Accreditation
 Increased Focus on Outcomes

Enhance flexibility of programs to innovate
 Standardization Balanced with Specialty Specificity
 Increased Emphasis on Patient Safety and Quality
Improvement
 Enhanced Continuity Across the Spectrum of
Training
 Transparency
 Enhanced Institutional Accountability
Thomas J. Nasca, MD, MACP
“Every system is
perfectly designed to
yield the result it
produces.”
The Building Blocks of
The Next Accreditation System
25
Trended Performance Indicators

Annual ADS Update
 Program Attrition – Changes in PD/Core Faculty/Residents
 Program Characteristics – Structure and Resources
 Scholarly Activity

Board Pass Rate – Rolling Rates

Resident Survey – Common and Specialty Elements

Clinical Experience – Case Logs or other

Faculty Survey – Core Faculty

Semi-Annual Resident Evaluation and Feedback

Annual Sponsor Site Visit (CLER) – Every 18 Months
26
ACGME Goals for Milestones
 Accountability for effectiveness of educational
programs in producing outcomes
 Collaboration with AAMC, LCME to improve
graduation level preparation
 Collaboration with ABMS, AHA, ACCME,
others to identify areas for milestone
improvement at the time of graduation from
residency
Milestones
ACGME Goals for Milestones
 Outcome focused
 Use of existing tools and observations of the
faculty
 Clinical Competency Committee triangulates
the progress of each resident

ABMS Board, Academy, Program Directors and RRC
jointly define the expectations (Milestones)

ABMS Board tracks the identified individual

ACGME Review Committee tracks unidentified
individual trajectories
 ACGME and ABMS are able to provide
accountability for effectiveness of the
educational program in producing outcomes for
individual trainees
T. Nasca, MD, MACP, 2012
ACGME Goals for Milestones
 Specialty specific normative data and common
expectations for progress of individual residents
 Less prescriptive ACGME program
requirements, lengthened program site visit
cycles, less frequent standards revision

Promotion of curricular innovation

Enhancement of curricular and rotation design
 Opportunity for communication and
improvement across the continuum of medical
education
 Development of specialty specific evaluation
tools and techniques
T. Nasca, MD, MACP, 2012
ACGME Goals for Milestones
 Able to provide accountability for effectiveness of
educational program in producing outcomes
ACGME can work with:
 AAMC, LCME to focus graduation level preparation
 ABMS, AHA, ACCME, others to identify areas for milestone
improvement at graduation from residency/ fellowship
30
Increase the Accreditation Emphasis on
Educational Outcomes
Specialty Specific
Teams
Board
PD Association
College
RRC
Residents
Milestones
&
Core Evaluation
Requirements
ACGME
Specialty Specific
Competency Evaluation
Program Requirements
for reporting
(Outcomes)
Thomas J. Nasca, MD, MACP
Modified by T. Brigham
Expectations for Milestones

When and where possible, identification of
appropriate assessment tools/methods/systems

Creation of a summary reporting document to be
used by a Clinical Competence Committee in
each program to report to the ACGME and
Specialty Board
Not to you
Not for you
With you!
Clinical Competency
34
Clinical Competency
Milestones: Observable developmental steps
moving from Novice to
Expert/Master
 Organized under the rubric of the
6 domains of clinical competency




Describe a trajectory of progress from neophyte
towards independent practice
Articulate a shared understanding of
expectations
Set aspirational goals of excellence
Provide a framework and language for
discussions across the continuum
General Competences
1) Patient Care and Technical Skill
 Compassionate, appropriate, effective
2) Medical Knowledge
 Know and can apply
 Can do and apply
3) Practice-Based Learning and Improvement
 Assessment of own patient care, evidence-based
approaches, improvement
4) Interpersonal and Communication Skills
 Effective exchange of information and collaboration
with patients, their families, and health professionals
5) Professionalism
 Committed to professional responsibilities, ethical
principles and sensitivity to diversity
6) Systems-Based Practice
 Awareness and utilization of the larger context and
system of healthcare in providing optimal patient care
The Six Competencies, and the Continuum of Clinical
Medical Education – Dreyfus (modified) Model1
Medical Knowledge
Patient Care and
Procedural Skills2
Interpersonal and
Communication Skills
Professionalism
Novice
Advanced Beginner
Competent
Proficient
Expert
Master
Practice Based Learning
and Improvement
Systems Based
Practice
Undergraduate
Graduate
Continuing
1 as
presented by Leach, D., modified by Nasca, T.J.
American Board of Internal Medicine Summer Retreat, August, 1999.
2 Patient Care Competency modified 9/2010 by ACGME and ABMS
37
Miller’s Pyramid of Clinical Competence1
1Miller,
GE. Assessment of Clinical Skills/Competence/Performance.
Academic Medicine (Supplement) 1990. 65. (S63-S67 )
38
Key Elements of Miller’s Pyramid
 Trained Observers

Common understanding of the expectations

Sensitive “eye” to key elements

Consistent evaluation of a given level of performance
 Minimum Numbers of Quality Observations
 Interpreter/Synthesizer Experts

Clinical Competency Committee
(Resident Evaluation Committee)
39
Trajectory of Milestones
40
Clinical Professional Development
41
The Goal of the Continuum of Clinical
Professional Development
42
Reporting Template
Milestone of Competency Development
Level 1
Level 2
Level 3
Level 4
Entry –
Baseline,
expected level at
time of entry
into residency
Mid-Program Developmental
levels of
performance
Offers road
map and
assurance that
residents are
attaining
appropriate
educational
goals
Mid-Program Developmental
levels of
performance
Offers road
map and
assurance that
residents are
attaining
appropriate
educational
goals
Graduation –
Expected level
of performance
at entry into
unsupervised
practice
Level required
to gain eligibility
for ABMS
certification
Level 5
Stretch Goals –
Exceeds
expectations
Comments:
43
Milestones: Guiding Principles
Feasibility
Quality
Reporting
• Balance costs
with benefits
• Manageable
number of
milestones
• Improvement
over current
approaches
• Meaningful &
substantive
• “Measurable”
• Specialty-wide
use of 5-level
template for
milestones
reporting
• Central data
repository
44
Next Steps in the Outcomes Project
Milestone definitions: Descriptions (in specific
behavioral terms) of the performance level expected
of a resident by a particular time during their residency
Aggregation of resident performance on
milestones as an indicator of overall
programmatic educational effectiveness
Board use as part of eligibility for certification
45
“Somebody has to do something,
and it’s just incredibly pathetic fantastic
that it has gets to be us.”
Jerry Garcia
The Grateful Dead
edits, TJ Nasca
Gratefully Not Dead
46
ACGME Milestone Project Team
Department of Education
Department of Accreditation Activities
Susan Swing, PhD
Vice President, Outcomes Assessment
Eileen Anthony, MS
Steve Nestler, PhD
Senior Milestone Consultant
Pamela Derstine, PhD
Laura Edgar, EdD
Senior Milestone Consultant
Caroline Fischer, MBA
Erin Axley
Milestone Project Administrator
Louise King, MS
Erin Berryhill
Milestone Project Administrator
Patricia Levenberg, PhD
Lynne Meyer, PhD, MPH
Peggy Simpson, EdD
Patricia Surdyk, PhD
Linda Thorsen, MA
Jerry Vasilias, PhD
Clinical Competence Committee
48
Six Month Evaluations
 Determined by the CC Committee
 Grouped by Core Competency
 Established by Professional Consensus
 Part of Programmatic Self-Evaluation and
Accreditation
 Measureable and Meaningful
49
Expert Panels Assembled by the
ACGME for:
 Systems-based Practice
 Practice-based Learning and
Improvement
 Communication Skills
 Professionalism
 Assessment
ACGME EXPERT PANEL ON MILESTONES
DRAFT VERSION 9-11-11
PROFESSIONALISM
INTERPERSONAL AND COMMUNICATION SKILLS
PRACTICE-BASED LEARNING AND IMPROVEMENT
SYSTEMS-BASED PRACTICE
Charge to the Panel: Develop milestones in Professionalism,
Interpersonal and Communications Skills, Practice-Based
Learning and Improvement, and Systems-Based Practice for
adoption and/or adaptation by specialty milestone groups.
Milestones were created to reflect the following expected
levels of performance: Level 1 – Typical graduating medical
student; Levels 2 and 3: Resident during the program; Level
4: Graduating resident; Level 5: Advanced, specialist
resident or practicing physician
This document is for ACGME milestone use only. Do not disseminate.
Professionalism
Elements of Professionalism
in Internal Medicine
Adheres to basic ethical principles
Demonstrates compassion and
respect for Patients
Provides timely, constructive
feedback to colleagues
Maintains accessibility
Recognizes conflicts of interest
Demonstrates personal
accountability
Practices individual patient advocacy
Milestones of Training
 Demonstrates empathy and compassion
to all patients (3rd month)
 Demonstrates a commitment to relieve
pain and suffering (3rd month)
 Provides support (physical, psychological,
social, and spiritual) for dying patient and
their families (24th month)
 Provides leadership for a team that
respects patient dignity and autonomy
(24th month)
Green, et.al. Journal of Graduate Medical Education. 1(1). 5-20. 2009
Milestone Framework
Competency
Subcompetency
(approximately
6)
Developmental
Milestones
Narratives
(approximately 5 narratives
of a Dreyfus or equivalent
trajectory)
“If you think you’re too small
to be effective, you have
never been in bed with a
mosquito.”
Betty Reese
(American officer and pilot)
The “Envelope of Expectations”
Professionalism:
Accepts responsibility and follows through on tasks
Medical
School
Expert
Proficient
Competent
Advanced
Beginner
PGY 1
PGY 2
PGY 3
PGY 4 effectively
PGY 5manages
MOC
Resident
multiple competing tasks, and
effortlessly manages complex
Resident alwayscircumstances.
prioritizes and willingly
Is clearly identified
works on multiple
bycompeting
peers andcomplex
subordinates as
and routine cases
in
a
timely
manner
by support in
source of guidance and
directly providingdifficult
patientorcare
or by circumstances.
unfamiliar
overseeing it. In difficult circumstances
appropriately seeks guidance. Is
Resident frequently
prioritizes
regularly
soughtmultiple
out by peers and
competing demands
and
completes
thethem guidance.
subordinates to provide
vast majority of his/her responsibilities in
a timely manner. Self identifies
circumstances and actively seeks
guidancecompletes
in unfamiliar
circumstances.
Resident routinely
most
assigned tasks in a timely manner in
accordance with local practice and/or
policy, but still requires guidance in
circumstances.
Resident completesunfamiliar
many assigned
tasks on time but needs extensive
Novice
guidance on local practice and/or
© 2012 Accreditation Council for
policy for patient care.
Graduate Medical Education (ACGME)
Resident frequently fails to
Professionalism
recognize or actively avoids
opportunities for compassion
or empathy. On occasion
demonstrates lack of
respect, or overt disrespect
for patients, family members,
or other members of the
health care team
Resident seeks out opportunities
to demonstrate compassion and
empathy in the care of all patients;
and demonstrates respect and is
sensitive to the needs and
concerns of all patients, family
members, and members of the
health care team.
Resident demonstrates
compassion and empathy in care
of some patients, but lacks the
skills to apply them in more
complex clinical situations or
settings. Occasionally requires
guidance in how to show respect
for patients, family members, or
other members of the health care
team.
Thank You!