Assigning Milestone Evaluations
in Internal Medicine
Melvin Blanchard, MD, FACP
Program Director, Internal Medicine
Chief, Division of Medical Education
Department of Medicine
Outline





Internal Medicine program overview
ACGME Charges re Competencies
Response by IM community
ACGME counter response
Our program’s approach to evaluation
Internal Medicine Overview
 Discipline encompassing the study and practice of health
promotion, disease prevention, diagnosis, care and
treatment of adults
 1 of 4 physicians in the US
 IM residents
Category
Positions
Number of training programs
393
Positions in match
6177
First year fellows
4584
 Our program:
 151 trainees
 2/3 inpatient; 1/3 outpatient
 BJH, VA, community, international
 ~50 rotations
ACGME Charge - 1
 1999 – ACGME launched the Outcomes Project
 Required PDs to assess trainees in 6 competencies






Patient Care
Professionalism
Practice-Based Learning and Improvement
Interpersonal and Communication skills
Medical Knowledge
Systems-Based Practice
 Competencies required of a physician to deliver competent
medical care
ACGME Charge - 2
 2009 – ACGME charged specialties with identifying
milestones of competency development
 Observable developmental steps
 How do we know that PGY-2 resident will be competent at
graduation?
 How do we know that graduates from the 393 IM
programs can deliver same quality of IM care?
IM Community Response
 2009-12:
 Published 142 Milestones
 Aka Curricular Milestones
 Point in development that facilitates assessment of progression
from beginner to expected proficiency at end of training
 Published 16 Entrustable Professional Activities (EPAs)
 KSAs critical to practice specialty
 Milestones and EPAs categorized by 6 competencies
EPAs










Manage patients with diseases across multiple care settings.
Provide age-appropriate screening and preventative care.
Resuscitate, stabilize, and care for unstable or critically ill patients.
Provide perioperative assessment and care.
Manage transitions of care.
Facilitate family meetings.
Enhance patient safety.
Improve the quality of health care at individual and systems levels.
Demonstrate personal habits of lifelong learning.
Demonstrate professional behavior.
ACGME Counter Response
 142 milestones is too numerous
 Not optimal format for reporting to the ACGME
 ACGME and ABIM combined forces
 Developed milestones for reporting educational
outcomes
 Aka Reporting Milestones or Milestones
 Grouped into 22 sub-competences
 Grouped into 6 competencies
 With each competency associated with 2 – 5 sub-competencies
From: The Internal Medicine Reporting Milestones and the Next Accreditation System
Ann Intern Med. 2013;158(7):557-559. doi:10.7326/0003-4819-158-7-201304020-00593
Figure Legend:
Example subcompetency for systems-based practice.
Copyright © 2012 The Accreditation Council for Graduate Medical Education and The American Board of Internal Medicine.
Date of download:
2/5/2014
Copyright © American College of Physicians.
All rights reserved.
•
•
•
•
•
•
•
Competencies
Curricular milestones
Reporting milestones
Milestones
Entrustable Professional Activities
Narratives
Sub-competencies
Competencies to Milestones
Competencies
P
MK
PC
PBLI
SBP
SBP
Subcompetencies
Works in teams
Cost conscious
Transition of care
Milestones
Disregards
communication
Inconsistent
Recognizes
importance
Uses resources
Coordinates care
Curricular
Milestones
5 levels of milestones: Critical deficiency to Aspirational
EPAs
Entrustable Professional Activities
What did we do?
• Broke 22 sub-competencies into sub-sub competencies
– Each sub-sub competency can be used as a question on a
rotation evaluation
• Rotations divided among 5 faculty
– Each faculty work with core faculty/rotation directors to select
evaluation questions appropriate to rotation
• Tracked assignment to assure each sub-competency
measured multiple times
• Data from conference attendance, ITE, journal club
participation, etc. also feed into evaluation system
1. Gathers and synthesizes essential and accurate information to define each patient’s clinical problem(s). (PC1)
Critical Deficiencies
Does not collect
accurate historical data.
Inconsistently able to
acquire accurate historical
information in an
organized fashion.
Does not use physical
exam to confirm history.
Does not perform an
appropriately thorough
physical exam or misses
key physical exam
findings.
Does not seek or is overly
reliant on secondary data.
Relies exclusively on
documentation of others
to generate own
database or differential
diagnosis.
Fails to recognize
patient’s central clinical
problems.
Fails to recognize
potentially life
threatening problems.
Inconsistently recognizes
patients’ central clinical
problem or develops
limited differential
diagnoses.
Ready for unsupervised
practice
Consistently acquires
Acquires accurate
accurate and relevant
histories from patients in
histories from patients.
an efficient, prioritized,
and hypothesis-driven
fashion.
Consistently performs
Performs accurate
accurate and appropriately physical exams that are
thorough physical exams. targeted to the patient’s
complaints.
Aspirational
Obtains relevant historical
subtleties, including
sensitive information that
informs the differential
diagnosis.
Identifies subtle or unusual
physical exam findings.
Seeks and obtains data
from secondary sources
when needed.
Uses collected data to
define a patient’s central
clinical problem(s).
Synthesizes data to
generate a prioritized
differential diagnosis and
problem list.
Efficiently utilizes all
sources of secondary data
to inform differential
diagnosis.
Effectively uses history
and physical examination
skills to minimize the need
for further diagnostic
testing.
Role models and teaches
the effective use of history
and physical examination
skills to minimize the need
for further diagnostic
testing.
Milestone distribution across
Rotations
Subcompetency
1
2
3
4
5
6
7
8
Competency
PC1
PC2
PC3
PC4
PC5
MK1
MK2
SBP1
Inpt Gen Med
2
3
1
1
Rheum
1
CAER
1
Neuro
1
1
1
1
GI consult
NF
1
1
1
1
1
1
2
1
1
1