MILESTONES, EPAS,
NAS…AND OTHER ACGME
JARGON
Committee on Graduate Medical Education
September 24, 2012
Sara LP Ross, MD
Objectives


To discuss the Next Accreditation System and what
is known about how that will look in 2019
To discuss the system of trainee evaluation –
Milestones and EPAs
NAS: Next Accreditation System

Goals






Complete realization of
the Outcomes promise
Free up “good programs”
to innovate
Assist poor programs in
improving
Reduce burden of
accreditation
Establish and implement
milestones to better track
program and institutional
performance
Provide accountability to
the public



July 2012: Seven initial
core specialties/RRCs
begin NAS training
July 2013: NAS officially
begins; seven specialties
“go live”; remaining
specialties begin training
July 2014: All
specialties/RRCs using
NAS
NAS: Next Accreditation System

10-year self-study visit model: next visit 2019 for all Peds department programs



Residents will submit a confidential consensus list of five strengths and “opportunities
for improvement” (OFIs) the residents wish to discuss


Structure, Resources, Core processes, Detailed processes, Outcomes
“What is your plan for the next 10 years to improve”
Site visitor will share strengths, but will only share OFIs if residents give permission (makes
residents feel more connected to the site visit)
Annual program surveillance


Performance indicators for each specialty developed by a “community of educators” within
the specialty
Annual Resident Survey





Core elements of the competencies
Levels of performance
Core methods of assessment
Annual Faculty Survey
Case Log Data
NAS: Next Accreditation System




Institutional 18 month review
Ongoing creation of Milestones
Programs will get a letter annually stating that they
meet all “performance indicator thresholds”
May be placed on “accreditation with warning” at
any point during the 10-year cycle
 May
warrant an “immediate visit”
 If problems not fixed during a given window of time,
program may be placed on probation
Site Visits

Structure of the visit: 2 site visitors
 Brief
meeting with PD
 Resident and faculty interviews
 Meeting with DIO
 Meeting with PD


PIF Elimination – YES (most likely)
More focus on strengths of programs
Competencies


Competence: the ability to do something successfully
(Oxford Dictionary of English)
Competencies: broad, general attributes of a good
doctor
 With
attempt at evaluation they get widdled down to
detailed skills/activities
 In the end don’t really reflect the original meaning of
the general competency


Competence = Attribute
Activity= Element of professional work
Entrustable Professional Activities
(EPAs)

Units of work that may be awarded a more or less
formal qualification at the moment when supervisors
confirm the trainee is ready to assume responsibility
for such activities
Entrustable Professional Activities
(EPAs)



Which critical professional
activities cover the relevant
competencies of the profession?
How can supervisors learn when
to entrust such activities to the
trainee?
“Trust reflects a dimension of
competence that reaches further
than observed ability. It includes
the real outcome of training – the
quality of care”







Part of essential professional
work
Require specific knowledge, skill
and attitude
Lead to recognized output of
professional labor
Confined to qualified personnel
Be independently executable
within a timeframe
Be observable and measurable
in its process and outcome (well
done or not well done)
Reflect one or more
competencies to be acquired
EPAs
Domains of
Competency
Competencies
Milestones
EPA/Competency Matrix
Viewpoint: Competency-Based Postgraduate Training:
Can We Bridge the Gap between Theory and Clinical
Practice?
ten Cate, Olle; Scheele, Fedde
Academic Medicine. 82(6):542-547, June 2007.
DOI: 10.1097/ACM.0b013e31805559c7
EPAs


May be acknowledged formally as a “statement of
awarded responsibility” (STAR)
Five levels of proficiency
1.
2.
3.
4.
5.
Has knowledge
May act under full supervision
May act under moderate supervision
May act independently
May act as a supervisor and instructor
Expected Levels of Confidence
Time to achieve
STAR in a specific
EPA dependent
on:
The EPA
The working
environment
The trainee
The clinical
teacher
EPA Mapped to Competencies/
Subcompetencies
Milestones



Developmental roadmap for the competencies and
subcompetencies
Observable developmental steps moving residents
from novices to experts/masters
Means of restructuring competencies into a
measurable rubric of six domains of clinical
competency
EPAs
Domains of
Competency
Competencies
Milestones
EPA
Serve as the
primary
admitting
pediatrician
for
previously
well
children
suffering
from
common
acute
problems
Milestone
Patient Care:
Gather
essential and
accurate
information
about the
patient
Patient Care:
Provide
transfer of care
that ensures
seamless
transitions
Medical
Knowledge:
Demonstrate
sufficient
knowledge of
the basic and
clinically
supportive
sciences
appropriate to
pediatrics
Level I
Level II
Level III
Gathers too little
info or
exhaustively
gathers info
following a
template
regardless of
patient's chief
complaint.
Recalls clinical
info in the order
elicited, with
ability to gather,
filter, prioritize,
and connect
pieces of info.
Advanced
development
of pattern
recognition
leads to the
creation of
illness scripts
which allow
Clinical
information to
experience
be gathered
allows linkage while it is
of signs and
simultaneously
symptoms of a filtered,
current patient prioritized and
to those
synthesized
encountered in into specific
previous
diagnostic
patients.
considerations.
Robust illness
scripts and
instance scripts
Well-developed lead to
illness scripts
unconscious
allow essential
gathering of
and accurate info essential and
to be gathered
accurate info in
and precise
a targeted and
diagnoses to be efficient
reached with
manner when
ease and
presented with
efficiency when all but the most
presented with complex or rare
most pediatric
clinical
problems.
problems.
Demonstrates
variability in
transfer of info
from one
patient to the
next. Frequent
errors of both
omission and
commission.
Uses a
standard
template for
the info
provided
during the
handoff.
Unable to
deviate from
that template
to adapt to
more complex
situations. May
have errors of
omission or
commission.
Neither
anticipates nor
attends to the
needs of the
receiver of
info.
Adapts and
applies a
standard
template to
increasingly
complex
situations in a
broad variety of
settings and
disciplines.
Ensures open
communication,
including but not
limited to readbacks, repeatbacks and
clarifying
questions
Does not know
or remember
the basic
content
knowledge of
common
pediatric
problems and
illnesses
Learns from
Able to analyze experience;
and evaluate
analyzes a
Understands knowledge in a situation,
the basic
way that allows evaluates what
Understands
content
the generation of worked well
the basic
knowledge of a meaningful
and what did
content
pediatric
differential
not, and
knowledge of practice, and is diagnosis and
creates, adapts,
pediatrics, but able to
can develop
or extrapolates
is still learning synthesize and meaningful
info
to apply it to
apply it in a
clinical
appropriately
clinical
clinical
management
to new clinical
situations
situation
plans
situations
Adapts and
applies a
standardized
template,
relevant to
individual
contexts,
reliably and
reproducibly,
with minimal
errors of
omission or
commission.
Allows ample
opportunity for
clarification
and questions.
Level IV
Level V
Adapts and
applies the
template w/o
error and
regardless of
setting or
complexity.
Internalizes the
professional
responsibility
aspect of
handoff
communication
.
What milestone levels equate
to different levels of
proficiency?
Who determines you can
practice the EPA
independently going forward?
Minimum standards for
advancement/graduation?
Where Are We in Pediatrics?




January 2012
September 2012
The Pediatrics Milestone Project (51)
Selection of 21 Pediatric Milestones to be
reported on semi-annually
- Requests for additional sites to study/
develop each of these milestones
Fall/Winter 2012 Program development of evaluation tool
July 2013
Implementation of Milestone reporting
Where Are We in Pediatrics?




January 2012
September 2012
The Pediatrics Milestone Project (51)
Selection of 21 Pediatric Milestones to be
reported on semi-annually
- Requests for additional sites to study/
develop each of these milestones
Fall/Winter 2012 Program development of evaluation tool
July 2013
Implementation of Milestone reporting
References



Carraccio C, Burke A. Beyond competencies and
milestones: Adding meaning through context. J of
Grad Med Ed. 2010;2(3):419-422.
ten Cate O. Trust, competence, and the supervisor’s
role in postgraduate training. British Medical
Journal. 2006;333:748-751.
ten Cate O, Scheele F. Competency-based
postgraduate training: Can we bridge the gap
between theory and clinical practice? Acad Med.
2001;82:542-547.