Competency Framework

Development of Technology
Standards for the
Performance Framework:
Progress Report by the
Competencies Working
Rosalyn P. Scott, M.D., M.S.H.A.
Wright State University
Department of Veterans Affairs
Tim Willett, M.D.
Valerie Smothers
Competencies Working Group
Rosalyn Scott, Department of Veteran's Affairs
Tim Willett, Sim-ONE
Susan Albright, Tufts University
Mary Pat Aust, American Association of Critical-Care Nurses
Dana Bostrom, Association of American Medical Colleges
Terri Cameron, Association of American Medical Colleges
Mary Jo Clark, American Association of Colleges of Nursing
Stephen Clyman, National Board of Medical Examiners
Matthew Cownie, University of the West of England
Robert Englander, Association of American Medical Colleges
Ted Hanss, University of Michigan
Sean Hilton, St. George's University of London
Joshua Jacobs, National University of Singapore
PJ Kania, Association of American Medical Colleges
David Kiger, Tufts University
JB McGee, University of Pittsburgh
David Price, American Board of Family Medicine
Isarin Sathitruangsak, Tufts University
Lesley Southgate, St. George's University of London
Nabil Zary, Karolinska Institute
Invited Experts
Connie Bowe, Partners HealthCare International
Kelly Caverzagie, University of Nebraska Medical Center
Allan Cumming, University of Edinburgh
Rachel Ellaway, Northern Ontario School of Medicine
Simon Grant, JISC-CETIS
Linda Gwinn, Georgetown University
Robyn Herring, University of Mississippi
Kevin Krane, Tulane University
Steven Lieberman, University of Texas Medical Branch
Linda Lewin, University of Maryland
Melanie Loving, University of Texas Medical Branch
Karen Macauley, University of San Diego Hahn School of
Nursing and Health Science
Chandler Mayfield, University of California, San Francisco
Daniel Rehak
Juliane Schneider, Harvard University
ANSI/MEDBIQ CF.10.1-2012,
Competency Framework
• Learner and educators can search for
learning resources addressing a particular
• Educators can determine where specific
competencies are addressed in a curriculum.
• Boards and hospitals can track and manage
competency data for the professional.
• Administrators can map one competency
framework to another.
Competency Objects and
The use of outcome and competency
frameworks is a growing part of health
professions education and regulation in many
To implement competency-based education
and assessment, competency frameworks must
be connected to curricula, learning resources,
and assessment data.
Unfinished Business….
• Assessing Competence
• Performance Levels
• How do detailed competencies fit together to
do the work we do on a daily basis?
• What does competence look like in the real
• We all know it when we see it!!
Performance Framework
• Requirements and XML data specifications
for the exchange of performance levels and
performance frameworks
• Initial focus on undergraduate and graduate
medical education
• North American and International paradigms
for medical education should inform the
structure of the specification
• Other healthcare professionals critical to
A competency object . . .
Current Specifications
Relates to other
objects . . .
Which can relate
to resources,
other data,
assessment . . .
Next Accreditation System:
ACGME Competencies and Outcomes
• Evaluate programs based on reporting of
outcomes through educational milestones
• Patient care
• Medical Knowledge
• Professionalism
• Systems-based practice
• Practice-based learning and improvement
• Interpersonal and communication skills
• Foster innovation and improvement by programs
What ACGME Plans……
• Continuous oversight not episodic sampling
• Milestone reports (semi-annual), Resident &
Faculty questionnaires (annually), Case
Logs (annually), Program & Institutional
Infrastructure (annually)
• Program Site Visits at ~10 year intervals
• Emphasis on program effectiveness as
evidenced by resident advancement through
the milestones
Assessing Competence
• Milestones: Granular
• Developmental framework from less to more
• Drill down to smaller elements of behavior that
provide the substance of formative feedback,
remediation, and a learning roadmap
• Entrustable Professional Activities: Holistic
• Integrate competencies within a clinical context
• Assess clusters of behaviors required to care for
ACGME Milestones:
A Developmental Roadmap
• Milestones are knowledge, skills, attitudes,
and other attributes for each of the ACGME
competencies organized in a developmental
framework. They are descriptors and targets
for resident performance as a resident moves
from entry into residency through graduation.
Orthopedic Surgery Milestone
Framing ACGME Competencies in
Entrustable Professional Activities
• Competencies are no longer abstractions
• Reflect complex integration of competencies
needed in actual day-to-day practice
• Address authentic practice
• Entrustable designates that “a practitioner
has demonstrated the necessary knowledge,
skills and attitudes to be trusted to
independently perform this activity.”
Level of Mastery of an EPA
Five stages of decreasing supervision:
(1) no task execution
(2) task execution under direct supervision
on site
(3) task execution with supervision quickly
available on call
(4) unsupervised practice
(5) supervision may be provided to juniors
How it all works together…….
How are use cases addressed?
• Map performance frameworks to competencies
• YES (CompetencyReference)
• Publish a performance framework for use in
curriculum planning and assessment
• YES (Requires IDs for Components and groups of
PerformanceLevels so that assessments may be tied to
performance levels)
• Describe an individual’s current performance for
purposes of formative or summative assessment
• YES (a score and a reference to a group of
PerformanceLevels should do the trick)
How are use cases addressed?
• View data regarding the performance levels in a
program for both external and internal
• YES (would require a spec that allows for the
exchange of aggregate data)
• Publish program data for public viewing
• YES (would require a spec that allows for the
exchange of aggregate data
• Define level of performance required to progress
to the next phase or block within a curriculum
• YES (will require updates to curriculum inventory)
How do you know a trainee is qualified and what
supervision is required?
Is competent in
Competency Framework
Professional Profile
Is entrusted to do
Competency Framework
At what level
Performance Framework