®
Development of Technology
Standards for the
Performance Framework:
Progress Report by the
Competencies Working
Group
Rosalyn P. Scott, M.D., M.S.H.A.
Wright State University
Department of Veterans Affairs
Tim Willett, M.D.
Sim-ONE
Valerie Smothers
MedBiquitous
Competencies Working Group
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Co-Chairs
Rosalyn Scott, Department of Veteran's Affairs
Tim Willett, Sim-ONE
Members
Susan Albright, Tufts University
Mary Pat Aust, American Association of Critical-Care Nurses
Dana Bostrom, Association of American Medical Colleges
Terri Cameron, Association of American Medical Colleges
Mary Jo Clark, American Association of Colleges of Nursing
Stephen Clyman, National Board of Medical Examiners
Matthew Cownie, University of the West of England
Robert Englander, Association of American Medical Colleges
Ted Hanss, University of Michigan
Sean Hilton, St. George's University of London
Joshua Jacobs, National University of Singapore
PJ Kania, Association of American Medical Colleges
David Kiger, Tufts University
JB McGee, University of Pittsburgh
David Price, American Board of Family Medicine
Isarin Sathitruangsak, Tufts University
Lesley Southgate, St. George's University of London
Nabil Zary, Karolinska Institute
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Invited Experts
Connie Bowe, Partners HealthCare International
Kelly Caverzagie, University of Nebraska Medical Center
Allan Cumming, University of Edinburgh
Rachel Ellaway, Northern Ontario School of Medicine
Simon Grant, JISC-CETIS
Linda Gwinn, Georgetown University
Robyn Herring, University of Mississippi
Kevin Krane, Tulane University
Steven Lieberman, University of Texas Medical Branch
Linda Lewin, University of Maryland
Melanie Loving, University of Texas Medical Branch
Karen Macauley, University of San Diego Hahn School of
Nursing and Health Science
Chandler Mayfield, University of California, San Francisco
Daniel Rehak
Juliane Schneider, Harvard University
®
ANSI/MEDBIQ CF.10.1-2012,
Competency Framework
• Learner and educators can search for
learning resources addressing a particular
competency.
• Educators can determine where specific
competencies are addressed in a curriculum.
• Boards and hospitals can track and manage
competency data for the professional.
• Administrators can map one competency
framework to another.
®
Competency Objects and
Frameworks
The use of outcome and competency
frameworks is a growing part of health
professions education and regulation in many
countries.
To implement competency-based education
and assessment, competency frameworks must
be connected to curricula, learning resources,
and assessment data.
®
Unfinished Business….
• Assessing Competence
• Performance Levels
• How do detailed competencies fit together to
do the work we do on a daily basis?
• What does competence look like in the real
world?
• We all know it when we see it!!
®
Performance Framework
• Requirements and XML data specifications
for the exchange of performance levels and
performance frameworks
• Initial focus on undergraduate and graduate
medical education
• North American and International paradigms
for medical education should inform the
structure of the specification
• Other healthcare professionals critical to
development
®
A competency object . . .
Current Specifications
Professionalism.xml
Competency
Framework.xml
xml
Relates to other
competency
objects . . .
xml
xml
Which can relate
to resources,
other data,
assessment . . .
xml
xml
Learning
Object
xml
xml
xml
xml
xml
xml
xml
Educational
Activity
Educational
Achievement
Data
Performance
Framework
®
Next Accreditation System:
ACGME Competencies and Outcomes
• Evaluate programs based on reporting of
outcomes through educational milestones
• Patient care
• Medical Knowledge
• Professionalism
• Systems-based practice
• Practice-based learning and improvement
• Interpersonal and communication skills
• Foster innovation and improvement by programs
®
What ACGME Plans……
• Continuous oversight not episodic sampling
• Milestone reports (semi-annual), Resident &
Faculty questionnaires (annually), Case
Logs (annually), Program & Institutional
Infrastructure (annually)
• Program Site Visits at ~10 year intervals
• Emphasis on program effectiveness as
evidenced by resident advancement through
the milestones
®
Assessing Competence
• Milestones: Granular
• Developmental framework from less to more
advanced
• Drill down to smaller elements of behavior that
provide the substance of formative feedback,
remediation, and a learning roadmap
• Entrustable Professional Activities: Holistic
• Integrate competencies within a clinical context
• Assess clusters of behaviors required to care for
patients
®
ACGME Milestones:
A Developmental Roadmap
• Milestones are knowledge, skills, attitudes,
and other attributes for each of the ACGME
competencies organized in a developmental
framework. They are descriptors and targets
for resident performance as a resident moves
from entry into residency through graduation.
®
Orthopedic Surgery Milestone
®
Framing ACGME Competencies in
Entrustable Professional Activities
• Competencies are no longer abstractions
• Reflect complex integration of competencies
needed in actual day-to-day practice
• Address authentic practice
• Entrustable designates that “a practitioner
has demonstrated the necessary knowledge,
skills and attitudes to be trusted to
independently perform this activity.”
®
Level of Mastery of an EPA
Five stages of decreasing supervision:
(1) no task execution
(2) task execution under direct supervision
on site
(3) task execution with supervision quickly
available on call
(4) unsupervised practice
(5) supervision may be provided to juniors
®
How it all works together…….
®
How are use cases addressed?
• Map performance frameworks to competencies
• YES (CompetencyReference)
• Publish a performance framework for use in
curriculum planning and assessment
• YES (Requires IDs for Components and groups of
PerformanceLevels so that assessments may be tied to
performance levels)
• Describe an individual’s current performance for
purposes of formative or summative assessment
• YES (a score and a reference to a group of
PerformanceLevels should do the trick)
®
How are use cases addressed?
• View data regarding the performance levels in a
program for both external and internal
evaluation
• YES (would require a spec that allows for the
exchange of aggregate data)
• Publish program data for public viewing
• YES (would require a spec that allows for the
exchange of aggregate data
• Define level of performance required to progress
to the next phase or block within a curriculum
• YES (will require updates to curriculum inventory)
®
How do you know a trainee is qualified and what
supervision is required?
Is competent in
Competency Framework
Trainee
Professional Profile
Is entrusted to do
Competency Framework
At what level
Performance Framework
®