Curriculum Inventory Developer’s Workshop Part II: Competency Framework Terri Cameron, MA Director of Curriculum Management Valerie Smothers, MA Deputy Director, MedBiquitous We will cover: • MedBiquitous Competency Objects and Competency Framework • How the curriculum structure, the Competency Framework, and the Physician Competency Reference Set (PCRS) work together • How to apply the Competency Framework and Competency Object Business Rules to Curriculum Inventory data • How to document school expectation mapping to PCRS using the MedBiquitous Competency Framework Vendor and Schooldeveloped Curriculum Management Systems Accreditation Standards SelfEvaluation Tool (ASSET) Institutionallydeveloped expectations will be matched to Physician Competency Reference Set Standardized Vocabulary for Instructional Methods, Assessment Methods, and Resources Curriculum Inventory ASSET Dashboard Curriculum data mapped to an expectations hierarchy (competencies to course objectives to session objectives) are uploaded to the Curriculum Inventory (CI) from school systems using the MedBiquitous CI data exchange standard (www.medbiq.org). www.aamc.org/medaps Review of Terminology • Previous Academic Year (LCME model – if current year is 2013-2014, previous academic year is 2012-2013) • Academic Level / Phase (formerly called ‘year of curriculum’) • Sequence Block (Course, Module, Unit, Block, Clerkship) • Event (Instructional or Assessment Session) (e.g., Lecture, Lab, Discussion, Assessment) www.aamc.org/medaps Review of Terminology, Cont’d. • Standardized Vocabulary (Instructional Methods, Assessment Methods) • PCRS (Physician Competency Reference Set) • Expectations (Competency / Objective / Milestone / EPA) • Verification Report (Curriculum Inventory data report used to verify curriculum details) • MedBiquitous (www.medbiq.org) www.aamc.org/medaps MedBiquitous Competency Object and Competency Framework ® Mission: To develop and promote technology standards for the health professions that advance lifelong learning, continuous improvement, and better patient outcomes. Not-for-profit, member-driven, standards development organization Building the Foundation: The MedBiquitous Architecture (www.medbiq.org) * American National Standards † Candidate standard Competency Concepts Competency Object - an umbrella term used to describe any abstract statement of learning or performance expectations, and information related to the statement.1 Competency Framework - an organized and structured representation of a set of interrelated and purposeful competency objects. 1 1 ANSI/MEDBIQ CF.10.1-2012 Competency Framework Specifications, October 3, 2012. Competency Object • Statements can be learning outcomes, competencies, learning objectives, professional roles, topics, classifications, collections, etc. • The Object is abstract in that it does not inherently contain information about connections of the statement to individuals, events or other objects. Competency Object Example From ACGME: • • • • URL Title (statement) Description Category http://example.com/competency/ 12345.xml Practice-based Learning and Improvement Residents must demonstrate the ability to investigate and evaluate their care of patients . . . Category = program-levelcompetency Competency Object Example From ACGME: • • • • URL Title (statement) Description Category <CompetencyObject > <lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry> http://example.com/competency/12345.xml </lom:entry> </lom:identifier> <lom:title> <lom:string language="en"> Practice-based Learning and Improvement </lom:string> </lom:title> <lom:description> <lom:string language="en"> Residents must demonstrate the ability to investigate and evaluate their care of patients... </lom:string> </lom:description> </lom:general> </lom:lom> <Category term=“program-level-competency"/> </CompetencyObject> A competency object . . . Competency Framework Statement Relates to other competency objects . . . Which can relate to external resources . . . Learning Object Assessment Performance data How Do We Describe Relationships in a Competency Framework? SKOS Mapping: Simple Knowledge Organization System (W3C, World Wide Web Consortium, www.w3.org) http://www.w3.org/TR/skos-reference/#mapping Related (skos:related) = “is related to concept” Broad (skos:broader) = “has broader concept” Narrow (skos:narrower) = “has narrower concept” Domain Competency Learning Objective Example A B C <CompetencyFramework> What competencies are included in ... <Includes> this framework? <Catalog>URI</Catalog> <Entry>http://www.example.org/A.xml</Entry> </Includes> <Includes> <Catalog>URI</Catalog> <Entry>http://www.example.org/B.xml</Entry> </Includes> <Includes> <Catalog>URI</Catalog> <Entry>http://www.example.org/C.xml</Entry> </Includes> <Relation> What are the relationships <Reference1> among the competencies <Catalog>URI</Catalog> <Entry>http://www.example.org/A.xml</Entry> included? </Reference1> <Relationship>http://www.w3.org/2004/02/skos/core#narrower</Relationship> <Reference2> <Catalog>URI</Catalog> <Entry>http://www.example.org/B.xml</Entry> </Reference2> </Relation> . .. </CompetencyFramework> Questions? Use Chat to submit questions • Please send to All Panelists. www.aamc.org/medaps How would you specify this relationship? 1 Has ________ concept 2 A: Related B: Broader C: Narrower Has ________ concept 3 How would you specify this relationship? 1 Has narrower concept 2 A: Related B: Broader C: Narrower Has narrower concept 3 The curriculum structure and the Competency Framework The Curriculum Inventory and Competency Frameworks Curriculum Inventory 3 I A M E E J B I 1 2 J O F 1 5 9 K C A G D 8 N 2 L H 3 P C Q G S R 7 4 8 5 9 6 10 H Competency Framework 11 The Curriculum Inventory uses Competency Frameworks to define the competencies/ objectives integrated into the curriculum. Curriculum Inventory: “Expectations” • Contain Competency Objects (e.g., learning objectives, competencies, learning outcomes) • Contain a competency framework that describes a set of relationships between competency objects • Are referenced from events or sequence blocks Curricular Structure and Competency Objects Sequence Blocks (e.g., courses) reference A Events and 1 Competency Objects Events (e.g., sessions) reference 1 Competency Objects and Elements (instructional methods, assessment methods, etc) I A E B F 1 G D 2 K C A J 7 L H Sequence Blocks, Events, and Competency Objects • Some schools map relationships between Sequence Block and Event learning objectives • Some schools map relationships between Program Level Objectives / Competencies and Sequence Block learning objectives OR Event learning objectives • Some schools do not have any mapping for Sequence Block learning objectives OR Event learning objectives • The MedBiquitous Competency Framework supports any of these options Indicating the Type of Competency • For the Curriculum Inventory, there are three categories of Competency Objects within the Framework: • Program-level This program instills the following competencies… • Sequence Block-level By the end of this course, you will be able to… • Event-level By the end of this session, you will be able to… Relationships in a Competency Framework Program-level competency Program-level competency Program-level competency Has Narrower Concept Sequence block-level competency Has Narrower Concept Event-level competency Event-level competency Has Narrower Concept Sequence block-level competency Has Narrower Concept Event-level competency Event-level competency School Competency Framework Medical Knowledge: Demonstrate basic knowledge of normal structure, development, and function of major organ systems and the body as a whole in the context of health and disease. Medical Knowledge: Describe common disease entities, including their characteristic signs and symptoms, etiology, epidemiology, and pathophysiology. Has Narrower Concept Has Narrower Concept Describe the major organ systems, their components and the relationships between them in the normal individual Has Narrower Concept Identify the organs involved in the renal system Explain the interaction of the heart and lungs in the circulatory system Identify common variations in structure and location of organs within the body Has Narrower Concept Describe the anatomical structure of the heart Explain how structural variations affect kidney function What do we do with 150+ Different Competency Frameworks? • We build an aggregation tool! • Physician Competency Reference Set (PCRS) • 58 Competencies across 8 Domains • Each school creates a crosswalk between school competencies and PCRS • Using MedBiq Competency Framework, School Program Objectives/Competencies are mapped to PCRS • Competency Mapping at other levels can be tracked through the Program Objective / PCRS relationship Expectation Levels PCRS Program Level Sequence Block Level Event Level U.S./ Canada Medical School Course/ Clerkship Session/ Experience Apply established and emerging bio-physical scientific principles fundamental to health care for patients and populations. Demonstrate basic knowledge of normal structure, development and function of major organ systems and the body as a whole in the context of health and disease. Identify common variations in structure and location of organs within the body. Describe the anatomical structure of the heart. www.aamc.org/medaps Physician Competency Reference Set Partial URI Physician Competency Reference Set (AAMC 2013) aamc-pcrs-comp-c0100 PATIENT CARE: Provide patient-centered care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health Perform all medical, diagnostic, and surgical procedures considered essential for the area of practice Gather essential and accurate information about patients and their condition through history-taking, physical examination, and the use of laboratory data, imaging, and other tests Organize and prioritize responsibilities to provide care that is safe, effective, and efficient Interpret laboratory data, imaging studies, and other tests required for the area of practice Make informed decisions about diagnostic and therapeutic interventions based on patient information and preferences, up-to-date scientific evidence, and clinical judgment aamc-pcrs-comp-c0101 aamc-pcrs-comp-c0102 aamc-pcrs-comp-c0103 aamc-pcrs-comp-c0104 aamc-pcrs-comp-c0105 Competency Framework and PCRS PCRS 1.5 Program-level competency Program-level competency Has Narrower Concept Event-level competency Event-level competency PCRS 6.99 Program-level competency Has Narrower Concept Sequence block-level competency Is Related Is Related Has Narrower Concept Sequence block-level competency Has Narrower Concept Event-level competency Event-level competency Competency Framework and Curriculum Structure: One Example Curriculum Structure Competency Framework Program (School Curriculum) PCRS 6.99 PCRS 1.5 The Programlevel Competency has narrower Sequence Block-level Competency skos: related skos: related P3 skos: narrower The Sequence Block-Level Competency has narrower Eventlevel Competency S5 Has Narrower Concept E1 P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency E2 Medical Knowledge: Demonstrate basic knowledge of normal structure, development, and function of major organ systems and the body as a whole in the context of health and disease. Sequence Block Has Narrower Concept Identify common variations in structure and location of organs within the body Event Has Narrower Concept Explain how structural variations affect kidney function Relationships and References Competency Framework Curriculum Inventory PCRS 6.99 PCRS 1.5 skos: related skos: related P2 skos: narrower P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency The Sequence Block References the Program-level Competency I P2 E B S5 The Sequence Block References the Sequence Block-level Competency F E2 A S5 skos: narrower The Event References the Event-level Competency J K C A G D 7 L H Learning Check How do you know the role of a competency in the curriculum? a) The Competency Object Category is tagged as Program-level, Sequence block-level, or Eventlevel b) The competency is referenced from a sequence block(s) or event(s) in the curriculum c) The competency framework tells you how the competencies are related hierarchically d) All of the above Learning Check How do you know the role of a competency in the curriculum? a) The Competency Object Category is tagged as Program-level, Sequence block-level, or Eventlevel b) The competency is referenced from a sequence block(s) or event(s) in the curriculum c) The competency framework tells you how the competencies are related hierarchically d) All of the above Questions? Use Chat to submit questions. • Please send to All Panelists. www.aamc.org/medaps AAMC Business Rules General Rules • Program-level competencies are required and must be mapped to PCRS using #related • All competencies, including PCRS, programlevel, sequence-block level, and event level competencies used in a <Relation> tag must have an <Includes> tag PCRS Mapping and Program Level Competencies • Program-level competencies may not be mapped to domain-level PCRS • Program Objectives / Competencies that do not match a PCRS competency can be mapped to the appropriate Domain ‘Other’ • (numbered 99 in each Domain) Matching Program Objectives to PCRS (Crosswalk) PCRS Program Objectives 2. Knowledge for Practice: Demonstrate knowledge of established and evolving biomedical, clinical, epidemiological and social-behavioral sciences, as well as the application of this knowledge to patient care. 2.1 Demonstrate an investigatory and analytic approach to clinical situations 2.2 Apply established and emerging bio-physical scientific principles fundamental to health care for patients and populations 2.3 Apply established and emerging principles of clinical sciences to diagnostic and therapeutic decisionmaking, clinical problem-solving, and other aspects of evidence-based health care 2.4 Apply principles of epidemiological sciences to the identification of health problems, risk factors, treatment strategies, resources, and disease prevention/health promotion efforts for patients and populations 2.5 Apply principles of social-behavioral sciences to provision of patient care, including assessment of the impact of psychosocial and cultural influences on health, disease, care-seeking, care compliance, and barriers to and attitudes toward care 2.6 Contribute to the creation, dissemination, application, and translation of new health care knowledge and practices 2.99 Other knowledge for practice 2. Demonstrate a broad working knowledge of the fundamental science, principles, and processes basic to the practice of medicine. 2.1 Understand the clinical relevance of scientific inquiry. 2.2 Apply this knowledge in a judicious and consistent manner to prevent common health problems and achieve effective and safe patient care. 2.3 Demonstrate the ability to evaluate emerging knowledge and research as it applies to diagnosis, treatment and the prevention of disease. 2.4 Utilize state of the art information technology and tools to retrieve, manage and use biomedical information in the care of individuals. 2.5 Utilize state of the art information technology and tools to retrieve, manage and use biomedical information in the care of populations. 2.6 Understand the indications, contraindications, and potential complications of common clinical procedures. 2.7 Perform the basic clinical procedures expected of a new PGY-1. Defining the Competency Objects <CompetencyObject> <lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry>http://medicine.edu/competencies/K_2_1</lom:entry> </lom:identifier> <lom:title> <lom:string>Understand the clinical relevance of scientific inquiry. </lom:string> </lom:title> </lom:general> </lom:lom> </CompetencyObject> Matching Program Objectives to PCRS <CompetencyFramework> (metadata and included competencies) <Relation> <Reference1> <Catalog>URI</Catalog> <Entry>http://medicine.edu/competencies/K_2_1</Entry> </Reference1> <Relationship>http://www.w3.org/2004/02/skos/core#related</Relationship> <Reference2> <Catalog>URI</Catalog> <Entry>http://aamc.org/pcrs/aamc-pcrs-comp-c0201</Entry> </Reference2> </Relation> PCRS Mapping and Sequence Block or Event-level Competencies • Mapping of Sequence Block-level or Eventlevel objectives / competencies directly to PCRS is not allowed. • Sequence Block and / or Event Objectives / Competencies if mapped, are mapped to each other and to Program Objectives / Competencies Competency Relationships PCRS 6.99 PCRS 1.5 Explanation P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency skos: related skos: related P2 skos: narrower <P2> related <PCRS 1.5> <P2> related <PCRS 6.99> S5 skos: narrower skos: narrower E2 E3 <P2> has narrower concept <S5> <S5> has narrower concept <E2> <S5> has narrower concept <E3> What’s Wrong with this Mapping? Partial URI PCRS aamc-pcrs-comp-c0100 PATIENT CARE: Provide patientcentered care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health Perform all medical, diagnostic, and surgical procedures considered essential for the area of practice Gather essential and accurate information about patients and their condition through historytaking, physical examination, and the use of laboratory data, imaging, and other tests Organize and prioritize responsibilities to provide care that is safe, effective, and efficient Interpret laboratory data, imaging studies, and other tests required for the area of practice aamc-pcrs-comp-c0101 aamc-pcrs-comp-c0102 aamc-pcrs-comp-c0103 aamc-pcrs-comp-c0104 Curriculum Structure (School Curriculum) Program-Level Competency Sequence BlockLevel Competency Event-Level Competency Correct Mapping Partial URI PCRS aamc-pcrs-comp-c0100 PATIENT CARE: Provide patientcentered care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health Perform all medical, diagnostic, and surgical procedures considered essential for the area of practice Gather essential and accurate information about patients and their condition through historytaking, physical examination, and the use of laboratory data, imaging, and other tests Organize and prioritize responsibilities to provide care that is safe, effective, and efficient Interpret laboratory data, imaging studies, and other tests required for the area of practice aamc-pcrs-comp-c0101 aamc-pcrs-comp-c0102 aamc-pcrs-comp-c0103 aamc-pcrs-comp-c0104 Curriculum Structure (School Curriculum) Program-Level Competency Sequence BlockLevel Competency Event-Level Competency Competency Object Rules • <Competency Object> tags must be used to identify competencies, along with their <Category> (level) • Program-level • Sequence block-level • Event-level • Competency object URIs must be unique • <Competency Object> tags must not be used to identify PCRS competencies Sample Code: Competency Objects <Competency Object> tag along with the <Category> and a unique URI <CompetencyObject> <lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry>http://med.sample.edu/competencies/ProgramPhysician-Patient-Rel</lom:entry> </lom:identifier> <lom:title> <lom:string>Physician-Patient Relations</lom:string> </lom:title> </lom:general> </lom:lom> <co:Category term="program-level-competency"/> </CompetencyObject> Competency Objects and the Curriculum Structure • All sequence block-level and event-level competencies must be referenced by at least one Sequence Block or Event (that was identified in the curriculum structure) Competency Framework Curriculum Inventory PCRS 6.99 PCRS 1.5 skos: related skos: related P2 skos: narrower P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency S5 The Sequence Block References the Program-level Competency The Sequence Block References the Sequence Block-level Competency I P2 E B S5 F J A Circular References Relationships must not result in circular references. A circular reference occurs when a child element draws a hierarchical relationship to its parent that infers the parent is lowerlevel than the child. PCRS 6.99 PCRS 1.5 skos: related skos: related P2 P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency skos: narrower S1 skos: narrower skos: narrower S2 skos: narrower S3 This relationship is not allowed because it creates a “Circular Reference” and violates a Business Rule. skos: narrower E4 Errors you may receive Error 1: URI 'http://medschool.edu/CourseObjective_9' referred in Competency Object was not found under Sequence Block Competency Reference Competency Framework Curriculum Inventory PCRS 6.99 PCRS 1.5 skos: related skos: related I P2 skos: narrower P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency P2 B S9 E F J A Solution: Add a Sequence Block Competency Reference (or delete the competency object) Competency Framework Curriculum Inventory PCRS 6.99 PCRS 1.5 skos: related skos: related I P2 skos: narrower P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency P2 E B S9 S9 F J A <SequenceBlock id="c" required="Required"> <Title>Year 1</Title> <Timing> <Duration>P12M</Duration> </Timing> <Level>/CurriculumInventory/AcademicLevels/Level[@number='1']</Level> <CompetencyObjectReference>/CurriculumInventory/Expectations/CompetencyObject[lom:lom/lom:general/lom:identifier/lom:en try='http://medschool.edu/CourseObjective_9']</CompetencyObjectReference> </SequenceBlock> Error 2: URI 'http://medschool.edu/CourseObjective_9' referred in Includes was not found under Competency Object <CompetencyObject> ... </CompetencyObject> <CompetencyFramework> ... <Includes> <Catalog>URI</Catalog> <Entry>http://medschool.edu/CourseObjective_9</Entry> </Includes> . .. </CompetencyFramework> Solution: Add a competency object for that competency: <CompetencyObject > <lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry> 'http://medschool.edu/CourseObjective_9 </lom:entry> </lom:identifier> <lom:title> <lom:string language="en"> Describe cardiovascular anatomy. </lom:string> </lom:title> </lom:general> </lom:lom> <Category term=“Sequence-block level"/> </CompetencyObject> Error 3: URI 'http://medschool.edu/CourseObjective_9' referred in Sequence Block competency object reference was not found under Competency Object Competency Framework Curriculum Inventory PCRS 6.99 PCRS 1.5 skos: related skos: related I P2 skos: narrower P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency P2 E B S9 F J A Solution: Add a competency object for that competency: <CompetencyObject > <lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry> 'http://medschool.edu/CourseObjective_9 </lom:entry> </lom:identifier> <lom:title> <lom:string language="en"> Describe cardiovascular anatomy. </lom:string> </lom:title> </lom:general> </lom:lom> <Category term=“Sequence-block level"/> </CompetencyObject> Error 4 URI 'http://medschool.edu/CourseObjective_9' referred in Relations was not found under Includes <CompetencyFramework> ... <Includes> <Catalog>URI</Catalog> <Entry>http://medschool.edu/EventObjective_2.xml</Entry> </Includes> <Relation> <Reference1> <Catalog>URI</Catalog> <Entry>http://medschool.edu/CourseObjective_9</Entry> </Reference1> <Relationship>http://www.w3.org/2004/02/skos/core#narrower</Relationship> <Reference2> <Catalog>URI</Catalog> <Entry>http://medschool.edu/EventObjective_2.xml</Entry> </Reference2> </Relation> . .. </CompetencyFramework> Solution Add an includes statement for that competency. <CompetencyFramework> ... <Includes> <Catalog>URI</Catalog> <Entry>http://medschool.edu/EventObjective_2.xml</Entry> </Includes> <Includes> <Catalog>URI</Catalog> <Entry>http://medschool.edu/CourseObjective_9.xml</Entry> </Includes> <Relation> <Reference1> <Catalog>URI</Catalog> <Entry>http://medschool.edu/CourseObjective_9</Entry> </Reference1> <Relationship>http://www.w3.org/2004/02/skos/core#narrower</Relationship> <Reference2> <Catalog>URI</Catalog> <Entry>http://medschool.edu/EventObjective_2.xml</Entry> </Reference2> </Relation> . .. </CompetencyFramework> 2014-2015 Schedule • • • • Staging available now School Portal Opens: August 1 School Portal Closes: September 30 Status e-mail sent to CI Primary Admin, Curriculum Dean, and School Dean: September 30 • Data clean-up/reporting starts: November 1 Resources • Curriculum Inventory Implementers’ Listserv: http://medbiq.org/mailman/listinfo/ciimplementer_medbiq.org • Competency Framework Specs and Schemas: http://www.medbiq.org/working_groups/competencies/index.html • AAMC Curriculum Inventory Developer Resources: https://www.aamc.org/initiatives/cir/about/349624/fordevelopers.html • Physician Competency Reference Set URIs: https://www.aamc.org/external/348732?url=https://www.mededportal.org/icollaborati ve/resource/793 • MedBiquitous: www.medbiq.org • Curriculum Inventory Working Group: http://www.medbiq.org/curriculum_inventory • Competencies Working Group: http://www.medbiq.org/working_groups/competencies/index.html • ‘For Developers’ Section of www.aamc.org/cir Sample XML file and Verification Report from UCSF CI XML Template from UVA Slides/Recordings from Past Training Webinars Registration for Parts II and III of 2015 Developer Webinars www.aamc.org/ Questions? Use Chat to submit questions. • Please send to All Panelists. www.aamc.org/medaps