Curriculum Inventory Workshop Terri Cameron, AAMC Valerie Smothers, MedBiquitous Ye Chen, University of Virginia James Galt, Rutgers Robert Wood Johnson Medical School Saniya Saifee, New York University School of Medicine Workshop Objectives By the end of the workshop, participants should be able to: • Describe and implement the CI Specification, Competency Framework, AAMC Business Rules, and Developer Resources • Create XML code that includes PCRS to Program Competencies Mapping • Create XML code that includes Event to Program Competency Mapping • Create XML code that includes Sequence Block and Event Data Elements • Upload a successful XML file to CI Staging MedBiquitous Curriculum Inventory V1 • Working Group launched in December 2010 • Susan Albright and Marc Triola, co-chairs 20102013 • Charter: “to develop XML standards for the exchange of curriculum data for benchmarking and educational research.” (Not just for AAMC Curriculum Inventory) • Reasons for Undertaking Work: • • • Enable local curriculum management systems to upload data to a central Curriculum Inventory Identify curricular trends Support curriculum benchmarking and reform http://medbiq.org/curriculum/vocabularies.pdf http://www.medbiq.org/sites/default/files/files/CurriculumInventorySpecification.pdf http://ns.medbiq.org/curriculuminventory/v1/curriculuminventory.xsd http://medbiq.org/sites/default/files/files/CI_ImplementationGuidelines.pdf https://www.aamc.org/download/348854/data/cibusinessrules.pdf Curriculum Inventory E J B 1 F M3 I E J O 2 N K C A GL D H P C Q GR S H A I Expectations 1 2 Events 3 7 4 8 5 9 6 10 11 Sequence blocks Integration blocks Competency framework Events I A E B C D J F G H K L • A curriculum is made up of many educational events • Events have metadata, descriptive information about the events • IDs required! Expectations 3 1 2 4 7 10 6 5 8 11 9 12 • Curricula may reference objectives , competencies, learning outcomes, or milestones. These are called competency objects. • IDs required! Competency Objects and Events 1 G 2 K 2 6 • An event may be intended to address one or more competency objects (learning objectives, competencies, and milestones) • A competency object may be associated with many events Sequence Blocks I A E B F 1 G D 2 K C A J 7 L H • Events can be organized into blocks of instruction, called Sequence Blocks (think course/module/ year/phase/clerkship etc) • A recurring event may be associated with more than one Sequence Block • Sequence blocks may be associated with competency objects (competencies, objectives, etc) • Sequence blocks have metadata, too • IDs required! Integration Blocks A I E J B 1 F K C A G L D H M3 I E J O 2 N P C Q G R S H • Curricular themes, or Integration Blocks, may apply to many sequence blocks, events, or objectives/ competencies and describe how the curriculum is semantically integrated. • An event or block may link to many Integration Blocks and an Integration Block may link to many events, blocks or competency objects • IDs required! Sequence A I E J B 1 F K C A G L D H M 3 I E J O 2 N P C Q G R S H • Sequence blocks can be put together to form a the main structure of the curriculum (a sequence of courses, modules, phases, clerkships) Sequence Block Nesting I A E J B F 2 1 I A E J B F 2 1 I A E J B F 2 1 I A E J B F 1 2 • Sequence blocks can be nested: a module or clerkship might be in a year which is within a phase and so on … • Blocks will inherit any relations to competency objects from their sub-blocks • No restriction on the number of nest levels • Recursion explicitly forbidden Types of blocks: ordered Block Sequence=ordered Required=true Block Required A E B I J F Block Required R D C J Q Z Block Required T Y A X M B - All sub-blocks within an ordered block happen with a prescribed sequence Types of blocks: unordered Block Sequence=unordered Required=true Block Required A E B I J F Block Required T Y A X M B Block Required R D C J Q Z Block Required L C D K N S - All sub-blocks within an unordered block happen for all students, but students may encounter them in a variable order Types of blocks: selectives - Every student must participate in [specify number] of the blocks, and the order is variable Block Required = Required Order=Unordered Minimum=1 Block Optional A E B I J F Block Optional OR R D C J Q Z Block Optional OR T Y A X M B Types of blocks: electives - Students may or may not participate in any of the blocks Sequence Block Required = Optional Order =Unordered Minimum=0 Sequence Block Optional I A E J B F OR Sequence Block Optional R C J D Q Z OR Sequence Block Optional T A X Y M B Types of blocks: parallel - All sub-blocks within a parallel block happen at the same time, but are distinct educational units Sequence Block Required = Required Order = Parallel Sequence Block Required R D C J Q Z Sequence Block Required I A E B J F Clerkships, required = Required, minimum = 2, maximum = 2 Level 1 Main Track, required = Optional, track = true Med/Geri 1 84 days required in track Pediatrics 1 42 days required in track Surg 1 56 days required in track Level 1 Common Clerkships, required = Required NeuroScience 28 days optional Ambulatory 1 28 days required in track Elective 21 days required Neurology 28 days required Anesthesiology 7 days required Psychiatry 28 days required Level 1 Ambulatory Track, required = Optional, track = true Med/Geri 2 63 days required in track Pediatrics 2 28 days required in track Surg 2 42 days required in track Ambulatory 2 77 days required in track Ob/Gyn 42 days required Level 2 Clerkships, July 4, 2011– April 27, 2012 Level 2 Clerkships Variable time, July 4, 2011 – April 27, 2012 order = unordered Subinternship required 28 days Emergency Med required 28 days Anatomic Radiology Required 14 days or online course Critical Care Required 14 days Electives 147 days (21 are usually taken during level 1 and the rest during the level 2 variable time Level 2 Clerkships, second half of the year, January 1, 2012 – April 27, 2012 Into to Internship required 14 days A competency object . . . Competency Framework Statement Relates to other competency objects . . . Which can relate to external resources . . . Learning Object Assessment Performance data Business Rules Program-level competencies are required and must be mapped to PCRS using #related All competencies, including PCRS, program-level, sequence-block level, and event level competencies used in a <Relation> tag must have an <Includes> tag Spec Required Elements Curriculum Inventory ReportID Institution Program Events • Event Title EventDuration in hours and min Instructionalmethod (primary=true/false) or Voca AssessmentMethod (purpose=formative/ summative) b ids Title ReportDate ReportingStartDate ReportingEndDate Language Description Expectations • CompetencyObject identifier title Category * AcademicLevels Sequence * SequenceBlock Requirements Attributes: id, required Title Timing (either duration in days or start and end dates – clerkships must specify both) Level SequenceBlockEvent SequenceBlockReference Matching Program Objectives to PCRS (Crosswalk) PCRS Program Objectives 2. Knowledge for Practice: Demonstrate knowledge of established and evolving biomedical, clinical, epidemiological and social-behavioral sciences, as well as the application of this knowledge to patient care. 2.1 Demonstrate an investigatory and analytic approach to clinical situations 2.2 Apply established and emerging bio-physical scientific principles fundamental to health care for patients and populations 2.3 Apply established and emerging principles of clinical sciences to diagnostic and therapeutic decisionmaking, clinical problem-solving, and other aspects of evidence-based health care 2.4 Apply principles of epidemiological sciences to the identification of health problems, risk factors, treatment strategies, resources, and disease prevention/health promotion efforts for patients and populations 2.5 Apply principles of social-behavioral sciences to provision of patient care, including assessment of the impact of psychosocial and cultural influences on health, disease, care-seeking, care compliance, and barriers to and attitudes toward care 2.6 Contribute to the creation, dissemination, application, and translation of new health care knowledge and practices 2.99 Other knowledge for practice 2. Demonstrate a broad working knowledge of the fundamental science, principles, and processes basic to the practice of medicine. 2.1 Understand the clinical relevance of scientific inquiry. 2.2 Apply this knowledge in a judicious and consistent manner to prevent common health problems and achieve effective and safe patient care. 2.3 Demonstrate the ability to evaluate emerging knowledge and research as it applies to diagnosis, treatment and the prevention of disease. 2.4 Utilize state of the art information technology and tools to retrieve, manage and use biomedical information in the care of individuals. 2.5 Utilize state of the art information technology and tools to retrieve, manage and use biomedical information in the care of populations. 2.6 Understand the indications, contraindications, and potential complications of common clinical procedures. 2.7 Perform the basic clinical procedures expected of a new PGY-1. PCRS Mapping and Sequence Block or Event-level Competencies • Mapping of Sequence Block-level or Event-level objectives / competencies directly to PCRS is not allowed. • Sequence Block and / or Event Objectives / Competencies, if mapped, are mapped to each other and to Program Objectives / Competencies Competency Object Rules • <Competency Object> tags must be used to identify competencies, along with their <Category> (level) • Program-level • Sequence block-level • Event-level • Competency object URIs must be unique • <Competency Object> tags must not be used to identify PCRS competencies Competency Objects and the Curriculum Structure • All sequence block-level and event-level competencies must be referenced by at least one Sequence Block or Event (that was identified in the curriculum structure) Competency Framework Curriculum Inventory PCRS 6.99 PCRS 1.5 skos: related skos: related P2 skos: narrower P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency S5 The Sequence Block References the Program-level Competency The Sequence Block References the Sequence Block-level Competency I P2 E B S5 F J A Circular References • Relationships must not result in circular references. • A circular reference occurs when a child element draws a hierarchical relationship to its parent that infers the parent is lowerlevel than the child. PCRS 6.99 PCRS 1.5 skos: related skos: related P2 P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency skos: narrower S1 skos: narrower skos: narrower S2 skos: narrower S3 This relationship is not allowed because it creates a “Circular Reference” and violates a Business Rule. skos: narrower E4 Other notes • Set the schema location to public copy: xsi:schemaLocation="http://ns.medbiq.org/curriculuminventory/v1/ http://ns.medbiq.org/curriculuminventory/v1/curriculuminventory.xsd" • Use namespace prefixes for competencies and competency framework • Use AAMC ids for schools • Competency Object identifiers must be URIs 1. 2. URLs do not have to be resolvable. As long as they uniquely identify, it will work. Add anchors to the URL so that each objective can be distinguished using the URL with the anchor included. • Duration uses xsd format, for example, P40D, PT2H • 1 week = 5 days PCRS --Physician Competency Reference Set “…a list of common learner expectations” •What competencies schools are incorporating into their curricula; •Where in their curricula schools are incorporating expectations and competencies; •How schools are teaching and assessing competencies; and •In what context and/or content competencies are being taught. Step 1: Match Competencies Step 2: Create XML File <Expectations> <CompetencyObject> </CompetencyObject> <CompetencyFramework> <cf:Includes> </cf:Includes> <cf:Includes> </cf:Includes> <cf:Relation> </cf:Relation> <cf:Relation> </cf:Relation> </CompetencyFramework> </Expectations> Insert one of your program competencies <CompetencyObject> <lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry>http://libraries.rutgers.edu/rwj/RWJMS_Co mpetencies2.html#V.A.2.b</lom:entry> </lom:identifier> <lom:title> <lom:string>Students will convey information honestly and tactfully; will engage in truthful interactions with patients, peers, and in professional work. </lom:string> </lom:title> </lom:general> </lom:lom> <co:Category term="program-level-competency"/> </CompetencyObject> <Expectations> <CompetencyFramework> Insert one of your program competencies <cf:Includes> </cf:Includes> Insert one PCRS <cf:Includes> </cf:Includes> </CompetencyFramework> </Expectations> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry>http://libraries.rutgers.edu/rwj/RWJMS_Competencies2.html#II.B </cf:Entry> </cf:Includes> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry>https://services.aamc.org/30/ci-schoolweb/pcrs/PCRS.html#aamc-pcrs-comp-c0108</cf:Entry> </cf:Includes> <Expectations> <CompetencyFramework> Show the relationship between your <cf:Relation> program competency & PCRS </cf:Relation> </CompetencyFramework> </Expectations> <cf:Relation> <cf:Reference1> <cf:Catalog>URI</cf:Catalog> <cf:Entry>http://libraries.rutgers.edu/rwj/RWJMS_Competencies2.html#I.C.2.d</ cf:Entry> </cf:Reference1> <cf:Relationship>http://www.w3.org/2004/02/skos/core#related</cf:Relationship> <cf:Reference2> <cf:Catalog>URI</cf:Catalog> <cf:Entry>https://services.aamc.org/30/ci-school-web/pcrs/PCRS.html#aamcpcrs-comp-c0108</cf:Entry> </cf:Reference2> </cf:Relation> Lessons Learned •Some benefit to moving a bit slow •Having more than one person understanding the specifications is important •Humans are not the weakest link •Networking is vital •Carving out dedicated time is crucial •Explaining the big picture to faculty is important •Developing a functioning end product is vital •Developing tools not just a product is key for the future Show Events Mapping in XML File Ye Chen yc5x@virginia.edu XML File Structure <CurriculumInventory > <ReportID> </ReportID><Institution></Institution> <Program> </Program><Title> </Title> <ReportDate> </ReportDate> <ReportingStartDate> </ReportingStartDate> <ReportingEndDate> </ReportingEndDate> <Language> </Language> <Description> </Description> <Events> <Event> </Event> </Events> <Expectations> <CompetencyObject> </CompetencyObject> <CompetencyFramework> </CompetencyFramework> </Expectations> <AcademicLevels></AcademicLevels> <Sequence> <SequenceBlock > </SequenceBlock> </Sequence> </CurriculumInventory> Events and Competencies Event (E1): White Coat Ceremony Event (E2): Orientation Lecture Describe the three functions http://nosuchdomain.edu/event_competency/4 Event (E3): Peds Part 1 Event (E4): Peds Part 2 List the organization, http://nosuchdomain.edu/event_competency/5 Gather, synthesize, and organize patient information. http://nosuchdomain.edu/course_competency/3 Professionalism http://nosuchdomain.edu/program_competency/1 Perform all medical, diagnostic http://nosuchdomain.edu/program_competency/2 List All Competencies <Expectations> <CompetencyObject> List competencies </CompetencyObject> <CompetencyFramework> </CompetencyFramework> </Expectations> <CompetencyObject> <lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry>http://nosuchdomain.edu/program_competency/1</lom:entry> </lom:identifier> <lom:title> <lom:string>Professionalism</lom:string> </lom:title> </lom:general> </lom:lom> <co:Category term="program-level-competency"/> </CompetencyObject> Repeat the same format for other 4 competencies This value will be refered in several places (event, include, and relationship) Show the relationships among competencies <Expectations> <CompetencyObject> List competencies </CompetencyObject> <CompetencyFramework>Show relationships</CompetencyFramework> </Expectations> Include all program level competencies, include sequence block level competencies and event level competencies if they have mappings with other competencies. other information for competencyFrameWork <lom:lom>------</lom:lom> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry>http://nosuchdomain.edu/program_competency/1</cf:Entry> </cf:Includes> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry>http://nosuchdomain.edu/program_competency/2</cf:Entry> </cf:Includes> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry>http://nosuchdomain.edu/course_competency/3</cf:Entry> </cf:Includes> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry>http://nosuchdomain.edu/event_competency/4</cf:Entry> </cf:Includes> Show the relationships among competencies (cont.) Describe the three functions http://nosuchdomain.edu/event_compete ncy/4 Gather, synthesize, and organize patient information. http://nosuchdomain.edu/course_competency/3 program level comptetencies to PCRS…….. Professionalism http://nosuchdomain.edu/program_competency/1 <cf:Relation> <cf:Reference1> <cf:Catalog>URI</cf:Catalog> <cf:Entry>http://nosuchdomain.edu/program_competency/1</cf:Entry> </cf:Reference1> <cf:Relationship>http://www.w3.org/2004/02/skos/core#narrower</cf:Relationship> <cf:Reference2> <cf:Catalog>URI</cf:Catalog> <cf:Entry>http://nosuchdomain.edu/course_competency/3</cf:Entry> </cf:Reference2> </cf:Relation> <cf:Relation> <cf:Reference1> <cf:Catalog>URI</cf:Catalog> <cf:Entry>http://nosuchdomain.edu/course_competency/3</cf:Entry> </cf:Reference1> <cf:Relationship>http://www.w3.org/2004/02/skos/core#narrower</cf:Relationship> <cf:Reference2> <cf:Catalog>URI</cf:Catalog> <cf:Entry>http://nosuchdomain.edu/event_competency/4</cf:Entry> </cf:Reference2> </cf:Relation> Map event to competencies Only show the direct mappings. Non direct mapping is taken care by the relationship. Event (E1): White Coat Ceremony Event (E2): Orientation Lecture Describe the three functions http://nosuchdomain.edu/event _competency/4 List the organization, http://nosuchdomain.edu/event _competency/5 Event (E3): Peds Part 1 Event (E4): Peds Part 2 Perform all medical, diagnostic http://nosuchdomain.edu/progra m_competency/2 <Event id="E1"> <Title>White Coat Ceremony</Title> ---------<CompetencyObjectReference>/CurriculumInventory/Expectations/CompetencyObject[lom:lom/lom:general/lom:identifier/lom: entry='http://nosuchdomain.edu/event_competency/4’]</CompetencyObjectReference> <InstructionalMethod primary="true">IM013</InstructionalMethod> </Event> /CurriculumInventory/Expectations/CompetencyObject[lom:lom/lom:genera l/lom:identifier/lom:entry= ] 'http://nosuchdomain.edu/event_competency/4’ inside <Event id=“E2”><Event id=“E3”>, there should be <CompetencyObjectReference> with different entry value. Curriculum Inventory Event and SequenceBlock Saniya Saifee NYU School of Medicine Event Event contains sub elements that describe a single education or assessment event Event-Subelements 0..∞ 0..∞ 0..∞ InstructionalMethod Required AssessmentMethod Optional Event XML <Events> <Event id="E1"> <Title>White Coat Ceremony</Title> <EventDuration>PT2H</EventDuration> <Description>First year students are given </Description> <Keyword hx:source="MeSH" hx:id="D010817"> <lom:string>Physician-Patient Relations</lom:string> </Keyword> <Interprofessional>false</Interprofessional> <CompetencyObjectReference>/CurriculumInventory /Expectations/CompetencyObject[lom:lom/lom:general/lom:identifier/lom:entr y='http://nosuchdomain.edu/zxcvb'] </CompetencyObjectReference> <InstructionalMethod primary="true"> IM013 </InstructionalMethod> </Event> <Events> Competency Object <Expectations> <CompetencyObject> <lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry>http://nosuchdomain.edu/ event_competency/4 </lom:entry> </lom:identifier> <lom:title> <lom:string>Describe the three functions of the medical interview </lom:string> </lom:title> </lom:general> </lom:lom> <co:Category term="event-level-competency"/> </CompetencyObject> </Expectations> Events-checklist An event must have at least one instructional OR assessment method. An event can only be reported if we can report timing. An event may have multiple instructional or assessment methods, but if it’s an instructional session, only one method can be primary. All assessment methods must be marked as formative or summative. Within <Event> tags, the <AssessmentMethod> and <InstructionalMethod> tags must reference a unique ID from their respective vocabularies SequenceBlock B A B A D D When 1 F E I F E Combined I 2 G H G H 3 Events Expectations Sequence Block SequenceBlock-Subelements Title Timing ID Level SequenceBlock ID Required ClerkshipModel 0..∞ CompetencyObjectRef… 0..∞ SequenceBlockEvent Required Optional 1..∞ SequenceBlockReference Timing Timing Date StartDate Duration EndDate SequenceBlockEvent EventReference required SequenceBlockEvent StartDate EndDate AcademicLevels <AcademicLevels> <LevelsInProgram>2</LevelsInProgram> <Level number="1"> <Label>Year 1</Label> <Description>Preclinical</Description> </Level> <Level number="2"> <Label>Year 2</Label> <Description>Preclinical</Description> </Level> </AcademicLevels> Sequence Block <Sequence> <SequenceBlock id="e" required="Required"> <Title>Introduction to Medicine</Title> <Timing><Duration>P1M</Duration></Timing> <Level> /CurriculumInventory/AcademicLevels/Level[@number='1'] </Level> <SequenceBlockEvent required="true"> <EventReference> /CurriculumInventory/Events/Event[@id='E1'] </EventReference> </SequenceBlockEvent> </SequenceBlock> </Sequence> Nested SequenceBlock <SequenceBlock id="c" required="Required"> <Title>Year 1</Title> <Timing> <Duration>P12M</Duration> </Timing> <Level> /CurriculumInventory/AcademicLevels/Level[@number='1'] </Level> <CompetencyObjectReference> /CurriculumInventory/Expectations/CompetencyObject [lom:lom/lom:general/lom:identifier/lom:entry= 'http://nosuchdomain.edu/course_competency/3'] </CompetencyObjectReference> <SequenceBlockReference> /CurriculumInventory/Sequence/SequenceBlock[@id='e'] </SequenceBlockReference> <SequenceBlockReference> /CurriculumInventory/Sequence/SequenceBlock[@id='f'] </SequenceBlockReference> </SequenceBlock> SB-checklist A SequenceBlock (Year, course, module) can only be reported if it has at least one “valid” event. The id attribute in the SB must be unique. SB should not be associated with a level number that has not been defined in <AcademicLevels> The <Timing> subelement of <SequenceBlocks> tags with a ClerkshipModel attribute must specify <Dates> and <Duration> Is this a valid Event? <Event id="E2"> <Title>Orientation Lecture</Title> <EventDuration>PT4H</EventDuration> <Description> Faculty discuss what the students are to expect in the coming year. </Description> <InstructionalMethod primary="true">Lecture </InstructionalMethod> </Event> It is not a valid event <Event id="E2"> <Title>Orientation Lecture</Title> <EventDuration>PT4H</EventDuration> <Description> Faculty discuss what the students are to expect in the coming year. </Description> <InstructionalMethod primary="true">Lecture </InstructionalMethod> </Event> IM013 Best Practices and project insights Saniya Saifee NYU School of Medicine Process followed Software program that creates XML Curriculum Managemen t System (CuRIE) Refactor And repeat AAMC Business rules errors CI Portal <CurriculumInventory > …….. …. </<CurriculumInventor y Validate Send XML XML against XSD Lessons learned -1 <CurriculumInventory > …. “Including ALL Academic levels, courses, events, Expectations in your first attempt while uploading file on CI portal” …. </CurriculumInventory > <CurriculumInventory > …. “Including ONE Academic levels, courses, events, Expectations in your first attempt while uploading file on CI portal” …. </CurriculumInventory > Lesson learned -3 Create a checklist of minimum requirements Lesson learned -3 Lesson learned - 4 Keep doing frequent and repeated submission CI Process • Validate XML against MedBiq CI Standard • Upload to CI • Successful submission results in ‘Submitted Status’ • Log into School Portal • Download and Review Verification Report • Download results in ‘Review Status’ • Reject or Verify Data • Rejection returns Status to ‘No Response’ • Verification results in ‘Verified Status’ Resources Curriculum Inventory Implementers’ Listserv: http://medbiq.org/mailman/listinfo/ciimplementer_medbiq.org Competency Framework Specs and Schemas: http://www.medbiq.org/working_groups/competencies/index.html AAMC Curriculum Inventory Developer Resources: https://www.aamc.org/initiatives/cir/about/349624/fordevelopers.html Physician Competency Reference Set URIs: https://www.aamc.org/external/348732?url=https://www.mededportal.org/i collaborative/resource/793 SKOS: Simple Knowledge Organization System http://www.w3.org/2004/02/skos/ MedBiquitous: www.medbiq.org Curriculum Inventory Working Group: http://www.medbiq.org/curriculum_inventory Competencies Working Group: http://www.medbiq.org/working_groups/competencies/index.html Questions?