Assessment Strategies for Institutional and Undergraduate Programs UHM, July 2007 Dr. Harry Hubball, Department of Curriculum Studies University of British Columbia Canada Outline UHM, June 28 • Global Context for Curricula Reform in Higher Education • Theory-Practice Integration: Innovative Learning Strategies • Critical Challenges, Stages, Curriculum/Faculty Support Initiatives UHM, July 25th • Program-level Assessment and Learning Outcomes • Action Research Methodology: Case Study Findings in College, institutional and Provincial Settings • The Scholarship of Curriculum Practice and Undergraduate Program Reform: Dissemination (PAIMAP) Stages of Curricular Reform Practice Stage Action Plan Stage Mobilisation Stage Initiative Stage Awareness Stage Pre-Awareness Stage Hubball & Burt, 2004 The WASC/Hawaii Department of Education Criteria 1. Organization: Vision and Mission (Purpose); School Culture; Governance, Leadership, and Resources 2. Standards-based Student Learning: Curriculum (What are students learning?) 3. Standards-based Student Learning: Instruction (How are students learning?) 4. Standards-based Student Learning: Assessment and Accountability (How is student assessment used?) 5. Quality Student Support for Student Personal and Academic Growth http://www.acswasc.org/about_criteria.htm#hawaiicriteria SoCP . . .An approach to undergraduate program reform and curriculum development in higher education that integrates research, teaching and learning 1. Scholarly Approaches to Curriculum Practice “On-going professional development, reflection and initiation of positive changes to curricula practices” ….Taking this to the next level of rigour 2. The Scholarship of Curriculum Practice (SoCP) “Dissemination of practice-driven curricula research in peerreview contexts” (Hubball & Burt, 2007; Hubball & Gold, In press) Action/Classroom Research Methodology: A Scholarly Approach to Programming and Curriculum Practice • • • • • • • • systematic, intentional and reflective inquiry Identify research problems/questions/hypothesis Literature review and conceptual framework Planning, action, observation and reflection Research design, methodology, data analysis, conclusions Theory-practice integration Implementing change/dissemination Individual and social contextual process (Altrichter, Psch, & Somekh, 1993; Mills, 2000; Peterat & Smith, 2001; Thompson, 1996; Winter, 1996) Integrating Theoretical Frameworks (SoCP) • • • • Models of Context-based Learning Models of Learning Communities Models of Learning-centred Curricula and Pedagogy in Higher Education Models of Action Research and SoTL (Cox & Richlin, 2004; Hansman, 2001; Green & Kreuter, 1999; Hubball & Burt, 2004, 2006; Hubball & Poole, 2004; Lave & Wenger, 1991;Peterat & Smith, 2001; Pratt & Asociates, 2001) Organisational Structure Needs Assessment Resources LEARNING CONTEXT SOCIAL CONTEXT PLANNING AR Process - Impact - Follow-up ASSESSMENT PROGRAMMING (Hubball & Burt, 2004) A Learning-centred Framework for Curriculum Development and Evaluation Program Completion Program Start E-3 E-1 E-2 PROGRAM EVALUATION PHASES CONTEXT CONSIDERATIONS: Audience, Objectives, Resources for Evaluation Projects E-1: Learning Context Evaluations E-2: Process Evaluations E-3: Impact Evaluations E-4: Follow-up Evaluations (Green & Kreuter, 1999; Hubball & Burt, 2007;Priest, 2001) E-4 Table 2. Implementation Analysis ______________________________________________ Q. 1 What are critical factors when developing programlevel learning outcomes? Q. 2 To what extent are learning outcomes reflected in program learning experiences? Q. 3 When and how do students demonstrate learning outcomes in this context? Q. 4 What are the overall reflections for implementation and alternative applications of learning outcomes to other academic activities in this context? Action Research Methodology: Data Collection & Analysis Methods (Quantitative and Qualitative) • • • • Minutes of Curriculum meetings Focus group meetings with curriculum stream leaders’ Review of course syllabi and samples of students’ work Review of course evaluation, student and pharmacists’ survey data • Review of faculty research activities related to SoTL GRADUATION External Clerkships Integrated Case-based Courses Yr 4 Courses CAPS Yr 3 Courses CAPS Program Sub-Disciplines Yr 2 Courses CAPS Pre-requisite courses from Arts and Sciences CAPS Ability-based Outcomes & Assessment Strategies Yr 1 Courses PETE 314/320: ASSESSMENT PROFILE Methods Start Presentation Formative Summative Learning O/C End 1-2,4-8 Participation 2-3 Final/Review 1-2,5-8 Unit Project 1-2,5-8 File/Journal 1,2, 4,9 Research Outcomes & Key Lessons Learned: SoCP and Implementation Analysis • Accreditation was the single biggest factor to influence the implementation of program-level learning outcomes • Strong (and adequately supported) curriculum leadership and the ability to engage the WHOLE learning community (including a critical mass within the sub-disciplines), through open dialogue and various communications and dissemination of SoCP • Flexibility to align bottom-up and top-down LO processes • Guest speakers and external consultants • Substantial time, effort and varying degrees of contribution and responsibility requires comprehensive Institutional and Faculty-level support/structures - Surface & deep levels of SoCP • Additional support required to target, champion and show-case best practices (innovation-leadership-integration) Hubball & Gold, 2007 SUPPORTING CURRICULAR & TEACHING CONTRIBUTIONS IN ORDER TO REALISE INSTITUTIONAL GOALS INSTITUTIONAL AND FACULTY LEVELS • Tenure and Promotion Process • Curriculum Leadership Awards • Innovative Course Design Awards • SoCP Grant Funding Opportunities • SoCP Excellence Awards • Faculty Certificate Program: The Scholarship of University Teaching and Learning (SoTL) • Curriculum Development and Pedagogy Support Service PROGRAM-LEVEL ASSESSMENT AND LEARNING OUTCOMES Thank you….DISCUSSION - Welcome! * Questions? * Challenges / Alternative Strategies? * Comments/reflections?