Assuring Learning and Teaching Standards through Inter-Institutional Peer Review & Moderation From Pilot to Implementation The Inter-Institutional Peer Review Model Brief Process Outline • Home Institution with at least one partner • 1 unit of study or capstone unit • Peer review of unit materials can be blind (3+ partners) • Range of assessment samples across 4 grade bands with grades removed • Reviewer “moderates” the 4 samples using home university criteria • Reviewer “Peer Reviews” the Unit materials How Did We Start? Establish A Baseline • In 2013 UWS undertook a university wide stock take of benchmarking activities which included • External Benchmarking & Peer Review Processes (e.g. professional accreditation) •Internal Peer Review Moderation Processes (assessment moderation, double marking, external marking) •Benchmarking of learning standards and/or assessment standards (who/what/when/outcomes) Engaged with Schools From the outcomes of the Stock Take • We were able to compliment internal approaches to assuring academic standards through the inter-institutional peer review model. • Schools nominated a course/unit and possible partner institutions to approach • Support has been provided centrally to facilitate the peer review process, coordination, document exchange and follow up. Identifying Partner Institutions and Reviewers – Partner Institution does not need to have like for like courses – More likely to identify expertise in the discipline area appropriate to review the particular “Unit” – Ideally have “2” Reviewers from each institution where there is only “1” external partner involved – UWS has coordinated the document exchange and offered admin support to both institutions including reporting findings – User Guide outlines confidentiality obligations Review materials Reviewers are provided with • User Guide • “Unit Outline” Template returned with Unit Outline, Unit Assessments, 4 Samples of Student Assessment, Marking Criteria, • “Peer Review Feedback Form” and partner unit materials. • Peer Review Findings Report & Debrief Opportunity Part 1 - Unit Overview Template • • • • Completed by each participating institution Provides context for the Unit within the course Outlines the expected learning outcomes Aligns the unit learning outcomes with the course level outcomes • Includes copies of requested unit materials are attached ie, unit outline, unit assessment items, marking guide, 4 samples of student assessment. Part 2 – Peer Reviewer Feedback Form • • • • Coversheet for each Peer Reviewer Reviews all the attached materials Tick and Comment boxes as appropriate Moderates the samples of student assessments using the marking criteria • The Feedback Form addresses the following questions Part 2 - FEEDBACK FORM SECTION A: YOUR FEEDBACK ON THE UNIT OUTLINE • To what extent does the curriculum content for this unit cover all that a final undergraduate unit on this topic should cover? (select the description that best represents your view) NOT AT ALL SOMEWHAT ADEQUATELY VERY WELL COMPLETELY • Please explain your rating. Please list up to three specific suggestions for improvement where appropriate • To what extent does the unit outline/learning guide explain how the assessment tasks relate to the unit learning outcomes? NOT AT ALL • SOMEWHAT ADEQUATELY VERY WELL COMPLETELY Please explain your rating. Please list up to three specific suggestions for improvement • To what extent does the unit outline/learning guide explain how the assessment tasks relate to the overall graduate outcomes of the degree program? NOT AT ALL SOMEWHAT ADEQUATELY VERY WELL COMPLETELY • Please explain your rating. Please list up to three specific suggestions for improvement where appropriate. • To what extent does the unit outline/learning guide explain clearly (preferably with examples) the requirements for achieving at various grade levels (e.g., what is required to achieve a credit, distinction etc.)? NOT AT ALL SOMEWHAT ADEQUATELY VERY WELL COMPLETELY • Please explain your rating. Please list up to three specific suggestions for improvement where appropriate • What, briefly, are the Best Aspects of the unit outline/learning guide? • Do you have any suggestions for further enhancing the unit outline/learning guide? SECTION B: YOUR FEEDBACK ON THE GRADING GUIDELINES • • In reflecting on the assessment grading guidelines provided for the samples of student work that you are reviewing: To what extent is it clear how student work will be awarded grades at different levels for that assessment task? (Please select) NOT APPLICABLE NOT AT ALL SOMEWHAT ADEQUATELY VERY WELL COMPLETELY • Please explain your rating. Please list up to three specific suggestions for improvement where appropriate. • To what extent are the grading criteria at an appropriate level for a final year undergraduate unit of study in this field of education? (Please select) NOT APPLICABLE • NOT AT ALL SOMEWHAT ADEQUATELY VERY WELL COMPLETELY Please explain your rating. Please list up to three specific suggestions for improvement where appropriate. SECTION C: YOUR FEEDBACK ASSESSMENT TASK/S • • In reviewing the list of assessment tasks which students have to complete in the unit of study: To what extent is the range of assessment tasks suited to assessing the key learning objectives listed in the unit outline? (Please select) ASSESSMENT NOT AT ALL SOMEWHAT ADEQUATELY VERY WELL COMPLETELY 1 2 3 4 5 Please explain your rating. Please list up to three specific suggestions for improvement where appropriate. Comments SECTION D: YOUR OVERALL FEEDBACK ON THIS PROCESS • Please provide brief feedback on this peer review process as a collegial way to monitor and assure standards in common units of study between different universities. • What, briefly, are the best aspects of this peer review process? • Which aspects of this peer review process do you think we could improve and how might this be achieved? SECTION E: MODERATION OF STUDENT ASSESSMENT SAMPLES • • • (Completed by: Unit Coordinators and/or Support Persons in each Partner University) The Partner Institution has provided 4 samples of student assessments for external moderation. Using the grading guides / criteria sheets from the Partner Institutions please mark and grade the samples. Please record your marks and final grade in the table below. You may like to make comments regarding the rubric and grading criteria that will be useful to discuss during the debrief between the partner institutions. Student Sample Number Mark Grade (F/P/C/D/HD0 Assessment 1 Assessment 2 Assessment 3 Assessment 4 • SECTION F (Optional Questions for peer review) Comments The Peer Review and Moderation Model Provides a relevant and feasible way to assure the validity, reliability and comparability of assessment outcomes and achievement standards in similar institutions Provides a validated approach for assuring subject achievement standards Outcomes of Peer Review • Each institution can determine how they utilise the findings of the Peer Review. • A debrief bringing together the reviewers has been beneficial to discuss the findings • Some accreditation bodies have been interested in the peer review process findings Lessons Learned • Coordinator required to manage process and monitor progress • Timing of timelines, delays, staff leave, follow ups. BE FLEXIBLE but not too FLEXIBLE • Partner Institution cooperation and awareness • Dealing with various assessment types • Blind Peer Review not considered essential. NEXT STEPS -The online tool being developed by UTas may streamline the peer review process for the reviewer and administrator. - Sector wide involvement in a peer review model would facilitate easier identification of expertise -The model is very flexible and can accommodate needs specific to institutions or disciplines.