University Code of Practice Application of the moderation process on programmes delivered by partners leading to awards of the University Document Reference: Identifier: Version: Moderation of Collaborative Provision QH: F19 2 01 Date: Aug 15 Approved By: Originator: ULTAC Learning Enhancement Practice (LEAP) Responsibilities: University departments and faculties Partners and Academic Application to collaborative provision: Mandatory Contacts: quality@hull.ac.uk Applications for exemptions to: Report Exemptions to: ULTAC ULTAC Cross Reference: UK Quality Code for Higher Education – Chapter B6: Assessment of students and the recognition of prior learning (Oct 2013) Summary/ Description: This code sets out the procedures for the use of moderation process by University departments/ faculties on all programmes delivered by partners leading to awards of the University. It devolves the responsibility to University faculties in determining what must be moderated and gives criteria for guiding the decision subject to the minimum standards defined Version 2 01 (Aug 15) makes the following changes: Replaces CDTE with LEAP Version 2 00 (Sep 11) makes the following changes: Changes Partner Institution to Partner Clarifies the two part process, scrutiny of assessment task and review of student output; review of student output previously called scrutiny of the marking process Adds in definitions and role descriptor for Academic Contact Includes proformas for tracking purposes Changes the structure of the code to follow chronological order of the tasks Version 1 02 (Oct 10) recognises the following changes: The revised committee structure Version 1 01 (Oct 06) incorporates formatting changes but makes no changes of substance. Moderation of Collaborative Provision LEAP Version 2 01 – Aug 15 QH:F19:1 This university Code has been written in accordance with the approach approved by ULTAC to enhance clarity (Quality Handbook section A2) involving the following terminology: must = mandatory should = advisable may = desirable. Where these terms are used they are emphasised in bold. This document is available in alternative formats from Learning Enhancement and Academic Practice Moderation of Collaborative Provision LEAP Version 2 01 – Aug 15 QH:F19:2 University Code of Practice Application of the moderation process on programmes delivered by partners leading to awards of the University INTRODUCTION TO THE CODE OF PRACTICE 1. Moderation is a standard feature of collaborative provision, a process that allows the University to assure itself that the standards of its awards are being maintained in those programmes delivered by partners which may have varying degree of experience in Higher Education (HE) and might be characterised by an ethos and mission quite different from that of the University itself. The process risks appearing intrusive and based on an assumption that partner colleagues require additional oversight. The focus for moderation is, however, not on the competence or otherwise of colleagues but on the assurance that students seeking an award from the University are being assessed in a way that is directly comparable to their counterparts on on-campus programmes. What this Code of Practice seeks to achieve as a product of the process of moderation is a clear developmental pathway both for individuals and for the collaborative link in general. 2. Moderation is defined as a specific process that seeks to ensure consistency, fairness and rigour in the assessment of students. Moderation is the process by which the University assures itself that any work undertaken by students on programmes validated by the University is set and assessed by all concerned in a consistent and fair manner, to ensure parity of standards and that the level of achievement of student reflects the required academic standards comparable to programmes at the University and nationally. 3. The moderation process carried out by the University is additional to the partners’ own internal moderation processes and to the role carried out by the University appointed external examiner. Partners must ensure that their internal processes for the approval of assessment tasks and the review of student output are rigorous and not reliant on the University moderation process. SCOPE 4. The Code of Practice applies to summative assessment tasks and student output on programmes delivered by partners (undergraduate and postgraduate) leading to awards of the University of Hull including those directly and indirectly funded by the University. 5. Formative assessment tasks and student output arising out of those tasks do not require moderation by the University. PURPOSE OF THE CODE 6. To ensure that consistent moderation practices are adopted across the University in its quality assurance of programmes delivered by partners 7. To define the roles and responsibilities of those involved in implementing and overseeing the moderation process. 8. To provided guidelines to University faculties in determining what must be moderated. 9. To provide University faculties and partners with a framework which will allow detailed forward planning and scheduling of processes. Moderation of Collaborative Provision LEAP Version 2 01 – Aug 15 QH:F19:3 GENERAL INFORMATION 10. Experience within the University suggests that the key to successful moderation lies partly in the scheduling and management of the processes by the University department. The two central roles are the Academic contact (a designated role that can apply to several academic colleagues who serve for the whole or parts of a collaborative programme) and the agreed University contact (a Faculty or Departmental role normally fulfilled by an administrative colleague). The relevant Faculty must identify the names of these role-holders in advance of the new academic year/session and convey them to the partner. In order to obtain the benefits of experience and continuity, the expectation is that these role-holders will serve for more than one academic year. Role descriptors are at the end of this Code of Practice. 11. Moderation is, however, not the sole responsibility of the University department. The University will require that partners have their own policy on internal scrutiny of assessment tasks and review of student output. The University department can assume that such a policy exists, will be informed of what that policy is and, as part of its own quality assurance processes, will be expected to check that the partner department has operated its institutional policy correctly. That itself should give a level of confidence and should be one starting point for dialogue on the University’s moderation. 12. Dialogue is a perfectly sound form of quality assurance. Where feasible, instead of the University department appearing to re-run the partners internal process, it is good practice for the appropriate person/s from the University department to participate in the partner’s own process. In that way, the University department can: Assure itself of the effectiveness of the partner’s internal moderation process Assure itself of the fairness, consistency and rigour of the assessment Assure itself of the comparability of standards Develop a collegial approach to securing the University’s standards which will diminish any sense of intrusiveness. 13. The collegial, simultaneous approach to moderation can apply both to the scrutiny of assessment tasks and review of student output. 14. Where the level of Higher Education experience in the partner, and a familiarity with the University’s approach to quality and standards and a shared understanding of collaborative practices between the two departments are undeveloped, the code identifies a need for a more intensive form of moderation. 15. In the moderation of non-comparable programmes, the External Academic Consultant fulfils the role of the Academic Contact. THE TWO PROCESSES OF MODERATION 16. The Code of Practice applies to two specific processes that serve as the minimal definition of moderation: 17. Scrutiny of assessment tasks Review of student output In this code, the two processes are explained in full, in chronological order. Moderation of Collaborative Provision LEAP Version 2 01 – Aug 15 QH:F19:4 SCRUTINY OF THE ASSESSMENT TASK 18. Assessment tasks are defined as: examination papers, coursework questions, and assignment briefs i.e. any task which generates an output that will contribute to the overall module mark. It is important to note that this applies to all assessment tasks, not just examination papers. The aim of the scrutiny of assessment task is to ensure that no student risks being at a disadvantage through inappropriate assessment or inaccurate wording, typography or presentation. 19. Typical outcomes of the scrutiny of assessment tasks might be a rewording of an assignment task or question, a clarification of format of an examination paper, replacement of a question in order to better to allow students to demonstrate learning outcome. 20. Assessment tasks, including reassessment tasks, must be approved by the Academic Contact and sent to the external examiner for comment, prior to being released to students (See University Code of Practice External Examining QH :D1 para 29 - 33). The partner may wish to include specific coursework assignment tasks in the relevant module/programme handbooks. The partner and the University department/faculty need to take fully into account the timescale and sequencing of this process in order to meet the deadlines. 21. All assessment tasks for a given module must be submitted for scrutiny at the same time, so that the University can consider and approve, and the External Examiner can consider, the full range of assessment tasks. It is acknowledged that reassessment tasks may have to be submitted at a later date. This must be agreed with the relevant University department. 22. Specific coursework assignments that are negotiated with students are subject to the same approval but the approval may relate to the nature and broad focus of the task. In these cases, the partner must clearly identify which tasks are to be negotiated with students and provide the information required under paragraph 21. Procedure for the scrutiny of assessment tasks 23. The partner must initiate the process by approaching the agreed University contact in the relevant department/faculty prior to the assessment tasks being released to the students (advisably in the module handbook). It is good practice to do this at least 4 weeks in advance. The information given to the University contact must include: Latest version of the Module Specification Name of partner Title of programme Name of module leader Contact details for the module leader eg email address/direct telephone number Period of assessment task eg Semester one or two Type of assessment task eg examination paper, essay titles, practical tasks Module learning outcomes addressed by the assessment task Assessment task weighting Marking criteria Model answers where appropriate Evidence of approval process at the partner 24. A tracking sheet is available as QH:F19 Annexe 1 and is designed to assist the process. Moderation of Collaborative Provision LEAP Version 2 01 – Aug 15 QH:F19:5 25. The agreed University contact must inform the Academic Contact that the material is available to carry out the scrutiny process and provide the Academic Contact with a standard report form for completion. 26. Following scrutiny, the Academic Contact must send the report back to the partner for the appropriate action to be taken (directly or via the agreed University contact). The report will fall into one of three categories: a) b) c) Approved: The assessment task is approved and can be sent to the external examiner Approved subject to minor changes: The assessment task is approved subject to minor changes being made prior to sending to the external examiner Not approved: The assessment task is not approved and substantial changes may necessitate a re-write of the assessment task. 27. The report from the Academic Contact should be returned to the partner within 5 working days of the receipt of the assessment tasks. 28. Where serious issues are raised in the report as in 26c above, in addition to completing the report, the Academic Contact should make direct contact with the module leader and programme leader at the partner to facilitate a discussion to resolve the issues. 29. Once approved by the University Academic Contact, the partner must send the assessment task to the external examiner for his/her comments (see University Code of Practice QH: D1 External Examining paras 29 – 33). 30. The partner must send the final versions of the approved assessment tasks to the agreed University contact for record-keeping purposes. Security of assessment tasks 31. Both the partner and the University must carry out the process in a secure manner to ensure the confidentiality of assessment tasks. Partners and University should monitor and record the despatch and receipt of all documentation relating to assessment tasks to and from each institution and keep copies of the reports generated by the scrutiny process. Where appropriate and with the agreement of both parties, secure electronic methods may be used for sending and monitoring the receipt and despatch of assessment tasks. Please be aware of the security risks of sending examination papers electronically. Password-protected documentation is recommended. Explanatory note: Password-protection for documents The easiest way with a Microsoft document (Word/Excel) is to create a password-protected folder. Instructions to create a password-protected folder are shown at the end of this Code. Moderation of Collaborative Provision LEAP Version 2 01 – Aug 15 QH:F19:6 REVIEW OF STUDENT OUTPUT 32. Student output is defined as: completed examination scripts, coursework, assignments, reports, practical work, dissertations i.e. work by students that will contribute to the overall module mark. 33. A typical outcome of the review of student output might be a rescaling of a whole batch of student outputs in relation to a module. Moderation in assessment works at the macro-level and does not relate to the output of an individual student (as second marking does). Although moderation involves judgement on the appropriateness and effectiveness of the marking system, it is to be expected that moderation should focus particularly on apparently unusual distribution of marks in a module or assessment task, whether that be a high number of first-class marks or fails or an excessive clustering of marks around a norm. The purpose is to ensure that any apparently unusual distribution is explicable in ways that are consistent with maintaining standards. 34. Review of student output, does not involve detailed second marking, even of a sample of student output; it involves rather a judgement on how the marking process has taken into account mechanisms intended to establish fairness, rigour and consistency (including, for example, the use of grade descriptors). 35. The aim of the University’s review of student output is to verify that the internal marking process at the partner, including second marking, is fair and consistent across the programme and within sector norms. Review of student output allows the University to ensure the integrity of the partner’s own procedures and standards and confirms that students have achieved standards comparable to programmes at the University. This process of academic contacts reviewing student output confirms that the standards and internal marking processes are comparable with others across the university. The role complements that of the external examiner checking that the standards are comparable to others in the sector. 36. As a matter of course in all collaborative provision, review of student output will be applied to those modules where there is an apparently unusual distribution of marks in the whole module or assessment task, whether that be a high number of firstclass marks or fails or an excessive clustering of marks around a norm. The partner must identify such modules and the action taken as a result of its own internal moderation processes. The Academic Contact must, through dialogue and sufficient sampling, confirm the effectiveness of the partner’s internal process, and where it is not deemed to be effective, arrive at an agreed set of moderated marks. 37. The modules identified for review under para 36 may not constitute a sufficient sample of the total programme to assure the University of the overall effectiveness of the marking process. What constitutes sufficiency in these terms would depend on aspects of the collaboration. In determining the student output to be reviewed per programme, University faculties, in conjunction with the relevant University departments must consider the following criteria: a) b) c) d) e) The length of time the partnership has been established The length of time the programme has been in operation The higher education experience of the lecturer marking the student output The level of the module and contribution to the overall degree classification The type of student output and the practicalities of implementing the review process eg art exhibitions and performances. Moderation of Collaborative Provision LEAP Version 2 01 – Aug 15 QH:F19:7 38. The following guidelines must be taken into account. Where the level of Higher Education in the partner, a familiarity with the University’s approach to quality and standards and a shared understanding of collaborative practices between the two departments are undeveloped, University departments must, as a minimum, review the student output from: a) b) c) At least 50% of the modules at the certificate and intermediate stages of a Foundation Degree or equivalent At least 50% of the modules at the intermediate and honours stages of an undergraduate programme 100% of modules of a masters programmes. 39. In such circumstances, University faculties must, before the programme starts, agree in writing with partners which student output will be reviewed for the forthcoming academic year. 40. Where the level of HE experience in the partner, a familiarity with the University’s approach to quality and standards, and a shared understanding of collaborative practices between the two departments are all high, the figures under a & b above may be reduced to no less than 20%. In such circumstances, University faculties/departments must, before the programme starts, agree in writing with partners which student output will be reviewed for the forthcoming academic year. 41. Where it is not practical to carry out the review process as detailed in the code because of the nature of the task, University departments/faculties must consider alternative means to facilitate this process such as attending, where possible, art exhibitions, presentations and dance and music performances. Procedure for the review of student output 42. Following first and second marking processes (as set out in the University Code of Practice QH: F1), the partner must make available the whole batch of student output for the modules for review to the agreed University contact prior to the Module Board. This should be provided at least 3 weeks in advance. The student output must be identified separately by each assessment task and module and include: 43. The final version of the assessment task Model answers (where appropriate) Assessment criteria Markers’ report Evidence of second marking and the judgements of second markers Mark grid showing: Title of programme, semester, module title, level, type of assessment, student registration number (and name if anonymity lifted by that stage in the process), plus agreed mark at the partner It is desirable that the total number of scripts in the batch be indicated Normally, the agreed student output is made available at the partner and the agreed University contact arranges for the Academic Contact to visit the institution. If this arrangement is not possible, the partner must send the whole batch of student output for review to the agreed University contact. Upon receipt of the student output, the agreed University contact must send the batch to the Academic Contact to carry out the review process with a standard report form for completion (Annexe 2). The agreed University contact must return the reviewer’s report together with the student output to the partner. Moderation of Collaborative Provision LEAP Version 2 01 – Aug 15 QH:F19:8 44. The report from the Academic Contact should be returned to the partner as soon as possible. Good practice would be to make this no less than 10 working days before the meeting of the Module Board. 45. Any recommendations for alterations to marks by the Academic Contact must be considered by the Module Board and changes made to the mark grids accordingly. Any recommendation for alteration to marks should either be whole cohort adjustment, or part cohort adjustment, for example with a particular part of the mark-range, but would not normally involve adjustment of individual student marks. 46. The partner must ensure that the Academic Contact’s report is included in the student output sent to the external examiner for transparency. Security of student output 47. Both the partner and the University must carry out the process in a secure manner to ensure the confidentiality of student output. Partners and University should monitor and record the receipt and despatch of all student output to and from each institution and keep copies of reports generated by the review process. Any documents sent electronically should be password protected as detailed above TERMS USED IN THE CODE OF PRACTICE Academic Contact – this is the academic contact within the university department responsible for overseeing the programme. Agreed University Contact – this will usually be the Faculty Collaborative Administrator or may be an alternative person nominated by the Dean. This person acting in this role may differ between faculties. External Academic Consultant – this is the person who acts as academic contact for the department for provision that is ‘non-comparable’. This person will be in close contact with the department responsible for the provision and will work with the Agreed University Contact / Faculty Collaborative Administrator to ensure duties are carried out appropriately. HE Higher Education JBoS – Joint Board of Study (See Collaborative Handbook for further information QH:A7) Non-comparable provision – provision that is outside the expertise of the University. In this case the University employs an expert in the subject to act as Academic Consultant for the programme/subject area. Quality Handbook - QH available at: http://www2.hull.ac.uk/administration/policyregister/qualityhandbook.aspx University faculty/department – this is used throughout the code as some University faculties operate their collaborative provision at a faculty level and others at the departmental level. To create a password-protected folder Before you protect a file it’s a good idea to make a copy of it first! In the drive where the document is located open the folder (or the same action can be done on your desktop). Put cursor over the document to be protected. Right click on the document, a drop down list will appear, go to 7-zip, click on add to archive, a box will appear - leave all sections as they are except the password on the right hand side, enter a password and then enter Moderation of Collaborative Provision LEAP Version 2 01 – Aug 15 QH:F19:9 exactly the same again in the re-enter password boxes, then press ok. This will create a password-protected zip file in the same folder as the original document. The zip file can then be sent via email in the usual way. The recipient will need the password to open the document inside the zip file; Remember to send the password to the recipient in a separate email. Academic Contact role descriptor from Collaborative Handbook QH:A7 For each programme or set of programmes within an academic discipline, the University will provide a named contact who will be an academic member of the University faculty nominated as Academic Contact or equivalent. On behalf of the faculty, the Academic Contact will be responsible for those areas identified in the Collaborative Handbook and specifically for: providing guidance to ensure that the programme of study and syllabus is appropriate for the named award of the University prior to PAC approval advising on the appropriate quality processes and on the submission of appropriate documentation overseeing the moderation process including scrutiny of the assessment tasks and review of student output in accordance with the Code of Practice on Moderation (QH:F19) representing the University on the Board of Examiners and advising accordingly where appropriate, attending any final exhibition, performance or other summative outcome advising on the appointment of external examiners and liaising with external examiners accordingly advising on programme development and quality enhancement of the subject, in line with University of Hull expectations of the student experience liaising with Faculty Collaborative Provision administrative staff as necessary over routine administration, and any problems that arise liaising, where appropriate, with Faculty Validation Co-coordinators/the Collaborative Provision Administrator, seeking their advice and guidance on generic quality issues, and keeping them informed on subject specific issues acting as the main contact within the faculty for the purposes of admissions, examinations, annual reporting etc. and liaising, where appropriate through Faculty Collaborative Provision administrative staff, with central University administration on such matters being member of relevant Faculty collaborative committees, and the appropriate JBoS (where they will help to co-ordinate items for discussion) reviewing annually programme specific marketing and recruitment information and reporting this to the JBoS identifying areas for development for University and partner staff providing guidance and support in the preparations for relevant external audits, and representing the area of work in internal and external review processes. For academic disciplines where the University does not offer comparable provision, the University should provide an academic contact to provide the appropriate leadership and management responsibilities and an academic consultant from outside the University to give subject-specific guidance Moderation of Collaborative Provision LEAP Version 2 01 – Aug 15 QH:F19:10