Access to HE Diploma Handbook for External Moderators 2015/16 – Version 2 (15/12/15) Contents Section 1: Introduction and Overview .................................................................................... 4 The Purpose of Moderation ............................................................................................... 4 The Role of the External Moderator ................................................................................... 5 Overview of External Moderation Model ............................................................................ 6 Internal Moderation............................................................................................................ 7 Purpose and Focus of the External Moderation Visits........................................................ 7 Recognising Good Practice ............................................................................................... 8 Identifying Poor Practice .................................................................................................... 8 Responsibilities of the AVA to Access to HE External Moderators ..................................... 9 Section 2: Timetable of Events ........................................................................................... 10 Section 3: The First Visit .................................................................................................... 12 Section 4: The Final (2nd) Moderation Visit ......................................................................... 14 Section 5: Sampling Guidance for External Moderators ..................................................... 16 Section 6: The Awards Board............................................................................................. 17 Section 7: Writing the External Moderation Report ............................................................. 18 Section 8: The Role of the Lead Moderator ........................................................................ 19 Section 9: Supplementary and Support Visits .................................................................... 20 Supplementary Visits ....................................................................................................... 20 Support Visits .................................................................................................................. 20 Section 10: QAA’s Licensing Criteria (relating to Moderation Practice) .............................. 21 Section 11: Summary of External Moderator Roles ............................................................ 29 Section 12: Code of Conduct for External Moderators........................................................ 33 Section 13: Maintaining Records of Continuous Professional Development ....................... 34 Section 14: Monitoring of External Moderator’s Performance ............................................. 36 Certa Access Diploma External Moderators Handbook - 2015/16 2 Certa Access Diploma External Moderators Handbook - 2015/16 3 Section 1: Introduction and Overview Certa (formerly Open College Network Yorkshire & Humber) is an Access Validating Agency (AVA) licensed by the Quality Assurance Agency (QAA), to develop and quality assure Access to HE Diplomas. The QAA provides an integrated quality assurance service for Higher Education in the UK and is also responsible for assuring the quality of approved Access to HE Diplomas through the ‘QAA Recognition Scheme for Access to HE in England, Wales and Northern Ireland’. This handbook aims to provide External Moderators with information relating to: i. ii. iii. iv. the purpose of moderation, the roles and responsibilities of the External Moderator, the procedures to be undertaken, and the completion of documentation. This handbook should be read in conjunction with the following documents: Certa Access to HE Handbook (primarily written for providers) QAA’s AVA Licensing Criteria for the Licensing and Relicensing of AVAs’ May 2011 (up-dated July 2014) Certa’s Moderation Protocol The Access Team can be contacted by email at: access2he@ocnyhr.org.uk The Purpose of Moderation Precept 5: Moderation, monitoring and certification of the QAA Access to HE Licensing Criteria relates specifically to the quality assurance and enhancement of Access to HE courses and academic standards. The quality model operated by Certa is designed to ensure compliance with the QAA Licensing Criteria and to meet the requirements of the AVA. External Moderators are key personnel in the quality assurance processes of Certa provide an important link between the Access to HE provider and Certa operate with due regard to objectivity, impartiality, fairness and equity are appointed annually demonstrate a sufficient degree of independence to be able to moderate the Access to HE Diploma(s) to which they are appointed. There will be no conflict of interest. All External Moderators must be external to the provider and not hold a position in any receiving institution, Certa Access Diploma External Moderators Handbook - 2015/16 4 The Role of the External Moderator It is essential that External Moderators carry out their roles and responsibilities in a professional manner and to the highest possible standard. All Certa External Moderators are curriculum specialists and are appointed to appropriate Diplomas. All Certa Access to HE Diplomas are subject to external moderation. At least one External Moderator will be appointed to each Access to HE Diploma offered by each provider. Where additional curriculum expertise is required, additional External Moderators will be appointed. The role of the External Moderator will include: Providing subject expertise Ensuring the award of the Diploma is secure Making contact with the provider to plan and undertake: support; supplementary; first visit and final moderation visits Attendance at awards boards Acting as AVA Rep, when required, to represent Certa at awards boards Ensuring the evidence of achievement is appropriate and sufficient Verifying the standards of student achievement leading to the award of credit for specific Access to HE Diplomas, paying due regard to fairness and equity. Verifying the standards of student achievement leading to the award of the Access to HE Diploma, paying due regard to fairness and equity Ensuring that assessment judgements are valid and consistent Ensuring grading implementation has been applied correctly across the curriculum and that grade decisions are consistent Ensuring that the delivery of the course is in accordance with the AVA’s Learning Programme Plan and the Provider’s Approval to Run documentation Monitoring the student experience and ensuring that all students are receiving fair and equitable treatment Monitoring that internal moderation is taking place in a planned and structured way Providing support to the course team and assisting them in continuous improvement of the course, identifying areas of good practice and areas of improvement or development Certa Access Diploma External Moderators Handbook - 2015/16 5 Overview of External Moderation Model An External Moderator is recruited from September in each academic year and is appointed for one year to a named Access to HE Diploma for a specific provider/s. An External Moderator can be appointed to a specific Access to HE Diploma for a specific provider for normally no longer than three years. This can be increased to four years in exceptional circumstances where consistency of the External Moderator would benefit the provider (for example: a new tutor team). The External Moderator will usually make two visits to the provider: January/early February following commencement of the Access to HE Diploma in September and a final moderation visit (usually July). The final moderation visit usually coincides with the awards board at which the External Moderator is normally required to attend. Where a named Access to HE Diploma is being run for the first time with a provider, the External Moderator will usually accompany the Access Development Officer to undertake an additional support visit during the first few weeks in order to offer additional support to the tutor team. Following each visit, External Moderators are required to complete a detailed report on the quality of a particular Access to HE course. External Moderators are also expected to attend up to two standardisation events each year. The Lead Moderator is responsible for writing an Annual Report which is informed by the individual External Moderators’ reports. This report may indicate actions for both External Moderators and the AVA. The report is submitted to the Access to HE Committee for approval as part of the AVA Self-Evaluation Report (ASER). To summarise, the main focus for the External Moderator must be the delivery and assessment of the Access to HE Diploma. External Moderators must check that the following are in place: Robust quality systems and processes Consistent and appropriate documentation and record keeping Consistent and high quality assessment practice Student satisfaction with the Access to HE course experience In addition, the External Moderator is responsible for the final sign-off of the Recommendation for the Award of Credit (RAC) at the awards board. Certa Access Diploma External Moderators Handbook - 2015/16 6 Internal Moderation The AVA requires that providers undertake a transparent and effective system of internal moderation. Providers will be required to have in place systems for pre-delivery internal moderation and on-going internal moderation throughout the Diploma. External Moderators are key to ensuring that this process is in place and is sufficiently robust to ensure internal quality. Purpose and Focus of the External Moderation Visits Whilst there may be some slight differences according to the needs of the provider (experienced providers may require a lighter touch than other providers), the purpose of a moderation visit is: To meet with the tutor team and review previous action plans To agree sampling methods To discuss assessment strategies, including grading To discuss assessment briefs and other assessment document To discuss issues raised from previous reports To discuss internal moderation policy and review internal moderation documents To sample student portfolios to verify assessment of achievement To review any requests for extensions To review any evidence presented for APL (in accordance with the AVA’s policy) To obtain information to enable a judgement to be made on whether the Access to HE Diploma course accurately represents the content of the documents previously supplied to the AVA as part of the validation process To safeguard the security of the Access to HE Diploma and to ensure it remains fit for purpose To provide a risk rating to enable the AVA to take appropriate action and to put in place appropriate safeguarding measures To meet with students at the first visit, listen to any concerns but also identify areas of good practice. External Moderators are not able to meet with individual students under the age of 18 or with vulnerable students without a parent or tutor being present The External Moderator is not required to observe teaching. External Moderators must always bear in mind the Lone Working Conditions as prepared by the AVA and to conform to health and safety requirements set by the provider. Certa Access Diploma External Moderators Handbook - 2015/16 7 Recognising Good Practice The AVA requires External Moderators to make every effort to identify and record good practice in the operation and delivery of Access to HE Diplomas. Good practice will differ across tutor teams, Diplomas and providers. It is for the External Moderator to seek out good practice and report accordingly. Good practice can be identified where policies, systems or other methods of operation, management or organisation of the Access to HE Diploma courses are seen to be exemplary. Examples of good practice could include, but are not limited to: Timely and thorough internal moderation both pre-delivery and throughout the academic year Innovation in teaching, learning and assessment practices Accurate monitoring of end of year progression routes Excellent assessor feedback using focused and developmental language to provide future skills development, particularly at distinction and beyond Effective and developing use of virtual learning environments (VLEs) to enhance teaching and learning Effective student induction including the early issue of assignment schedules Effective use of specialist software programmes to detect plagiarism eg Turnitin Tutor teams who possess a strong sense of ownership of the Access to HE Diploma course Regular and minuted team minutes with action points Identifying Poor Practice External Moderator comments should be detailed, clear and constructive and report on areas for improvement referring to appropriate evidence and clear timescales for action required. Certa Access Diploma External Moderators Handbook - 2015/16 8 Responsibilities of the AVA to Access to HE External Moderators The AVA is fully committed to supporting External Moderators. They are ambassadors for Certa. Our commitment to External Moderators includes: Offering at least two full training days each academic year when External Moderators are required to attend. They receive payment and travel expenses Ensuring that External Moderators receive an early indication of their allocated provision, usually September/early October Payment and re-imbursement of travel expenses for attendance at up to two standardisation events each academic year Additional support and advice provided by the Lead Moderator who can be contacted, via Certa. The aim is to respond to enquiries and queries within two working days Ensuring that the latest documentation, policies and procedures are always available on the Certa website Provision of a Certa email address. Full details will be provided on appointment. It will be of the format: e.moderator@certa.org.uk Prompt and efficient response time from the Access to HE Team to queries and enquiries Acknowledging receipt of all reports promptly and forwarding these to the providers once any inconsistencies identified by our quality checking have been amended Listening to any recommendations for improvement and considering such recommendations as part of the AVA’s on-going development and improvement Certa Access Diploma External Moderators Handbook - 2015/16 9 Section 2: Timetable of Events Date Activity July/August (prior to the External Moderators will be required to complete a CPD start of the academic Record and submit a self-assessment report on their previous year) year’s activity. September Certa compiles a provisional EM allocation which is formally approved by the Access to HE Committee. Once student enrolment numbers are confirmed by providers, the AVA will appoint External Moderators to specific named Diplomas. The previous year’s External Moderator reports will be supplied to the appointed EM by the AVA. October/early November External Moderators may be required to undertake a support visit with the AVA’s Access Development Officer when a provider is offering a named Diploma for the first time or where there is a new tutor team. October External Moderators are required to attend the first of two mandatory training events. These CPD events will up-date External Moderators on any new processes/procedures, consider issues and trends that have arisen from the previous year and prepare them for their first visit to the provider. November The External Moderator will make contact with the provider’s quality contact and suggest dates for the first visit. The quality contact is responsible for ensuring that details are passed to the appropriate curriculum contact. Once the date is confirmed, this should be advised to the Access to HE Senior Administrator. A visit planner is then prepared by the External Moderator and submitted to the provider. January/early February The first visit is undertaken (and the External Moderator meets with the students). A report is written and submitted to the AVA within ten working days. The report is reviewed before being forwarded to the provider. March External Moderators are expected to attend up to two standardisation events within their areas of specialism, to assist the Lead Moderator. April External Moderators are required to attend the second mandatory training event. This will be an opportunity for External Moderators to up-date their knowledge and prepare for the final moderation visit and awards board. Certa Access Diploma External Moderators Handbook - 2015/16 10 May/June The External Moderator will make contact with the provider’s quality contact and suggest dates for the final visit. A visit planner is then prepared by the External Moderator and submitted to the provider. June/July The awards board date is usually planned at the previous year’s award board, although this will be confirmed by the Access to HE Senior Administrator nearer the date in discussion with the provider’s quality contact. External Moderators attend for the final moderation visit and awards board. Where more than one awards board is being held, the External Moderator may be asked to act as AVA representative for a different diploma. The final report is submitted to the AVA within ten working days to be reviewed before being forwarded to the provider. The awards board minutes are completed at the time of the board, signed immediately and subsequently submitted to the AVA together with the signed Record of Achievement (RAC). After the awards board, the EM (or AVA Representative) will post hard copies of both the RAC and awards board minutes to Certa. July/early August External Moderators will receive any queries from the AVA and the provider relating to extensions agreed at the awards board for students to complete work. July Submit final claim for fees and travel expenses for payment in August. There may be circumstances where an additional supplementary visit is thought to be required by the External Moderator. In these circumstances, the External Moderator must discuss this requirement with the Access to HE Manager before finalising arrangements. Certa Access Diploma External Moderators Handbook - 2015/16 11 Section 3: The First Visit The first visit should take place in January/early February. It is expected that the visit will not last longer than three hours. The purpose of the visit is to: Meet the quality contact and course co-ordinator Meet as many of the tutors and internal moderators as is convenient, given teaching commitments Check that the course co-ordinator has all the necessary documentation Ensure that problems that may prevent the confirmation of award of credit are resolved early in the delivery of the course Check that recommendations made from the previous external moderation report have been addressed To sample evidence of achievement to date To meet with students to monitor and discuss: o o o o o how they settled in their understanding of the assessment process and grading requirements the support available to them by the provider the requirement to appoint a student representative who will be invited to Certa to take part in the student representative meeting students’ intended progression routes To check that the Record of Achievement (RAC) is being completed as assessment decisions are made Use the RAC to choose a sample of assessed student work and if possible externally moderate completed units by signing off those units, ensuring that a copy of the RAC is retained by the External Moderator To check that all quality assurance processes are progressing satisfactorily. The course co-ordinator should maintain a quality file which contains the following documents: o o o o o o o o o Course Handbook calendar of quality assurance activities set at the start of the course, including all quality assurance activities required by the quality manager of the provider list of assessors and tutors table of internal moderators and curriculum responsibility for the process pre-issue verification tasks, reports and resultant incorporation into a rolling action plan copy of Course and Learning Registration Forms (CLRF) sampling frame or other evidence for internal moderation samples completed quality assurance reports of visits to students and tutors completed internal moderation reports for all assessed units and resultant incorporation into a rolling action plan Certa Access Diploma External Moderators Handbook - 2015/16 12 o o o o minutes of meetings which clearly identify internal moderation and quality assurance items with the current rolling action plan with responses reports from internal standardisation activities and resultant incorporation into a rolling action plan reports from external standardisation activity and resultant incorporation into a rolling action plan a rolling action plan indicating responses to External Moderator’s recommendations or conditions, internal moderation reports, and any internal quality assurance activity Certa will forward to the External Moderator: The previous year’s moderation reports relating to each of the Diplomas allocated to the External Moderator The AVA’s Learning Programme Plan for each Access to HE Diploma A list of all other External Moderators appointed to the provider The most recent Access to HE Handbook (the latest version will always be available for reference on the Certa website) At least ten working days before the visit, External Moderators will send a visit plan to the quality contact or nominated curriculum contact as appropriate. The template for the visit plan is available to download from the Certa website. External Moderators provide oral feedback to the quality contact and the course team. It is expected that at the end of the visit, the External Moderator will discuss the findings from the visit with provider staff and clearly indicate areas of good practice and those for development. Such discussions will ensure that any recommendations are written in such a way as to be effected successfully by the provider. External Moderators write a report after the visit, which will be sent directly to the Access to HE team at Certa. This will then be reviewed and may be referred back to the External Moderator for amendment or clarification. Once the final version is agreed, the report will be forwarded to the provider’s quality contact by the Access to HE team. It is the responsibility of the quality contact to cascade the report internally. The report proforma is available to download from the Certa website. These reports are considered by all parties concerned with the quality of the Diploma provision: the Lead Access to HE Moderator, the AVA’s Access to HE Committee, the Principal or Head and the Quality Manager of the provider, as well as the course team. If there is a serious problem, the External Moderator should immediately contact the Access to HE Manager to enable the AVA to initiate prompt action. Certa Access Diploma External Moderators Handbook - 2015/16 13 Section 4: The Final (2nd) Moderation Visit The final moderation visit should take place between mid-June and the middle of July to complete the sampling and external moderation of all units, and to confirm the achievement of the Diploma and/or credits. The second visit will be that of final moderation when students’ work is sampled, and the quality assurance documentation is closely monitored. The purpose of the visit is to: Sample from formally assessed work to confirm that the student has achieved against specific learning outcomes and assessment criteria and to assist the AVA to mitigate against malpractice To assure that assessment decisions are appropriate, consistent, fair and transparent across the sample To assure the correct implementation of grading requirements To assure that grade indicator and grade decisions are appropriate, consistent and fair across the sample To assure that grading processes are correctly applied to result in a grade for the unit, and that any grade information is clearly shown at all stages of the process To check the quality file to ensure that all activities planned in the calendar for quality assurance have been completed To ensure that the internal quality assurance system has been effectively applied to the provision throughout the year The External Moderator is responsible for confirming the achievement of the Access to HE Diploma at this visit. The External Moderator is charged with sampling from formally assessed work to confirm that the student has achieved against specific learning outcomes and assessment criteria. The External Moderators do not and should not need to see the trail of learning which has led the student to the point of being able to complete the assignment. All internal moderation and standardisation must be completed at least one week before the final visit so that the External Moderator can make a decision about the sample they wish to see. It is considered to be good practice for providers to hold a pre-awards board meeting to confirm readiness for final moderation. Any student portfolio that requires a tutor to walk the External Moderator through it is not ready for external moderation. Such portfolios should be referred back to the student for additional work. At least ten working days before the visit, External Moderators will send a visit plan to the quality contact or nominated curriculum contact as appropriate. The template for the visit plan is available to download from the Certa website. External moderators provide oral feedback to the quality contact and the course team. It is expected that at the end of the visit, the External Moderator will discuss the findings from the visit with provider staff and clearly indicate areas of good practice and those for development. Such discussions will ensure that any recommendations are written in such a way as to be effected successfully by the provider. Certa Access Diploma External Moderators Handbook - 2015/16 14 External Moderators write a report after the visit, which will be sent to the Access to HE team at Certa within ten working days. This will then be reviewed and may be referred back to the External Moderator for amendment or clarification. Once the final version is agreed, the report will be forwarded to the provider’s quality contact by the Access to HE team. It is the responsibility of the quality contact to cascade the report internally. The report proforma is available to download from the Certa website. These reports are considered by all parties concerned with the quality of the Diploma provision: the Lead Access to HE Moderator, the AVA’s Access to HE Committee, the Principal or Head and the Quality Manager of the provider, as well as the course team. The final moderation visit is usually followed by the awards board. However, this may have been arranged on a different day where there is large provision or where specialist subjects such as Art and Design are concerned. Certa Access Diploma External Moderators Handbook - 2015/16 15 Section 5: Sampling Guidance for External Moderators Providers will often make the whole body of student work available to the External Moderator at the final moderation event. However, an External Moderator may decide for a variety of reasons to choose a sample before the event (for example: a number of sites are involved and there is a logistical problem, there is a large number of students, level of satisfaction of quality processes in assessment at previous visits). It is essential that for new and high risk providers, the whole body of evidence is available. The completed electronic form of the RAC must be available to the External Moderator at least one week before the event so that a satisfactory sample can be selected. The sample chosen by the External Moderator is then identified on a hard copy of the RAC. Hence, the AVA has a record of the sample taken. In choosing a sample there will be a strategy that takes a number of key factors into account: Delivery sites Tutors/assessors Curriculum strands Unit level and credit value Rules of combination – mandatory and optional units Delivery format: one/two year, evening provision etc Assessment methods Work that has been internally moderated Grade range Unusual or unexpected patterns in the award of grades Reasonable adjustments to meet special requirements Resubmission, appeals, representation and referral cases. Issues from previous internal and external moderation External Moderators will sample in two ways: by looking in detail at the work of individual students throughout the Diploma and by sampling particular tasks or units across a range of students to evaluate grading decisions. If there are less than 10 students it is advisable to sample all portfolios. If there are 20 students, a 50 % sample should be taken. No minimum percentage is stipulated. External Moderators should consider the profile of grades awarded to students and establish the reasons for the profile. For example, if there are a larger than expected number of distinctions awarded for a particular unit or by an individual tutor, the reasons should be explored. A RAC not completed on the arrival of the External Moderator will result in the cancellation of the event. The provider will be charged to meet the cost of a reconvened event in accordance with Certa’s charging policy. If in doubt, the External Moderator should contact the Access to HE team for advice during the early stages of a visit. Certa Access Diploma External Moderators Handbook - 2015/16 16 Section 6: The Awards Board All Access to HE Diploma courses are required to hold a formal awards board. It is the responsibility of the provider’s quality contact to arrange the awards board. It is chaired by a member of senior management from the provider. The final awards board is normally held in late June or early July and will usually take place immediately following the final moderation visit. This is to ensure that: (i) (ii) results can be processed and uploaded to UCAS well before A-level results day in mid-August, and certificates can be printed and forwarded to providers for onward transmission to students There are some exceptions to these dates (for example: courses which commence in January). If more than one awards board is taking place on the same day, External Moderators are required to act as the AVA Representative. Certification for students can only be progressed when all documentation is accurately completed and forwarded to Certa. This includes: Accurate minutes of the awards board which are fully completed and signed by the Chair and External Moderator Fully completed RAC which is submitted both electronically and in hard copy External Moderators should make themselves fully aware of the contents of the awards board Policy and Specification. Whilst the Access to HE team will have contacted the quality contact of the provider to finalise the date and advise who should be in attendance at the awards board, it is the provider’s responsibility to ensure that the awards board business is completed on the day. An experienced External Moderator (the ‘Moderation Co-ordinator’) is allocated to each provider, from the pool of External Moderators who have been appointed to that provider, to assist with the co-ordination of the final visits and awards board. Certa Access Diploma External Moderators Handbook - 2015/16 17 Section 7: Writing the External Moderation Report In order to address all the relevant points, every section in the current report proformas must be completed. If there is nothing to report under a heading then there should be some explanation. The external moderation report should: Clearly express issues concisely in an objective way so that they are unambiguous and easily understood by the potential audience Be written about the particular Access to HE Diploma. Certa will not accept reports that are not clearly about the individual Diploma. Where there are reasonable comments that can apply to all diplomas at a provider (for example: resources, guidance to students), this must be clearly identified Avoid abbreviations and provide complete sentences, not notes. Be presented to a high standard using short sentences – complex ones can be misunderstood Omit names, use designations only Use terms used in QAA documentation consistently and correctly (for example: grade descriptor components) Use terms relating to the principles of assessment accurately i.e. validity, reliability etc. Be word-processed and on the proforma available on the AVA website Provide balanced judgements citing evidence. Clearly identify recommendations/conditions, following logically from points made in the body of the report Always provide evidence of good practice where possible and developmental action where appropriate Be sent directly to Certa, copies are not to be given to the provider directly. The reports will then be reviewed and may be referred back to the External Moderator for amendment or clarification. Once the final version is agreed, the report will be forwarded to the provider’s quality contact by the Access to HE team. It is the responsibility of the quality contact to cascade the report internally Certa will send the final copy of the report to the provider’s quality contact and to the External Moderator The external moderation reports should not be discussed with, or made available to anyone outside of Certa or the provider. In addition to the formal report form, the AVA has available an Information and Issues Form (available to download from the Certa website) that can be used to identify any issues that are not appropriate for the visit report. This form alerts the AVA to any potential sensitive issue, potential change at the provider, or confidential issue that the AVA should be aware of. Certa Access Diploma External Moderators Handbook - 2015/16 18 Section 8: The Role of the Lead Moderator The AVA has a Lead Moderator for Access to HE whose role includes: Acting in a supporting and advisory role to the Access to HE Manager and AVA teams Attending meetings with QAA, and others, where appropriate Supporting and/or chairing validation panels Planning and facilitating the annual standardisation events which take place in March/April each year Accompanying new External Moderators as part of their induction process (other experienced External Moderators with an appropriate curriculum expertise may also cover this role) Contributing to Access to HE CPD events Supporting External Moderators to carry out their roles and responsibilities in a professional way and to the highest possible standard Writing a report following the annual AVA standardisation events Writing an annual report based on the External Moderator reports. This report includes recommendations for the AVA’s Access to HE Committee The Lead Moderator can be contacted via Certa. The aim is to respond to enquiries and queries within two working days. Certa Access Diploma External Moderators Handbook - 2015/16 19 Section 9: Supplementary and Support Visits Certa may arrange for support or supplementary visits in the following situations: Supplementary Visits When a provider offers a variation on the normal once-a-year intake of students When a provider has transferred from another AVA Support Visits The first run of a new Diploma as a support to the provider. Certa’s Access Development Officer may also accompany the External Moderator when a Diploma has undergone substantial quality assurance or curriculum development or is in fragile circumstance Certa Access Diploma External Moderators Handbook - 2015/16 20 Section 10: Practice) QAA’s Licensing Criteria (relating to Moderation Criteria for the licensing and relicensing of Access Validating Agencies May 2011 (updated July 2014) Precept 5: Moderation, monitoring and certification The organisation (Certa) maintains the quality and standards of recognised Access to HE courses, and it makes Access to HE Awards to students who have the met the specified requirements for achievement and performance. These criteria relate to the ways in which AVAs maintain consistency and sufficiency of demand of recognised Access to HE courses, the quality of the provision and the academic standards that are required for the award of Access to HE Diplomas. The criteria also require AVAs to operate a regulated and secure process for making awards and issuing Access to HE Diplomas, credits and grades. Managing moderation and course monitoring Systems of moderation and course monitoring, and the divisions between them, vary according to AVAs' different quality assurance systems and different models of organisation of Access to HE provision. Moderation is principally concerned with ensuring that academic standards on recognised Access to HE courses are maintained, in terms of a) the specific requirements of approved Diplomas, b) the academic challenge and demand made by the curriculum, delivery and assessment of courses, and c) the achievement and performance demonstrated by the students who are recommended for the award of specific Diplomas, credits and grades. Course monitoring is principally concerned with confirming that the delivery of Access to HE Diplomas a) provides a positive and appropriate student experience and outcomes, and b) is consistent with the criteria for provider/centre approval, including requirements relating to internal course self-evaluation and review. Alternative routes to moderation and monitoring Moderation systems are often used to support the AVA's systems for course monitoring, and moderators may be asked to provide information about the quality of provision and the student experience. As a result, some of the requirements of course monitoring are sometimes met through the moderation process. Criteria The organisation (Certa) can demonstrate that it meets the following criteria. (All parts and sub-parts of the criteria apply, unless otherwise indicated.) Certa Access Diploma External Moderators Handbook - 2015/16 21 Moderation processes Moderation involves the appointment of moderators by the AVA. Systems and organisation of moderation differ, however, and different models may involve, for example: i teams of moderators considering assessment and achievement on individual Access to HE courses, each moderator looking at one or more particular area(s) of the course ii teams of moderators considering assessment and achievement on all of a particular provider's/centre's Access to HE courses, individual moderators looking at related areas on several courses, with a single lead moderator coordinating the moderation iii teams of moderators considering assessment and achievement on particular Diplomas delivered by more than one provider/centre iv teams of moderators who consider assessment and/or achievement in relation to certain subject areas (or sets of units) across a number of courses or Diplomas v teams of moderators who determine all aspects of assessment on a set of courses or Diplomas (and who may have no direct relationship with any particular provider/centre), and separate (or the same) teams who consider achievement on those courses or Diplomas, with visits to provider/centres being conducted as a separate process. 5.1 It maintains a system of regular external moderation, conducted by moderators appointed by the AVA, which applies to all recognised Access to HE courses. 5.2 It provides information about the operation of its moderation processes (including details about alternative processes that may be operated by the AVA), in guidance to moderators and to providers/centres. 5.3 Its guidance includes clear information about: a the structural organisation of the AVA's moderation processes, including the different stages and parts of its processes and how they are related b moderators' roles and responsibilities (including information about the roles of different types of moderator) c the basis, extent and limits of moderators' authority d the purpose of visits and/or meetings and activities undertaken by moderators at visits and/or meetings (including information about different purposes/activities for visits/meetings which take place at different stages) e the frequency of moderator visits and/or meetings f requirements for monitoring and verifying student achievement, including criteria for the sampling of evidence of student achievement g moderation outcomes relating to student achievement, and how these are taken forward to final awards boards Certa Access Diploma External Moderators Handbook - 2015/16 22 h requirements for moderator reports, including information about the nature, structure, content and frequency of reporting i how, and to whom, any concerns or problems identified should be referred j the submission of reports to the AVA and associated follow-up activity. 5.4 It routinely reviews and updates its guidance to ensure its continuing currency. Moderation responsibilities 5.5 Its moderation (and/or course monitoring) system(s) and processes ensure that: a the curriculum delivered on a recognised Access to HE course is consistent with the requirements of its definitive Diploma documentation and is appropriate for its purpose in providing a preparation for study in HE for its target student groups b course organisation allows all students to meet the rules of combination and any other completion requirements for the Diploma on which they are registered c approved providers/centres continue to meet the terms of provider/ centre approval d providers/centres monitor and enhance the quality of the student experience, including taking action to elicit and respond to student evaluation and feedback e assessment practices on Access to HE courses are consistent with The Access to Higher Education Diploma and credit specifications and grading requirements f the conduct of assessment and application providers/centres is thorough, rigorous and fair of assessment regulations by g details of assignment requirements and means by which they are assessed: support the needs of target groups provide an appropriate level of academic challenge and demand provide opportunities for differentiation of performance support the development of academic skills and the promotion of learning allow students to demonstrate their learning allow the assessor to identify whether the specified learning outcomes have been achieved ensure rigorous assessment of student achievement h minimum standards required for threshold achievement and performance are consistent with requirements for achievement of the relevant learning outcomes and grade descriptors i consistent standards are applied in assessment, so that equivalent student achievement and performance is reflected in consistent recommendations for credits and grades Certa Access Diploma External Moderators Handbook - 2015/16 23 j students awarded the Access to HE Diploma have successfully completed the required learning at the required standard, and their performance is reflected in grades awarded k providers/centres operate rigorous internal moderation/verification procedures which meet the AVA's requirements, and which ensure that consistent standards are applied to assessment and achievement on recognised Access to HE courses l public information about Access to HE courses is current and accurate. Moderator recruitment, selection and appointments 5.6 It operates standard procedures, using clear and transparent criteria, for the selection and appointment of moderators, which ensure that moderators: a have relevant experience in adult, further or higher education b have relevant and current subject knowledge for their area(s) of moderating responsibility. 5.7 Where moderators are involved in moderation at subject level, they are qualified and/or experienced in teaching in the area(s) that they are responsible for moderating, at a level at or beyond that at which the subject has been taught and assessed (that is, moderation of level 3 units is undertaken by those with a qualification and/or teaching experience in a related discipline at level 3 or above). 5.8 Moderators are appointed by, and are directly responsible to, the AVA and not a providing or receiving institution or any other body. 5.9 Contractual agreements between the moderator and the AVA set out the moderator's specific duties and term of service (see paragraph 5.10), and the criteria and process for termination of a moderator's employment. 5.10 Moderators are external to the provider/centre and do not hold a position in any receiving institution, the nature of which could create a conflict of interest or limit students' progression opportunities (for example, through direct involvement in admissions decisions for students progressing from the Access to HE course). 5.11 Where the moderator's role involves a direct relationship with individual Access to HE courses, the moderator's period of office for work with the individual course(s) is, in normal circumstances, no longer than four years. 5.12 The moderators involved in the moderation of any individual Access to HE course jointly provide sufficient expert moderation at the subject level for the number and range of subject areas included in the course. Certa Access Diploma External Moderators Handbook - 2015/16 24 Moderator induction and training 5.13 Moderators receive a planned programme of mandatory induction and training, supplemented by written guidance (see paragraph 5.4), that enables them to carry out their role effectively. 5.14 The AVA routinely updates moderators on any changes in the AVA's requirements, or changes which relate to the Access to HE Diploma. 5.15 There are mechanisms to enable moderators to compare standards and judgements across the AVA's Access to HE courses. 5.16 It monitors moderators' performance, and takes appropriate action if performance is deemed to be unsatisfactory. Moderation reports 5.17 It operates standard procedures to ensure that action is taken in response to matters requiring action raised in moderation reports, including procedures for: a receiving and responding promptly to individual moderation reports b providing copies of moderators' reports to appropriate provider/centre staff c ensuring that matters raised in reports which require specific action by providers/centres are brought to providers'/centres' attention, a clear timescale for action is given, and implementation of actions is confirmed d taking action if providers/centres fail to address issues raised in reports, including procedures for the ultimate sanction of withdrawal of course recognition e ensuring that matters raised in reports proposing modifications to Diplomas are formally considered, and any necessary modifications are made (in accordance with the AVA's Diploma modification procedures, see paragraphs 4.35 - 4.39) before the next run of the Diploma f taking action in the event that moderators do not meet the AVA's requirements in the execution of their duties g analysing the overall outcomes of the moderation (and course monitoring) system(s) across all of the AVA's recognised Access to HE courses on a regular (normally annual) basis (see paragraph 5.24). 5.18 It provides opportunities for providers/centres to provide feedback to the AVA on the moderation process. Certa Access Diploma External Moderators Handbook - 2015/16 25 Course monitoring and review 5.19 It monitors and reviews Access to HE courses on a regular (normally annual) basis, collecting information which will allow it to confirm the impact, quality and standards of its recognised Access to HE courses and to assess the effectiveness of its own quality assurance procedures. 5.20 It collects statistical data about Access to HE courses and students according to a standard set of categories, including those required for reports to QAA. 5.21 The process for course reviews, and the format for reports required by the AVA, takes account of providers'/centres' own self-assessment processes and those that they undertake to report to other bodies and draws on relevant material from the outcomes of those processes. The information may be provided in a single course report, or through other mechanisms which are appropriate to the AVA's quality assurance systems and the organisation of provision. 5.22 While course reviews may allow providers/centres to report on all Access to HE provision at the same time, they differentiate between outcomes and distinctive features of different Diplomas that they deliver. 5.23 Providers'/centres' course reviews provide information and comment to the AVA about: a student recruitment, retention, achievement and success b student progression and destinations c standards of student performance d student feedback received about the course and the quality of their learning experience, and the provider's/centre's response to issues raised e examples of innovation and good practice. 5.24 It seeks feedback from its partner HE institutions about the performance of Access to HE students and the appropriateness of the preparation provided by its recognised Access to HE courses. 5.25 It analyses and formally considers the information and feedback (including statistical information) that it gathers from providers/centres and HE partners, and, in response: a identifies concerns about the quality of provision or standards of student achievement on individual Access to HE courses or Diplomas which had not been raised by the moderation process b takes effective action to address issues identified, including systematic follow-up when issues have been notified to the provider/centre c disseminates examples of good practice among providers/centres d identifies lessons for the effectiveness of its own quality assurance procedures, and ensures that, where required, appropriate changes are made to procedures. Certa Access Diploma External Moderators Handbook - 2015/16 26 5.26 The designated body within the organisation's governance structure that has responsibility for quality and standards on Access to HE courses (for example, an Access to HE Committee or other body which includes representation from HE) maintains oversight of the moderation and course monitoring processes and it: a ensures the robustness of the processes and consistency of their operation b considers and confirms any decisions that relate to potential withdrawal of course recognition c receives the AVA's analysis of the overall outcomes of the moderation and course monitoring processes d ensures that the outcomes of the moderation and monitoring processes are disseminated and used to improve and enhance recognised Access to HE provision within the AVA. Standardisation 5.27 It takes steps to ensure that equivalent standards and requirements for achievement apply on different Access to HE courses which lead to Diplomas, within the AVA and across different student cohorts, with reference to The Access to Higher Education Diploma qualification and credit specifications and grading requirements, and: a where Access to HE Diplomas or common units are available on more than one course, it operates mechanisms to ensure that consistent standards are applied across providers/centres, in relation to assessment requirements and judgements about achievement (including graded achievement), and its moderation systems take account of the outcomes of this process b where Access to HE Diplomas or units are available in cognate subject areas on different courses, it provides opportunities for those involved in Access to HE assessment and moderation to define and compare specific standards of required achievement for the award of credits and grades, and its moderation systems take account of the outcomes of this process. Award of Access to HE Diplomas 5.28 It operates standard specified procedures for the award of Access to HE Diplomas to students that are consistent with The Access to Higher Education Diploma and credit specifications, grading requirements and assessment regulations and, in particular: a makes clear that decisions about the award of Access to HE Diplomas, credits and grades are made by final awards boards b provides explicit guidance about the function, process, and appropriate membership of final awards boards, consistent with the requirements of QAA's assessment regulations for Access to HE c operates effective procedures to confirm that all students recommended for the award of an Access to HE Diploma have achieved the requirements of its specific rules of combination Certa Access Diploma External Moderators Handbook - 2015/16 27 d ensures that evidence of achievement (including evidence relating to the accreditation of prior learning) has been verified, and the moderation process has been completed, before recommendations for the award of credits, grades or Diplomas are made to the AVA e decisions made by the final awards board about the award of credit, grades and Diplomas are forwarded to the AVA in a standard format and are transmitted securely f ensures that all students on Access to HE courses who complete the required learning achievement are awarded an Access to HE Diploma, including the award of credits and grades for relevant units g ensures that all students who do not complete the learning required for the award of a Diploma are awarded credits for all units successfully completed. 5.29 It operates secure data systems which maintain records of all Access to HE Awards made by the AVA (see paragraph 3.9, f and g). QAA856 - Jul 14 © The Quality Assurance Agency for Higher Education 2014 Southgate House, Southgate Street, Gloucester GL1 1UB Tel: 01452 557 000 Email: enquiries@qaa.ac.uk Website: www.qaa.ac.uk Registered charity Certa Access Diploma External Moderators Handbook - 2015/16 28 Section 11: Summary of External Moderator Roles The AVA needs to ensure the continuity of experienced External Moderators and has introduced a range of roles: Lead Moderator External Moderator Mentor Mentee Handover Moderator Observer AVA Rep Subject Moderator Moderation Co-ordinator These roles and the applicable fees are described in the following table. Certa Access Diploma External Moderators Handbook - 2015/16 29 SUMMARY OF EXTERNAL MODERATOR ROLES Role Experience External Moderator Substantial experience in delivery at Level 3 and above, including assessment and quality assurance in adult education Mentor Worked as an experienced external moderator Subject Moderator Experienced subject specialist Summary of Activity Required Full range of activity in line with issued external moderator contract Invited by Certa to fulfil role and formally accepted external moderator contract Full range of activity in line with issued external moderator contract Liaise with mentee when required for attendance at the first and/or final visit but retain responsibility of the moderation process required at the visit Provide feedback to the Access to HE Manager on mentee(s) assigned during the academic year Encourage and oversee the mentee being involved in moderation of student work Will normally accompany the external moderator at the final visit unless issues have been identified at the first visit (ie no work available for moderation at first visit), when a subject moderator will visit on their own to moderate particular subjects Will provide an oral report to the external moderator based on the sections in the EM report Will provide a summative report at the end of all visits to be sent to Certa Certa Access Diploma External Moderators Handbook Fees/Payment In accordance with the contract plus reasonable travel expenses* Fee payable on completion of feedback report on appointed mentee = £20 Fee = £60 plus reasonable travel expenses* 30 Role Experience AVA Representative for awards board Experienced external moderator with full knowledge of Certa procedures Mentee (there is no guarantee of an EM contract during the current or following year) Summary of Activity Required To attend at the time and date provided by the AVA AVA to provide relevant AVA quality documents To ensure awards board follows the AVA procedures To offer information/guidance on AVA policies as required No report to write To carry out checks on RAC as required during the awards board To carry out checks on both the summary sheet and the results sheet to ensure accuracy Liaise with experienced external moderator Accompany experienced external moderator with Will attend the CPD event for same area of specialisms, for normally one visit external moderators or been through induction with the Lead during the academic year, to decide whether to continue with application to become an external Moderator moderator Invited to attend at a standardisation event within their curriculum specialism Maintain confidentiality at all times May have limited experience of No report to write Access to HE but will have A mentee may be invited to take on full external significant experience within a moderator responsibilities in-year on successful subject specialism provision at feedback from mentoring external moderator Level 3 or above Certa Access Diploma External Moderators Handbook Fees/Payment Fee payable when attending at AVA invitation (ie not already at the provider premises undertaking external moderator duties) = £50 plus reasonable travel expenses* and postage costs for sending hard copies of documentation to Certa No payment Reasonable travel expenses to be paid but advised to Certa in advance* 31 Observer (there is no guarantee of an EM contract during the current or following year) Moderation Co-ordinator Liaise with experienced external moderator May have limited experience of Accompany experienced external moderator with same area of specialism, for one visit during the Access to HE but will have academic year, to decide whether to continue with significant experience within a application to become an external moderator subject specialism provision at Maintain confidentiality at all times Level 3 or above Invited to attend at a standardisation event within their curriculum specialism No report to write Going out for own interest and CPD opportunity with a potential view to possible external moderator work at a later stage Experienced external moderator with full knowledge of Certa procedures To liaise with other external moderators at the same provider to organise the date for the final awards boards No payment No travel expenses No additional payment as included within external moderator fee *Reasonable travel expenses = normally the standard rail fare or 40p per mile whichever is the most convenient. Other potential expenses should be discussed, and agreed, in advance, with the Head of Access. An expenses claim form is available Certa Access Diploma External Moderators Handbook 32 Section 12: Code of Conduct for External Moderators In accepting a Contract from Certa to undertake the role of an External Moderator, an External Moderator must abide by the following Code of Conduct: Act in ways which do not bring the AVA into disrepute Promote the best interests of the AVA at all times Maintain an appropriate level of objectivity and independence, notifying the AVA where there is any possibility of this being compromised Act in a supportive and professional manner at all times Treat students, provider staff, Certa colleagues with courtesy and respect at all times Not discriminate against students or tutors on the grounds of their origin, race, status, sexual orientation, age, disability or beliefs Respect the privacy of students and not divulge any confidential information Respect the confidential nature of the External Moderator’s role and not divulge nor communicate any confidential information to any company, business entity, organisation or the media Respect the laws of copyright Give precedence to the professional responsibility of the role of the External Moderator over personal interests Observe the Health and Safety Policy of both the provider and Certa and comply with accident reporting procedures. Declare any conflict of interests In addition, External Moderators are required to: Supply a telephone number on which Certa can make contact (including a mobile telephone number when the External Moderator is away from home) Access their Certa email on a regular basis and respond to AVA correspondence promptly Read, and respond to, the Monthly Bulletin Certa Access Diploma External Moderators Handbook 33 Section 13: Maintaining Records of Continuous Professional Development Certa will support External Moderators in their work through both formal training (attendance at the two mandatory CPD events each academic year) and through informal guidance. The informal guidance can be provided either by the Lead Moderator or the Access to HE Manager. In August each year, the AVA will contact each External Moderator and request the following information to be supplied. Responses to the request will in part determine whether the AVA feels that a further year’s contract can be offered to the External Moderator: A report on their experiences throughout the year A record of their CPD Expression of interest in future work with Certa It is recommended that a log of CPD undertaken is maintained throughout the year ready for submission to the AVA. An example is given on the following page. Certa Access Diploma External Moderators Handbook 34 Summary of CPD undertaken during 2014/15 Name: NB: CPD may include lectures attended, membership of professional bodies, working for other Awarding Organisations or Access Validating Agencies, papers written, research undertaken, training delivered etc. Date Length of time Summary of CPD undertaken (eg 2 hours) 4 September 2014 4.00 pm to 6.30 pm Certa – Assisting providers with the implementation of the new Access to HE Diploma 14 October 2014 9.30 am to 4.00 pm Preparing for 1st External Moderation Event 15 April 2015 9.30 am to 4.00 pm Preparing for Final Moderation and awards board 2 May 2015 4.00 pm to 6.00 pm Visiting lecture on creative assessment held at the University of Hull Membership of Professional Bodies Name: British Psychology Society Date Joined: 27/2/1999 Certa Access Diploma External Moderators Handbook 35 Section 14: Monitoring of External Moderator’s Performance Certa is required to monitor the performance of External Moderator’s and take appropriate action as required. Certa will support External Moderators in their work through training and informal support by the Access to HE team, the Lead Moderator and the Quality Team. To assist Certa, the following are in place: Mandatory attendance and participation at External Moderator CPD events Evaluations completed by providers following External Moderator visits (using the SurveyMonkey on-line tool) An observation cycle of External Moderators in practice Reviewing of Visit One and Final Visit external moderation reports to establish a standardised approach to External Moderator practices Mentees are assigned a mentor and are supported and observed in practice. The mentee may not be subsequently appointed if the AVA considers that the mentee does not meet the expected criteria A letter of resignation is required from moderators wishing to withdraw from any duties. Notice of one calendar month is requested in order to ensure that the post can be filled with minimum disruption to the programme. Where an External Moderator fails to comply with the terms of the contract, the Chief Executive Officer (CEO) will review each case individually and will normally informally discuss any concerns with the External Moderator. If it is not possible to resolve an issue informally: the CEO will collate evidence and will invite the External Moderator to a meeting to be held at Certa House within 14 days of the informal discussion. The meeting will be minuted actions required by the External Moderator will be forwarded to the External Moderator within 10 working days of the meeting the External Moderator will be required to respond in writing to the actions within a further ten days of the date of this communication If any issue cannot be resolved as above, the Chair of the Access to HE Committee will consider the evidence and responses and will convene a hearing within 40 working days of the initial meeting with the CEO. The hearing will include: the Chair of the Access to HE Committee, the CEO, the Access to HE Manager, the External Moderator and the External Moderator’s representative if required A letter will be sent to the External Moderator within 10 working days of the hearing The hearing may decide to: Take no further action To suspend the External Moderator until further training and support has taken place To inform the External Moderator that no further contracts will be issued and that any existing contracts will be terminated with immediate effect Certa Access Diploma External Moderators Handbook 36 Certa, Certa House, Lower Warrengate, Wakefield. WF1 1SA Phone: 01924 434600 Fax: 01924 364213 accesst2he@certa.org.uk http://www.certa.org.uk/ Certa Access Diploma External Moderators Handbook 37