External Moderator Handbook All UAL Awarding Body Qualifications – Levels 1, 2, 3 & 4 This handbook for External Moderators is designed to provide guidance in the moderation of University of the Arts London Awarding Body qualifications. UAL Awarding Body is delighted to be working with its partners in providing high quality, robust, quality assured qualfications, qualfications. and We hope this handbook supports you in your commitment to upholding our standards. Introduction Background and Context 1.1 Introduction from the Head of Academic Standards At the University of the Arts London Awarding Body our policy for the appointment of External Moderators has always been that, as one of our representatives, you should be an expert in your field. We believe that the moderation process should be a positive and worthwhile experience for both you as a moderator, and the centre you visit. Your dialogue around creative practice and assessment will maintain and improve standards and in turn, enhance the student experience. We expect you to carry out this work with integrity and honesty, so we will do our best to support you by offering regular training and standardisation sessions. We will also strive to answer any queries you have within the shortest possible time frame. We hope that you will find this handbook a useful resource. If you would like to see additional content in future versions please do contact us to let us know. Sarah Atkinson Head of Academic Standards UAL Awarding Body 2 1.2 The purpose of moderation This handbook is designed to provide guidance on the systems and procedures of external moderation for qualifications awarded by the University of the Arts London Awarding Body (hereafter referred to as UAL Awarding Body). External moderation by representatives of UAL Awarding Body is designed to ensure that: Centres maintain and provide accurate and consistent standards of assessment and grading Centres maintain and provide accurate and consistent standards when awarding credit and qualifications The qualification has been delivered in accordance with the specification and any other documentation provided by UAL Awarding Body Certification is agreed in line with published procedures Support and guidance is given to centres to encourage continuous improvement A record of centre performance is maintained. UAL Awarding Body requires External Moderators to be fair and objective and the external moderation visit should be supportive and developmental. Feedback for centres should identify strengths and highlight good practice as well as any areas for improvement. It is expected that External Moderators will explain and discuss any issues that require formal actions with the Course Leader. 1.3 Qualifications overview The qualifications UAL Awarding Body currently offers are: L3 and L4 Diploma in Art and Design – Foundation Studies L1 Award and Diploma in Art, Design & Media L2 Award and Diploma in Art & Design L3 Diploma and Extended Diploma in Art & Design L2 Award and Diploma in Creative Media Production & Technology L3 Diploma and Extended Diploma in Creative Media Production & Technology L1 Award and Diploma in Music, Performing & Production Arts L2 Award and Diploma in Music Performance & Production L3 Diploma and Extended Diploma in Music Performance & Production L2 Award and Diploma in Performing & Production Arts L3 Diploma and Extended Diploma in Performing & Production Arts L2 Award and Diploma in Fashion Retail & Business L3 Diploma and Extended Diploma in Fashion Retail & Business L4 Diploma in Fashion Retail L4 Diploma in Buying and Range Planning for Fashion Retail L4 Diploma in Visual Display and Branding for Fashion Retail L4 Diploma in Garment Technology for Fashion Retail L4 Diploma in Merchandising for Fashion Retail L2 and L3 Certificates and Awards in Drawing L2 and L3 Certificates and Awards in Photography L2 and L3 Certificates and Awards in Printmaking L2 and L3 Certificates and Awards in Animation The role of the External Moderator 2.1 Job description and role External Moderators, acting on behalf of UAL Awarding Body, use the external moderation process to verify the achievement of students, leading to the award of credit for UAL Awarding Body accredited qualifications. External Moderators are responsible to the Head of Academic Standards at UAL Awarding Body. UAL Awarding Body has produced a full job description outlining the main duties and responsibilities of External Moderators (appendix II). For centres with large cohorts, UAL Awarding Body has a policy of sending in a team of External Moderators to view the sample of student work and confirm that appropriate quality assurance mechanisms are being applied. To ensure the effectiveness of this process, UAL Awarding Body will appoint a Lead Moderator to coordinate and oversee the external moderation visit (appendix iii). 2.2 External moderation procedures UAL Awarding Body will publish a job specification for the post of External Moderator Applications for the post of External Moderator will only be considered on completion of the relevant application form and submission of a CV UAL Awarding Body will select suitably qualified and experienced applicants for training Initial training will provide an introduction to UAL Awarding Body’s systems and procedures External Moderators will be appointed on an annual basis to specific centres External Moderators will be issued with personlised itineraries detailing the centres to which they have been appointed Contact procedures and details will be circulated to External Moderators and centres New External Moderators will initially be allocated to an external moderation team overseen by a Senior External Moderator Within the first year of their appointment, External Moderators will be required to attend a qualification standardisation event Within the first year of their appointment, External Moderators will be observed and evaluated to provide the UAL Awarding Body with a record of performance (appendix IV). Further observations on external moderators will be carried out on a 3–5 year cycle Travel and accommodation for External Moderators will be arranged by UAL Awarding Body within agreed financial limits The Lead External Moderator within a team of External Moderators will be responsible for overseeing the external moderation process including: liaising with External Moderators and the centre and producing the final report. 2.3 Support for External Moderators UAL Awarding Body will provide all new and existing moderators with access to the relevant policy documents. On allocation of a centre or centres, External Moderators will be provided with: Contact details for centres and co-moderators if applicable Previous moderator report for the allocated qualification Report form for completion. We advise all External Moderators to familiarise themselves with the current process for external moderation. This and other materials to support external moderation can be downloaded from our website: www.arts.ac.uk/awarding 2.4 Standardisation events Within the first year of their appointment, External Moderators will be required to attend a qualification standardisation event. Only in exceptional circumstances, and only if allocated to an external moderation team, will new External Moderators be allowed to ‘miss’ a first standardisation event. External Moderators will be expected to attend a standardisation event once every two years. training during a two-year period. They are encouraged to attend on an annual basis. Standardisation events are primarily for the assurance of External Moderator standards. However, where possible UAL Awarding Body will also open up the event to centre delivery staff as a means of extending good practice and supporting centre development. 2.5 Conferences and regional meetings External Moderators are encouraged to attend annual qualification specific training, conferences and celebratory events. These provide an opportunity to share information on good practice, new approaches, updates to education policy and funding arrangements. In addition, UAL Awarding Body hosts an annual meeting for all moderators, which provides an opportunity to meet as a group, discuss roles, and ensure a consistent approach is taken to communications and the advice and guidance given to centres. External Moderators will not be permitted to continue in post unless they have attended a standardisation event and/or 3 External moderation 3.1 The external moderation visit The external moderation visit will be carried out on the date chosen by, and agreed in advance, with the centre. Prior to the visit, the External Moderator will contact the relevant Course Leader to confirm: The date of the visit The names of people in the external moderation team (if applicable) The expected time of arrival – normally 10.00am The schedule for the day (appendix V) Any other requirements Proposed grades. On the day of the external moderation visit, the External Moderator will: Confirm that Candidate Authentication Forms have been completed for each student Ensure that the Results Sheet has been completed via the online UAL Centre Portal and is available to sign off Review the centre’s approach to internal assessment, confirming it conforms to good practice as outlined in Section 3.5 Identify the sample, unless it has been agreed in advance as outlined in Section 3.3 View the assignment or project where it has been set by the centre, or the project proposal determined by each student in the sample 4 Review all available and admissible evidence for each candidate in the sample, to ensure that grading and awarding reflect the evidence produced for the assessment Determine if agreed standards have been met or not Where standards have not been met and internal assessment decisions are considered to be incorrect, the External Moderator will ask the centre to regrade parts of the cohort, or the entire cohort The External Moderator will not change individual grades Extend the sample by 5% if further evidence is required to confirm that agreed standards are being met Confirm provisional grades with the centre where agreed standards have been met Identify action to be taken where agreed standards have not been met Suggest recommendations that can be implemented by centres at their own discretion Provide additional feedback to the centre, identifying areas for improvement or of good practice Agree the content of the written report. Candidate results will be entered online and endorsed by the External Moderator. Specific details of this process will be made available in a separate document and published online. 3.2 External moderation report The External Moderator will write a report outlining their observations and recommendations and/or actions and submit this to UAL Awarding Body within 10 days of the visit. Report forms require External Moderators to comment on the evidence produced for assessment, assessment methodology, areas of good practice and any recommendations that should be considered by the centre. The report should only contain information discussed with the centre in the feedback session at the end of the visit. Further advice and prompts for the completion of report forms can be found in section 7 and on the UAL Awarding Body website: www.arts.ac.uk/awarding A copy of the External Moderator report will be sent to the centre’s Quality contact and the Course Leader. Centres may provide a response to the external moderation report to UAL Awarding Body. Recommendations and actions to be taken by a centre should be completed within an agreed time frame. 3.3 The sample The sample for UAL Awarding Body qualifications is usually set at 10% of the cohort. This figure ensures that a sufficient number of sets of assessment evidence, covering all disciplines and/or pathways and a range of internal assessors, are externally moderated. Where a cohort is below 50 students, a minimum sample of six is acceptable. The External Moderator will always select the sample. Discussion with the course representative should confirm that any reasonable adjustments have been notified and agreed with UAL Awarding Body. In some instances, where a centre is considered to be low-risk, additional arrangements relating to the sample will apply. These are outlined in Section 3.4. On the day of the external moderation visit, the External Moderator will choose the sample that they wish to see, ensuring that it contains: Grades in all categories (Pass, Merit and Distinction) where a qualification is graded Sufficient examples of the Referral category, to establish the Pass/Referral boundary A range of disciplines, reflecting the range of the qualification experience covered at the centre Assessment evidence assessed by a range of internal assessors. If, after reviewing the initial sample, the External Moderator is uncertain that agreed standards are being applied to assessment and grading decisions, they must extend the sample by a further 5% of candidates. If they are still uncertain that agreed standards are being applied to assessment and grading decisions after extending the sample, the External Moderator must use the procedures set out in section 4.2. Centres must make all possible efforts to facilitate external moderation. Work must be appropriately displayed and/or presented and clearly labelled. 3.4 Risk-based approach to assessment UAL Awarding Body takes a risk-based approach to external moderation. This involves tracking and monitoring external moderation reports in order to make decisions about the risk posed by a particular centre in relation to assessment. External moderation can place a high administrative burden on centres. Through a process of risk management, UAL Awarding Body is keen to reduce this burden for centres with very large cohorts who prove that they have robust systems and processes in place to internally assess and verify student work. With the agreement of UAL Awarding Body, the External Moderator may pre-select the sample to reduce the administrative burden on such centres. The approach taken will vary from qualification to qualification, and will take account of whether a qualification is considered to be ‘high stakes’ or ‘low stakes’. Qualifications designed for progression to the next level are considered to be ‘high stakes’, whilst those that provide enrichment are considered to be ‘low stakes’. All full-time progression qualifications are considered to be ‘high stakes’. At the point of external moderation the centre should make all work available and External Moderators should follow the sampling process outlined in Section 3.3. It is recognised that different centres use different methodologies and nomenclature specific to their culture. UAL Awarding Body’s Centre Handbooks for qualifications outline some examples of good practice for the internal moderation of units. However, all centres are required to ensure that their methodologies are suitably robust as outlined in the UAL Awarding Body Guidance for Good Practice in Assessment and Internal Verification (appendix VII). 3.5 Internally assessed units In addition to moderating assessment and grading decisions for externally moderated units, External Moderators are required to confirm that the internal assessment methodology for internally assessed units is rigorous and ensures assessment decisions are fair, valid, consistent and free from bias. Practically, this means that centres should provide External Moderators with: An outline of the structure of the course assessment and standardisation process (internal verification) Examples of assessment and standardisation records sufficient to enable the External Moderator to confirm that the methodology is working in practice. 5 Additional assessment information 4.1 Referral If a centre identifies a student’s failure to meet assessment criteria following assessment and internal moderation, the centre must refer the student. Centres must discuss the unit assessment criteria that they have failed to meet with the student, and provide them with opportunities to work toward meeting those assessment criteria within a suitable period of time. Candidates are only allowed one opportunity to redeem a referral in each unit. Records of all referrals will be recorded on the results sheet and discussed with the External Moderator A relevant course tutor will agree with the student a time framed action plan for the submission of additional work required to meet the assessment criteria. Centres will notify UAL Awarding Body of the final date for the submission of further evidence. The final date for submission of the required work will be determined by the centre. The centre will assess the student’s re-submitted work against the assessment criteria. If the centre’s previous assessment decisions have been agreed as meeting national standards then no additional visit from an External Moderator is necessary. 6 However, where there have been large numbers of referred students or other reasonable concerns, UAL Awarding Body reserves the right to arrange another visit by an External Moderator. The cost of any additional visit will be borne by the centre. 4.2 Failure to apply assessment and grading criteria to agreed standards If the centre’s proposed assessment decisions are found by the External Moderator/Moderation team not to meet agreed standards then the following procedure will apply: If a small subset of the sample is in question, (e.g. a specific unit) then the centre team will be required to re-assess and re-submit assessment decisions to the External Moderator/ Moderation team for confirmation on the day of the moderation visit If the External Moderator/ Moderation team is able to confirm that the re-submitted decisions are now in line with the assessment criteria, the grades can be authorised If the re-submitted decisions are still not in line with the assessment criteria, the grades for the entire cohort cannot be authorised. A second visit, within a reasonable time scale, will then be made by a Senior External Moderator/ Moderation team. The cost of this additional visit will be met by the Centre. If a resolution cannot be reached the centre may appeal against the assessment decision Where incorrect assessment occurs across the entire cohort, or there is substantial inconsistency in the assessment decisions (e.g. across all units), then the External Moderator/ Moderation team will recommend that the centre team re-assess across the entire cohort. If this cannot be done on the day then the centre and the External Moderator/Moderation team will arrange a mutually convenient time for a second moderation visit. The cost of this additional visit will be met by the centre If the External Moderator is able to confirm that the re-submitted decisions are now in line with national standards then the cohort’s grades can be authorised If the re-submitted decisions are still not in line with the assessment criteria then the grades for the entire cohort cannot be authorised. A further visit, within a reasonable time scale, will then be made by a Senior External Moderator/ Moderation team. The cost of this additional visit will be met by the centre. If a resolution still cannot be reached, the centre may appeal against the external assessment decision. 4.3 Reasonable adjustments A reasonable adjustment is any action that helps reduce the effect of a disability or difficulty that places the candidate at a substantial disadvantage in the assessment situation. Reasonable adjustments are approved before the assessment activity takes place; they constitute an arrangement to give the candidate access to the qualification. It is the responsibility of centres to ensure that appropriate adjustments are made for candidates during the learning process, and UAL Awarding Body will ensure that candidates are given appropriate adjustments to the assessment process to give them an equal opportunity. All adjustments must be deemed reasonable and not give the candidate an unfair advantage There are two types of reasonable adjustment: An adjustment that is centre-devised An adjustment that UAL Awarding Body must approve. Full details of both types of adjustment can be found in the reasonable adjustments policy online. Where a centre applies to UAL Awarding Body for approval, a reasonable adjustments form must be completed. Supporting evidence will be required in all cases. Evidence could include medical reports from doctors, psychiatrists, educational psychologists, specialist teachers, or funding agencies. 4.4 Special consideration Special consideration can be applied after an assessment if there was a reason the candidate may have been disadvantaged during the assessment. These are instances that could not have been predicted, or were outside of the candidate’s control, which may have impacted upon their performance. Examples of special consideration are temporary illness, injury, indisposition or those who suffer adverse circumstances at the time of the assessment. Special consideration may result in a small post-assessment adjustment to the grade of a candidate. The size of the adjustment will depend on the circumstances during the assessment but will always be a minor adjustment so as not to jeopardise the standard. Further details regarding eligibility criteria for special consideration are listed in the special considerations policy online. Special consideration must be applied for no later than two-weeks after the assessment, via the submission of a special consideration application form and accompanied by supporting evidence where appropriate. 4.5 Aegrotat award Aegrotat awards may be awarded in exceptional circumstances, which may include chronic incapacitating illness or death. Aegrotat awards should only be considered where there is no prospect of the student ever being able to be reassessed. Centres who wish to apply for an aegrotat award on behalf of a student, must do so in writing giving a full explanation for their request. The timings of such requests will be dependent on the qualification but must be made before any formal claim for certification is made. Aegrotat awards can only be made if the student has generated sufficient evidence of achievement upon which to base the aegrotat assessment decision. In cases where an aegrotat award is made, the maximum grade achievable is a Pass. of the Arts London website and type ‘safeguarding’ into the search panel: www.arts.ac.uk 4.7 Policies and procedures There are a series if policies, guidance documents and forms that complement this handbook and should be read alongside it. These can be found via the main policy directory on the UAL Awarding Body website: www.arts.ac.uk/awarding 4.8 External assessment To meet DfE requirements for inclusion in Performance Tables for 2016/17, Level 3 Music Performance and Production and Level 3 Fashion Business and Retail require an element of external assessment. Please refer to the qualification specification documents and the UAL Awarding Body website for further guidance: www.arts.ac.uk/awarding 4.6 Awareness of safeguarding UAL Awarding Body recognises that it has a responsibility to safeguard the wellbeing of children, young persons and vulnerable adults engaged in the learning of our qualifications. We do so by adhering to the University of the Arts London Safeguarding Policy, which has been developed in collaboration with the NSPCC to comply with statutory duties. For further information, please visit the main University 7 Allocation and remuneration of External Moderatorsw 5.1 Allocations UAL Awarding Body approved centres are each allocated one or more External Moderator, to undertake a minimum of one on-site moderation visit per year. Centres are normally allocated the same moderator(s) for a three year period although moderators’ contracts are renewed annually. Each year, External Moderators are requested to confirm their availability for the peak moderation period (April to July). This is undertaken via the online UAL Centre Portal and detailed guidance is provided. At the same time, External Moderators complete and submit a signed conflict of interest declaration. If a Moderator is employed by a UAL Awarding Body approved centre, the name of the centre must be included in the declaration. Similarly, any centres that could be viewed as potential competitors to a centre at which a Moderator is employed should be identified. Simultaneously, centres notify UAL Awarding Body of the dates of their exhibitions and when candidate work will be available to moderate. 8 The UAL Awarding Body Quality and Operations team then matches moderator availability and expertise against these dates - taking in to account conflicts of interests - and allocates one or more moderators to each centre. The provisional list of visits is made available to Moderators via the online UAL Centre Portal in March. Moderators have the opportunity to accept the proposed visits, or by exception, decline visits if their availability has changed since their original submission. Once confirmed, Moderators are able to view, download and print a detailed itinerary to take with them on their visits. This provides centre contact details, the size of sample to be moderated, specialist pathways, and if applicable, contact details of other External Moderators attending the same visit. 5.2 Travel and accommodation booking All travel and accommodation bookings are confirmed well in advance of the external moderation visit taking place. The cost will be met by UAL Awarding Body in line with agreed financial limits. 5.3 Remuneration and claims The fee for undertaking External Moderation duties includes the activity attached to the visit and the completion and submission of the moderation report. The parameters for the remuneration of External Moderators are defined in the UAL Awarding Body notes for the allocation and remuneration of External Moderators to UAL Awarding Body Centres document (appendix VI). Guidance on claiming fees and expenses can be found in the policy directory available online. All claim forms and documentation to support external moderation can be found on the UAL Awarding Body website: www.arts.ac.uk/awarding Contacts 6.1 Contacts at UAL Awarding Body Name Role Responsibility Contact details Sarah Atkinson Head of Academic Standards Management of Senior External Moderators, External Moderator team, External Moderator observations and advice and guidance for all UAL Awarding Body qualifications s.j.atkinson@arts.ac.uk Andy Sankey Chief Examiner — Music and Media Management of External Moderation team, advice and guidance for Music and Media and qualifications a.sankey@arts.ac.uk Marc Mollica Chief Examiner — Performing Arts Management of External Moderation team, advice and guidance for Performing Arts qualifications m.mollica@arts.ac.uk Martin Vella Chief Examiner — Art & Design Management of External Moderation team, advice and guidance for Art and Design qualifications m.vella@arts.ac.uk Sue Cook Chief Examiner — Foundation Studies — Art & Design Management of External Moderation team, advice and guidance for Foundation Studies — Art and Design sue.cook@arts.ac.uk All administrative support moderation.awarding@arts.ac.uk Quality and Operations team All External Moderators will be allocated a Senior External Moderator to contact for advice and guidance. 9 Checklist 7.1 Checklist This external moderation checklist is relevant to all UAL Awarding Body qualifications, and is shared with UAL centres and UAL External Moderators. The aim of this list is to act as a reminder for some of the key activities associated No. 10 with external moderation and the certification process. It should be used in conjunction with the Centre Handbook specific to the qualification being delivered and assessed. Please note the highlighted areas for action by External Moderators below. Activity Who When 1 Submit registrations to UAL Awarding Body via the online UAL Centre Portal. Centre October 2 Submit preferred external moderation dates to UAL Awarding Body. Centre December 3 External Moderators contact centres to confirm the arrangements for the external moderation visits. Lead External Moderator to contact External Moderator(s) in their team(s). External Moderators Minimum of four weeks before each external moderation visit 4 Allocate all visits to the appropriate External Moderators. UAL Awarding Body Quality and Operations team January and February 5 Confirm or decline proposed external moderation visits via the online portal Quartzweb. Once completed a personalised itinerary will be available to view and download which summarises all visits. UAL Awarding Body Quality and Operations team March 6 Publish details of confirmed external moderation visits to centres. UAL Awarding Body Quality and Operations team March 7 External Moderators to confirm bookings for travel and accommodation External Moderators Minimum of four weeks before each visit 8 Centre to prepare student work for external moderation visit as outlined in the Centre Handbook Centre By date of visit 9 Endorse candidate results via the online UAL Centre Portal. External Moderators On day of visit or maximum five working days post visit 10 External Moderators complete report and submit via the online UAL Centre Portal. External Moderators Within 10 days of visit 11 Process candidates’ results and issue certificates to centres. UAL Awarding Body Quality and Operations team Following External Moderator endorsement Completing the Report Form Reports should be completed in the third person and grammar and spelling should be corrected before submitting them to UAL Awarding Body. External Moderators are advised to complete the reports using typeface Arial or Calibri, with a minimum font size of 11. Part 1 Candidate sample The candidate sample must be at least 10% of the cohort. The candidate sample may be extended by an additional 5% if it is necessary to clarify the accuracy of grading decisions. Candidates in the sample should be identified only by their Unique Learner Number (ULN) which can be retrieved from the Course Leader. Part 2 Candidate authentication forms Indicate either YES/NO to confirm that Candidate Authentication Forms were made available at the point of external moderation. If the response is no this must be an action for the following year. Part 3 Profile of assessment evidence This section should be used to report on the general strengths of the candidate evidence in relation to unit aims and assessment criteria. External Moderators should write an introduction to the course to place the report in context this should note the following: Number of candidates in the cohort Number of candidates in the sample How the course is delivered - across specialist areas or pathways List the pathways and select a representative sample as % of each area (10-15% of each) Number and % of D / M / P / R/W/F Any special consideration or reasonable adjustment should be supported by the correct UAL paperwork. External Moderators should use sub-headings for this section of the report and comment on any areas of strength or good practice and any areas for development or improvement, it is important to note differences between specialist areas or pathways. External Moderators may identify but not name candidates as examples by using the ULN. Within the subheadings outlined in the grade criteria for the qualification and level being assessed, External Moderators should select the appropriate definition to comment on from the following examples: Context Quality of project proposals Word-processed Articulate Comprehensive Level of challenge enabling students to meet higher criteria Critical and contextual references Relevance and depth of bibliography Harvard referencing Planning and evaluation. Research Thoroughness Appropriateness Relevance Integration throughout the project Contextual understanding Impact on practice Primary and secondary Extended research. Problem Solving Breadth of experimentation Recording methodology Practical and technical challenge Investigation and decision making Initiative and ability to adapt to unforeseen challenges Measurement of impact Intention, content and theoretical challenge. Planning and production Ability to work effectively within a timeframe Effectiveness in collaboration Accessing resources Documentation of planning and production Self-direction. Practical skills Level and application of skills across all pathways Access to resources Consistency of technical support Range of available materials Supportive teaching practices for technical skills Ambition and quality of outcomes. Evaluation and reflection Regularity of ongoing reflection Recording methodology Inclusion of peer and self-assessment Level of visual analysis Level of written analysis Self-awareness and perception Quality of final evaluative statement. Presentation Quality of exhibition, display or performance of work Professionalism of presentation Strategies explored and considered within constraints Organisation and presentation of supporting work Clarity of communication to the audience Independence in realisation. 11 Completing the Report Form Part 4 Assessment methodology This section should be used to confirm that the centre’s assessment methodology conforms to good practice. If the assessment methodologies are unsound, the External Moderator should provide a commentary on those aspects, which require re-design or strengthening. These should be noted as actions and a date for completion agreed with the centre. External Moderators should use this section to confirm, through discussion with the assessment team and examination of relevant records, that: Assessment paperwork is fit for purpose, assessment judgments clearly relate evidence to the criteria and feedback is relevant and constructive Assessment methodology for internally assessed units is rigorous and ensures assessment decisions are fair, valid and free from bias. This should include a description of the centre assessment process e.g. team, blind marking, cross pathway marking etc. Assessment, internal verification and standardisation processes are suitably robust (centres should provide the External Moderator with examples of assessment and Internal Verification paperwork for internally assessed units) Assessment and Internal Verification documentation clearly evidences the decision making process in ‘borderline candidates’ 12 Assessors refer to the UAL Awarding Body exemplification matrices at the relevant level to ensure consistency of grading decisions Referred students are given clear feedback and time frame for re-submission Grading decisions across all pathways are consistent The External Moderator is able to confirm that assessment decisions made by the centre are consistent with national standards. Part 6 Recommendations This section should be used to outline any recommendations that should be considered by the centre, including areas for development based on the findings of the external moderation visit above. The External Moderator should acknowledge any recommendations from the previous years report that have been acted on and any resulting improvements. The External Moderator should also note examples of good practice and any recommendations. Recommendations are for guidance only and can be carried out at the discretion of the centre. Part 5 Part 7 This section should be used to provide a brief summary of your conclusions identifying areas of good practice as appropriate e.g. If the responses to any of the above sections require actions to be taken by the centre or UAL Awarding Body, the External Moderator should complete this section. General commentary Use of handouts in introducing the final unit Students are clear in their understanding of criteria and expectations as evidenced in self-assessment Opportunities for tutorials, group critiques and peer review Employer engagement Quality of project proposals Range of topics Breadth and depth of research in informing project development Level of investigation and experimentation Quality of centre documentation Student progression Overall quality of work by pathway. Action plan Actions should be agreed, signed and dated by the centre representative at the time of the external moderation visit. The centre should retain one copy of the action plan and one copy should be included in the report sent to UAL Awarding Body. Actions are essential measures for improving centre standards for assessment and must be carried out by the agreed date. The External Moderator must acknowledge that any previous actions placed on the centre have been completed. Before sending the report to UAL Awarding Body the External Moderator should ensure that they have completed the details and information on the front cover sheet. This needs to include the following: The level of the qualification The full qualification title Details of the pathways moderated The name of the Course Leader. Appendix I Glossary Aegrotat award Centre An award made in exceptional circumstances where there is no prospect of the student ever being able to be reassessed. An organisation or consortium accountable to an awarding body for the assessment arrangements leading to a qualification or units. Appeal Certificate A process through which an awarding body may be challenged on the outcome of an enquiry about results or, where appropriate, other procedural decisions affecting a centre or individual. Assessment The process of making judgements about the extent to which a candidate’s work meets the assessment criteria for a qualification or unit. The record of attainment in a unit or qualification issued by the awarding body. External Moderator An individual appointed by the awarding body to ensure accurate and consistent standards of assessment across centres and over time. Presiding over the decision / discussion / mediation Qualification An award made to a student for the achievement of the specified combination of credits. Quartz The online UAL Centre Portal. Reasonable adjustment Adjustments made to an assessment for a qualification before the activity takes place. They help reduce the effect of a disability or difficulty that places the candidate at a substantial disadvantage in the assessment situation. The requirements that candidates need to meet in order to successfully complete the learning outcomes for a unit or qualification. Records details of candidates sampled for the external moderation and a commentary on the conduct of the internal assessment and grading methodology at each centre. Authentication Internal assessment Special consideration Assessment criteria Confirmation that evidence was produced by the candidate who is putting it forward for assessment (in the form of a declaration of authenticity by the candidate). Candidate A person who is registered with an awarding body for a qualification or unit. Assessment where candidate’s work is assessed wholly within the candidate’s centre, subject where appropriate to external moderation. Internal verifier Individual(s) appointed by the centre to ensure accurate and consistent standards of assessment across assessors. A process to ensure that the assessment criteria for a qualification or unit are applied consistently by assessors. Standardisation can be carried out within centres, (internal moderation) as well as by awarding bodies across their centres. Unit The smallest part of a qualification that is capable of certification in its own right. Results sheet Electronic record of achievement indicating candidates’ achievement. Results are entered online via the online UAL Centre Portal and endorsed by the External Moderator responsible for moderating that particular cohort. External moderation final visit report form Standardisation of assessment Consideration given to a student who has temporarily experienced an illness or injury, or some other event outside of the student’s control, which has had, or is reasonably likely to have had, a material effect on that their ability to take an assessment or demonstrate his or her level of attainment in an assessment. 13 Appendix II Job description Overall purpose of the role: To act on behalf of the University of the Arts London Awarding Body (UAL Awarding Body) to: Verify, through the process of external moderation, the achievement of students leading to the award of credit for UAL Awarding Body accredited provision Ensure the quality and consistency of standards within and across centres offering UAL Awarding Body accredited provision Support the continuous improvement of UAL Awarding Body accredited provision. 14 Main duties and responsibilities: 1.Carry out the process of external moderation for a specified qualification 2.Visit centres as stipulated on work profile / agreement 3.Ensure that UAL Awarding Body accredited units are delivered in accordance with the approved specifications 4.Sample assessment activity and evidence to ensure the consistency of award of credit 5.Actively contribute to UAL Awarding Body standardisation activity and attend external moderator standardisation meetings arranged by UAL Awarding Body 6.Monitor and report on internal moderation for the centres/qualifications for which they are responsible 7.Attend induction and training as required by the UAL Awarding Body 8.Produce and submit robust and timely moderation reports to UAL Awarding Body 9.Verify the award of credit to students by signing off relevant documentation 10.Respond promptly to requests from UAL Awarding Body for information relating to moderation activity 11.Provide critical advice to centres in order to support the continuous improvement of UAL Awarding Body accredited provision and not bring UAL Awarding Body into disrepute 12.Work closely and collaboratively with the Head of Academic Standards, Chief Examiners and other UAL Awarding Body staff. Appendix III Role of the Lead Moderator For centres with large cohorts, UAL Awarding Body has a policy of sending in a team of External Moderators to view the sample of student work and confirm that appropriate internal quality assurance mechanisms are being applied. In order to ensure that this team of External Moderators functions effectively, UAL Awarding Body appoints a Lead Moderator who will carry out further functions in addition to those of the team members and external moderators working on their own within a centre. The main duties and responsibilities of the Lead Moderator are: Communicating with the Course Leader at the centre being externally moderated to confirm final arrangements for the external moderation visit, including the start and finish times and the agenda for the day Liaising with and coordinating the team of External Moderators to ensure consistency in the operation of moderation across the course Becoming familiar with the centre to be externally moderated, its internal assessment /verification arrangements, the delivery of the course and any relevant background information and to communicate this to the External Moderation team Seeking guidance from the UAL Awarding Body Chief Examiners and Senior Quality Officer regarding any issues of concern Coordinating and writing the final External Moderation Report and submitting this to UAL Awarding Body Signing off the grades proposed by the centre Summarising the findings of the External Moderation visit and feeding these back to the course leader and course delivery team. Role of the Senior Moderator Senior External Moderators (SEM) are appointed to assist the Chief Examiners in providing a consistent approach to quality assurance for centres. In addition to external moderation activity, the SEM will carry out allocated duties as set out below: Be part of a senior team which sets the tone for all UAL Awarding Body external moderators, representing the UAL Awarding Body approach and ethos Provide support and guidance to UAL Awarding Body External Moderators to ensure consistency of approach and to resolve any issues Support the Chief Examiner in the delivery of annual Chief Examiner Reports for a specified qualification Carry out observations of External Moderators and complete reports on their performance Read and review external moderation reports completed by UAL External Moderators and approve their release to centres Carry out advisory visits to UAL Awarding Body centres Participate in and / or lead qualification specific events such as standardisation events, qualification training meetings etc. 15 Appendix IV External Moderator Observation Report Aim To provide UAL Awarding Body with a record of External Moderator performance. The criteria for observation and evaluation of External Moderator performance are related to the External Moderator job description and person specification. Name of External Moderator: Date of observation: Centre at which observation took place: Qualification externally moderated: The External Moderator can demonstrate the ability to apply assessment and grading criteria to agreed standards The External Moderator can demonstrate the ability to effectively communicate judgements The External Moderator can demonstrate the ability to work effectively with others Recommendation Lead EM Team EM Solo EM Additional comments: Name of Observer: Signature of observer: Date: 16 Appendix V External Moderation visit: sample contact letter and schedule Dear name of Course Leader/centre contact Name of institute – name and level of qualification I am writing to confirm arrangements for the forthcoming name of course moderation visit, and I am attaching a proposed schedule for the day. External moderation visit contact Letter and schedule – date To ensure that the assessment process is sound and that the standard meets the UAL Awarding Body assessment criteria for level of course, I will be moderating a sample of completed and formatively assessed assignments for the name of course – unit number. The sample size for the qualification will be 10% of the cohort (or a minimum of six) and the actual candidates decided in discussion with you on the day of moderation, to ensure that a representative range is included. Contact: name Moderation team: name(s) Please have available: schemes of work, assignment briefs, list of assessors, completed assessment sheets, internal verification and tracking documentation for all units and completed final results sheets with student ULNs. Please refer to section 1.6 of the centre handbook and ensure that all students complete and present a statement of authenticity with their work. The moderation team will be myself and insert name (if applicable), cc’d. AM 10.00am – Arrival and meeting with Course Leader to discuss sample • Qualification delivery model • Size of cohort • Scheme of work • Sample assignments and related support materials • Assessment methodology and IV • Achievement and marking • Select sample Please do not hesitate to contact me if you have any queries. 10.30am – External moderation team to view sample I look forward to seeing you on confirm date of visit. 12.30 – Lunch break Best wishes, PM 1.30pm – External moderation team to complete sample I will also need to see evidence of the tracking, monitoring, assessment and internal verification of the non-graded units. Please ensure that schemes of work, assignment briefs and assessment tracking for unit numbers are made available. It is not necessary to present student work for these preceding units. Your name and contact details 3.30pm – External moderation team to discuss and agree feedback to Course Leader and team 4.00pm – F eedback to Course Leader and team/sign-off Grade Confirmation Form 17 Appendix VI Parameters for the allocation and remuneration of External Moderators to UAL Awarding Body centres Allocation An External Moderator can be allocated to a centre for up to three years but is contracted annually An External Moderator can be re-allocated, subject to an acceptable standard of performance, every three years to ensure clarity of judgements and to extend experience across a range of centres. Length of working day The normal External Moderator working day is six hours, of which approximately one hour should be spent receiving guidance from the Course Leader on the location and format of the evidence and the proposed grades (if allocated). External Moderators should also spend this time reporting judgments and conclusions, and double-checking evidence where necessary. Full-time course workload On average, External Moderators are expected to review the evidence of three externally moderated units per hour. This takes into account that some sets of evidence will require more time and others less An External Moderator for one of the large qualifications should therefore be able to moderate up to 15 sets of evidence in a day. 18 Short courses workload For stand-alone units, External Moderators are expected to review an average of four sets of evidence for an Award (two units) or three sets of evidence for a Certificate (four units) per hour External Moderators should be able to moderate up to 20 sets of evidence per day for a stand-alone award, and up to 15 sets of evidence per day for a stand-alone Certificate For units embedded in a ‘host’ qualification, External Moderators are expected to review an average of three sets of evidence for an Award and two sets of evidence for a Certificate per hour External Moderators for an embedded Award should be able to moderate up to 15 sets of evidence per day, or up to 12 sets of evidence per day for an embedded Certificate. Sample size The normal size of a sample for external moderation is 10%, although this may be extended by 5% if External Moderators are unclear if agreed standards are being met A ratio between registrations at a centre and the number of External Moderators required can therefore be established, e.g. a 10% sample from a centre with 100 Foundation Diploma in Art & Design candidates would be 10 requiring one External Moderator for one day. Remuneration The fee for undertaking External Moderation duties includes the activity of moderation and the production of a report within the parameters set by the Awarding Body When External Moderators work in teams, the Lead External Moderator will be remunerated at a higher rate for coordinating the activities of the team. The report should be based on contributions from all members of the team An External Moderator attending a centre to moderate two qualifications on the same day is required to produce separate reports e.g. Foundation Diploma in Art & Design and Short course Drawing, and will receive payment for both qualifications. Definitions of actions and recommendations Actions Advice given by an external moderator to a centre Essential measures for improving centre delivery, to meet quality assurance standards for assessment processes and for the continuing validation of UAL Awarding Body Actions to be taken within a period of time agreed between UAL Awarding Body and the centre; this would usually be prior to the final assessment of the next completing cohort. Recommendations Guidance given by an External Moderator to a centre Desirable measures for continuing improvement to centre delivery, quality assurance standards and assessment processes Recommendations to be carried out at the discretion of the centre in discussion with UAL Awarding Body. Appendix VII Guidance for good practice on assessment and internal verification Formative assessment A formative assessment tutorial supported by verbal and written feedback on achievement and areas for improvement should be given at least once during the delivery of a unit or units. This gives the tutor an opportunity to monitor the progress of a student and to ensure that actions are taken to address any areas of concern prior to project deadlines. Self-assessment and peer assessment Evaluation and reflection should be an ongoing part of the student learning experience. Centres should also encourage students to self-assess against the assessment criteria prior to a formative or summative assessment. The process of self-assessment reinforces an understanding of how assessment decisions are arrived at and enables students to identify their own strengths and areas for improvement. Assessment for nongraded units Initial assessment should be undertaken by the one or more members of the teaching team. Ideally each student should receive a one-to-one assessment tutorial to provide them with an opportunity to present and explain their work and achievements. Verbal and written feedback against the assessment criteria should be given, with tutorial notes and action plans noted. Indicative grading feedback may be used. Any borderline decisions should be discussed and agreed with another member of the course team. Summative assessment for graded units Internal standardisation To ensure parity of grading decisions the teaching team should agree benchmarks for Pass, Merit and Distinction across disciplines before assessing the whole cohort. Internal verification To check the accuracy of individual assessment decisions the Internal Verifier should select a sample of at least 10% of the assessed cohort. The sample should represent grading decisions across all levels and disciplines. In addition, all disputed and borderline decisions should be seen and agreed by the Internal Verifier. Referrals The criteria where a student has failed to reach the required level should be made clear. Students should receive written feedback and guidance against the criteria on what they must do to achieve a Pass. Ideally the student should also be given the opportunity to attend a tutorial. An appropriate date for re-submission should be given – students are allowed only one opportunity to resubmit. On re-submission referred students are always capped at Pass. The initial summative assessment should be undertaken independently by one or more members of the teaching team. Blind marking and double marking are recommended. Student work should always be assessed against the unit assessment and grading criteria. Once assessment is complete, the team should meet to discuss and agree decisions. Minutes of discussion on borderline, disputed and subsequently agreed grades should be recorded and presented to the external moderator. Where assessors have recorded different grades and no agreement is reached these submissions should be added to the internal verification sample. 19 Want to find out more? Contact: UAL Awarding Body 272 High Holborn London WC1V 7EY Tel: 0207 514 9851 Email: moderation.awarding@arts.ac.uk UALawardingbody UALawardingbody Version 5.5 arts.ac.uk/awarding