VOCATIONAL EDUCATION & TRAINING ASSESSMENT VALIDATION KIT 2012 CONTENTS Introduction ....................................................................................................... 3 Assessment Validation and the Role of the VET Coordinator ........................... 4 Assessment Validation and the Role of the VET Teacher .................................. 5 Instructions for the use of Validation Templates and Checklists ...................... 6 VET Course Assessment Plan ............................................................................. 8 Assessment Task Cover Sheet ............................................................................ 9 Assessment Design Checklist for Teachers ...................................................... 10 Assessment Validation Cover Sheet................................................................. 11 Validation Meeting Participation Record ......................................................... 12 Pre-Assessment Validation Checklist ............................................................... 13 Post-Assessment Validation Checklist ............................................................. 14 Competency Assessment Event Checklist ........................................................ 15 Assessment Validation Checklist ...................................................................... 16 Validation of Assessment Tools ....................................................................... 17 Assessment Validation Kit Assessment Schedule Validation Checklist ...................................................... 19 2 VET Assessment Validation Kit Introduction The purpose of this Assessment Validation Kit is to equip VET Coordinators and VET Teachers with the tools required to implement a process for the validation of assessments conducted by VET teachers. The Kit is designed to be used by VET Coordinators and VET teachers. This Kit provides procedures for: Development of validation action plans Guidance for VET teachers in the design and presentation of assessments Checklists and Record-keeping templates for use in the conduct of validation meetings and activities All the checklists and templates have been designed using Microsoft Word and can therefore be modified to include school logos etc. Assessment Validation Kit An electronic version of the kit can be downloaded from the CEO website. 3 Assessment Validation and the Role of the VET Coordinator The VET Coordinator has a vital role to play in ensuring that effective assessment validation practices are implemented and documented in each school. Validation of assessment at faculty meetings should take place at least twice per year: once in Term 1, where VET teachers can validate their Assessment Plans for the year; and once during Term 4, where specific completed assessment tasks may be validated. It is not feasible that all assessment tasks be validated each year. VET Coordinators should ensure that all teachers are involved in validation of a representative sample of tasks. All assessment validation activities should be documented. The VET Coordinator is responsible for: Assessment Validation Kit The Development of a Validation Action Plan for all VET courses and for all VET teachers at his or her school Facilitating VET faculty meetings where the VET teachers: o are provided with information regarding assessment validation requirements o to conduct validation activities Being the first point of reference for VET teachers requiring information and guidance with assessment validation Ensuring new VET teachers have a suitable “mentor” to provide support with all aspects of VET delivery and assessment. “Mentors” may be another experienced VET teacher from within the school or from another school. Maintaining records of all assessment validation activities and documenting any action taken to improve the quality and consistency of assessment including samples. Providing assessment validation records to the CEO as required to meet NVR Standards for Registered Training Organisations 4 Assessment Validation and the Role of the VET Teacher VET Teachers are responsible for: Conducting assessments of their students that meet the requirements of the various Training Packages for each of the Industry Curriculum Frameworks. Generally, these requirements can be found in the relevant BOS Syllabus documents. Designing an Assessment Plan for each VET course using the template included in this kit Participating in assessment validation activities facilitated as part of VET faculty meetings at schools. Validation of assessment at faculty meetings should take place at least twice per year: once in Term 1, where VET teachers can validate their Assessment Plans for the year; and once during Term 4, where specific completed assessment tasks may be validated. Ensuring that a representative sample of assessment tasks is validated each year. Participating in assessment validation activities facilitated by the CEO as part of VET Teacher Network Meetings Documenting any other assessment validation activities undertaken with the aim of improving the quality and consistency of assessments Cooperate with VET Coordinators in maintaining records of all assessment validation activities and documenting any action taken to improve the quality and consistency of assessment Providing assessment validation records to the CEO as required to meet NVR Standards for Registered Training Organisations. Assessment Validation Kit 5 Instructions for the use of Validation Templates and Checklists The following templates and checklists are provided for VET Coordinators and VET Teachers to use to conduct and record Assessment Validation strategies: 1. 2. 3. 4. 5. 6. 7. 1. Assessment Plan Template VET Assessment Task Cover Sheet Assessment Design Checklist for Teachers Assessment Validation Materials Cover Sheet Assessment Validation Participant Record Pre-assessment Validation Checklist Post-Assessment Validation Checklist Assessment Plan Template The VET Course Assessment Plan is designed for use by VET Teachers to list all assessment tasks/events to be used to assess students in each course. Each VET Teacher should have a VET Assessment Plan for each VET course delivered. VET Coordinators should also keep copies of the VET Assessment Plans. Copies may also be given to VET students. Assessment Dates may be exact or cover a period of time to indicate a starting or completion date for particular assessment tasks. The Validation Activity column can be used to record particular validation strategies undertaken by teachers. Examples of Validation Activities include: Pre-assessment & Post-assessment Validation (at faculty or VET Teacher network meetings) Collaboration with colleagues in setting tasks Use of commercially developed/ network developed assessment materials Mentor Meetings Joint marking of assessment tasks 2. VET Assessment Task Cover Sheet The VET Assessment Task Cover Sheet is designed for the use of VET teachers in the design and presentation of VET Assessment tasks to students. The use of this cover sheet should ensure consistency in the way students are informed of assessment requirements. Assessment Validation Kit It is not necessary to validate every assessment task. It is the responsibility of the VET Teacher & VET Coordinator to determine what sampling of assessment tasks should be validated. It would seem reasonable to suggest that each year, each VET Teacher present at least two assessment tasks for validation at either VET faculty meetings held at the school or VET teacher network meetings. 6 3. Assessment Design Checklist for Teachers This checklist is for VET teachers to use for self-assessment when planning and designing assessment tasks. VET teachers should keep records of their use of this checklist as evidence of any action taken to improve the quality and consistency of assessment. 4. Assessment Validation Materials Cover Sheet This sheet is for use by VET teachers presenting assessment tasks for validation at VET faculty and network meetings. The cover sheet should be completed and attached to tasks. Please note that it may be necessary to provide multiple copies of tasks presented for validation at meetings. 5. Assessment Validation Participant Record This sheet is for the use of those facilitating assessment validation meetings (usually VET Coordinators). VET Coordinators should keep records of Assessment Validation Participation. 6. Pre-assessment Validation Checklist This checklist is for use as part of an assessment validation meeting. Participants should use the checklist to discuss an assessment task. The Comments column can be used to affirm “good practice” and also to make suggestions for improvements to the design and presentation of tasks. Completed Pre-assessment Validation Checklists should be kept by VET Coordinators with a copy returned to the relevant VET teacher. Post-assessment Validation Checklist This checklist is for use as part of an assessment validation meeting where teachers meet to discuss an assessment task that has been completed by a student. Examples might include completed and marked examinations, assignments and projects as well as evidence collected from practical tests. As for the Pre-assessment Validation Checklist, the Comments column can be used to affirm “good practice” and also to make suggestions for improvements to the design, presentation and marking standard of tasks. Completed Post-assessment Validation Checklists should be kept by VET Coordinators with a copy returned to the relevant VET teacher. 8. Appendix: Optional Checklists and Templates Assessment Validation Kit 7. 7 VET Course Assessment Plan School: Year: Unit/s of Competency Training Package: Qualification Title: Teacher: Assessment Event Assessment Date Validation Activity Validation Date Assessment Validation Kit Course: Qualification Code: 8 Assessment Task Cover Sheet Course: Qualification Code: Qualification Title: Training Package: Student Name: Teacher/Assessor: Assessment Task Name: Assessment Type: Date Issued: Date Due: Date Handed in: Unit/s of Competency being assessed in this task Unit Code: Unit Name: Will you need additional help to complete this task? Yes No Will you be seeking RPL for this Assessment Task Yes No If you have answered yes to either of these questions speak to your teacher. Employability Skills Self-management Initiative and enterprise Learning Communication Teamwork Planning and organising Problem solving Technology Cross Cultural Understanding Task description and instructions to student (requirements, procedures, diagrams) Assessment Validation Kit (Attach extra documents) 9 Assessment Design Checklist for Teachers Use this checklist to assist you to design your assessment tools and strategies. Gauge your assessment against the following statements, and where you are unable to answer YES, re-work your approach. Course: School: Qualification Code: Qualification Title Assessment Task: Training Package: Year 9 Year 10 Year 11 Year 12 Teacher: Validity Yes/No Comment: The assessment tasks are based on realistic workplace activities Yes No The assessment tasks have been designed to allow for holistic and integrated assessment Yes No Reliability Yes/No Exemplars and checklists have been designed for the collection of evidence Yes No Instructions for students are clear and consistent Yes No Students are provided with guidelines on requirements when work samples are used as evidence of competency Yes No Flexibility Yes/No The assessment approach can be adapted to meet the needs of all candidates Yes Fairness Yes/No Info for students covers methods, procedures, the criteria against which they will be assessed, when and how they will receive feedback and the mechanism for appeal Yes Comment: Comment: No Comment: No Assessment Validation Kit Year: 10 Assessment Validation Cover Sheet Course: Qualification Code: Qualification Title: Training Package: Unit(s) of Competency: Assessment Writers: School: Validation Date: Validation Panel: Documents provided for validation process Assessment Tools & Marking Guides Assessment Writer N/A Comments if applicable by the Validation Panel Practical observation checklist Quiz / Test / Oral question list Tool for collecting third party evidence Assignment or case study Portfolio / journal Other - please specify Answer guide / Sample of expected outcome/s, project, Expected outcomes checklist etc Course Training & Optional: May include timeline, topics, activities, units of Assessment Plan / competency and assessment events Program Syllabus Document - Highlighting unit/s covered by the assessment tool/s Information to students about the assessment task/s and/or event/s. Written instructions for students must include points in next column * indicates provided within the student guide Assessment Writer N/A Validation Panel N/A Instructions clearly explain the task/s: - procedure - equipment required - resources provided - evidence to be collected Conditions: - timing - location - timing and mechanism for feedback - allowable adjustment * - right of appeal * Do the tools clearly identify: - the candidate, assessor and date - unit/s of competency being assessed - training package and national code - how the tool will be used, eg 1 of 3 tasks Are the instructions in plain language and appropriate to the intended audience? Comments by the Validation Panel: _____________________________________________________________________________________ __________________________________________________________________________________________________________________ Assessment Validation Kit The following checklists can be used when designing assessment tools as well as in the assessment validation process. 11 Validation Meeting Participation Record Date: Venue: Name of participating Assessors/Teachers Task to be Assessed / Competency Delivery Site / School Proposed Assessment Dates Description of Activities Undertaken during Validation Meeting: Validation Coordinator: ________________________________________________ Date: _______________________ Assessment Validation Kit Follow up Action required: 12 Pre-Assessment Validation Checklist Use this checklist as part of an assessment validation meeting to evaluate assessment tools, tasks or strategies. Course: Qualification Code: Qualification Title: Training Package: Assessment Task: School: Year: Yr 11 Yr 12 Teacher(s): Assessment Activity Yes/No Action - Clearly describe the adjustment needed Assessment task instructions and assessment conditions are clearly identified. Yes No Written information is worded appropriately. Yes No The assessment activity addresses the evidence requirements for the competency or competencies being assessed Yes No The level of difficulty of the activity is appropriate to the competency or competencies being assessed. Yes No Evidence guides or assessment checklists are available for use in making an assessment decision. Yes No If NO modification required (as identified under actions): Date: Changes made to assessment items: Yes No Date: Assessment task is ready for use: Yes No Date: Names of Validators (print name, school & sign) Print School Name: Signature: Date: Assessment Validation Kit Print Name: 13 Post-Assessment Validation Checklist Use this checklist as part of an assessment validation meeting to evaluate assessment evidence provided to meet particular performance criteria. Course: Qualification Code: Qualification Title: Training Package: Assessment Task: School: Year: Yr 11 Yr 12 Teacher(s): Describe briefly the items of evidence provided by students to meet the performance criteria of the task (eg. powerpoint, report, product, plated food) Evidence criteria Yes/No Valid Is the evidence provided relevant to what is being assessed? Have a variety of assessment strategies been used? Are skills and knowledge assessed in an integrated way? Sufficient Is the evidence provided by this assessment sufficient for a judgment of competence? If NO, What additional evidence is required? Consistent Is the evidence consistent with industry benchmarks? Can any inconsistencies between this evidence and that expected in industry be identified? Authentic Can you verify the evidence provided is the work of the candidate? Does the evidence provided demonstrate real work in relation to the performance criteria? Action - Clearly describe the adjustment needed Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No If NO modification required (as identified under actions): Date: Changes made to assessment items: Yes No Date: Assessment task is ready for use: Yes No Date: Names of Validators (print name, school & sign) Print School Name: Signature: Date: Assessment Validation Kit Print Name: 14 Appendix: Optional Support Documents Competency Assessment Event Checklist In designing competency assessment events for VET courses, it is important to ensure that assessment events include: Industry Curriculum Framework and VET Board Endorsed Courses Training Package code and name Qualification code and name Competency assessment event title An indication of suitability for HSC or Preliminary course Competency assessment date (proposed) Units of competency being assessed Elements being assessed Performance criteria (statements) being assessed Task description and instructions to student Additional material or attachments if required Student feedback, areas for improvement and appeals option Employability skills checkbox AND for the teacher to include: Checklist for assessment - the list of holistic criteria that integrates the task description with the elements and performance criteria being assessed Further questions to consider: How well does the event/task address the elements and performance criteria selected? Have the BOS key terms been selected to provide clear directions to students, where appropriate? Who is validating the task? Assessment Validation Kit Has spelling and formatting been checked? 15 Appendix: Optional Support Documents Assessment Validation Checklist Description Does the task/s assess the requirements of the unit/s of competency Assessment task/s and tool/s cover criteria within the evidence guide. Is the task/s sufficient to cover the performance criteria, critical aspects, essential knowledge and skills, and range of variables within the element/s and unit? Is the task assessing what it should be assessing Is the task assessing exactly what the syllabus and the evidence guidelines indicate. No additional outcomes that are beyond the unit of competency requirements. Eg No HSC components or higher level skills assessed. Is the task/s based on realistic workplace activities and contexts Task/s based on realistic workplace activities and contexts, for example conditions, volume, use of appropriate equipment, machinery and /or methods. Does the task/s reflect the AQF level of performance required by the unit/s of competency. Task/s meet the AQF level of performance required by the performance criteria. Explanations of performance level of each AQF certificate levels is outlined in Part A of the syllabus document headed AQF VET Qualifications. Does the task/s allow for integrated assessment of underpinning knowledge, skills and attitudes. Is the task able to assess the underpinning knowledge and skills that is required to apply the skill in another location or with different equipment etc? Is the attitude appropriate to a work place situation? Are OH&S procedures being followed for example. Does the task/s allow for holistic and integrated assessment of the key competencies / employability skills Key competencies are generic employment related competencies that are essential for effective participation in the workplace, for example, teamwork, communication, use of technology etc. See explanation at the end of each unit in most syllabus documents for specific information. (Some exceptions) Is the timeframe appropriate for assessment event/s Is there enough time to realistically complete the task to the required workplace standard? Or is there too much time? Are a variety and appropriate assessment strategies used Are a range of assessment strategies being used? Do assessment tools provide more than one opportunity and more than one way of gathering evidence for the unit of competency? Does the task/s collect sufficient evidence for judgment of competence Does the task/s collect sufficient evidence for judgment of competence? Has evidence been collected from more than one source and/or more than one occasion to make a judgment? Can authenticity be verified Are you certain that it is the students own work? Does the task/s ensure evidence collected is current. Competency requires demonstration of current performance, Will the evidence collected prove that a person can competently complete the task to workplace standards today. Currency is important particularly when using work samples or third party evidence. Can the assessment approach be adapted to meet the needs of all candidates (allowable adjustment) Is there flexibility in the task and tools? Can it be adapted for students with a special need, or a different workplace or using different machinery? Does it provide for allowable adjustment? Does the task/s ensure that evidence is collected in a timely and cost effective way It is important to consider if an assessment method, task or tool is appropriate to a particular context. In some cases a valid task and tool might be an effective way of gathering evidence with a small group of students but could not be efficiently applied to a larger group of students. Is the assessment activity suitable for use in other contexts? Can the task and tools be used in other situations? Eg recognition, trainees, workplace Eg recognition, trainees, workplace. This is not essential in a valid task. Assessment Validation Kit Evidence Criteria 16 Appendix: Optional Support Documents Validation of Assessment Tools This checklist can be used when designing assessment tools as well as in a post assessment validation activity. . Pre-Assessment Activity Please tick Post-Assessment Activity Course: Training Package: Qualification Title: Qualification Code: Training Package: Assessment Task: School: Year: Yr 11 Yr 12 Student Name: Teacher/Assessor: Evidence Gathering Tools: Folio Work Oral Questioning Role Play Practical Assessment Observation Oral Presentation Scenarios Diary Journal Work Placement Written Test Other (please specify) Case Study Unit Code: Elements of Competency to be Assessed: Performance Criteria: Clearly describe in the action column adjustments that are required to the tools. Yes/No Action to be taken Clearly identify the candidate, the assessor and the date Yes No Clearly identify unit(s) of competence being assessed Yes No Clearly identify the training package Yes No Clearly identify national codes Yes No Are user friendly Yes No Use ‘Plain English’ Yes No Use language appropriate for the intended audience Yes No Assessment Validation Kit Do the tools? 17 Appendix: Optional Support Documents Include instructions on how the tool is used Yes No Reflect the assessment environment Yes No Reflect the AQF level requirements Yes No Contain activities that are holistic Yes No Ensure assessment of the relevant competence(s) Yes No Contribute to validity and reliability of assessment Yes No Clearly identify skills, knowledge and understanding relating to competence(s) being assessed Yes No Ensure evidence collected is current Yes No Ensure evidence collected is authentic Yes No Ensure evidence collected is sufficient Yes No Provide guidance on reasonable adjustment(where relevant) Yes No Use methods that can be adjusted to suit special needs Yes No Ensure evidence is collected in a timely and cost effective way Yes No Include assessment of the dimensions of competence Yes No Contain version control Yes No Assessment tools are ready for use: Yes No Date: Validated and authorised by: Print School Name: Signature: Date: Assessment Validation Kit Print Name: 18 Appendix: Optional Support Documents Assessment Schedule Validation Checklist Use this checklist to ensure that participant validation schedules/plans are valid. Course: Qualification Code: Qualification Title: Training Package: School: Yr 11 Year: Evidence Criteria Action Yes/ No Does the assessment schedule? Yr 12 Clearly describe the adjustment needed Address the units of competency being assessed during the Preliminary or HSC course Yes No Supports the holistic assessment of units of competency Yes No Have written information worded appropriately Yes No Make clear how and when assessment will take place Yes No Indicate timelines and assessment activities Yes No Allow for practical evidence gathering activities Yes No Provide opportunity to gather sufficient evidence over a period of time Yes No Show version control Yes No Allow for cost effective assessments Yes No Allow for fair, non discriminatory and inclusive assessment Yes No If NO modification required (as identified under actions): Date: Changes made to assessment schedule: Yes No Date: Assessment Schedule is ready for use: Yes No Date: Names of Validators (print name, school & sign) Print School Name: Signature: Date: Assessment Validation Kit Print Name: 19