MANAGE AN ASSESSMENT SYSTEM FOR TRAINING OUTCOMES D1.HRD.CL9.08 D1.HHR.CL8.06 Slide 1 Element 1: Communicate the assessment system Performance criteria are: Define the purpose of the assessment system Document and circulate assessment system features and procedures to relevant stakeholders Establish procedures for keeping relevant stakeholders informed about key features of the assessment system Slide 2 Define the purpose of the assessment system Some purposes for developing an assessment system may include: Recognising or determining the current competencies or skills of staff Identifying training needs of staff Checking that workplace skill levels are being maintained Issuing internal certificates of achievement or awards for excellence Recognising progress in learning or achievement of desired learning outcomes Identifying areas for improvement or change Maintaining current skill levels of staff Slide 3 Document and circulate assessment system features and procedures to relevant stakeholders Does Shows How Performance/Action Competence Knows How Knows Slide 4 Document and circulate assessment system features and procedures to relevant stakeholders (cont’d) There are many different parts to an assessment system and they may include: An ability to determine the required competency levels and qualifications of assessors Policies and procedures for obtaining, updating and storing records and information of candidates and assessors How to conduct assessment including the timing and duration specifics Flexibility in order to respond to individual needs of participants who may have particular challenges such as physical or language and literacy needs Slide 5 Document and circulate assessment system features and procedures to relevant stakeholders (cont’d) There are many different parts to an assessment system and they may include: A review and evaluation process to determine if the assessment system is effective or needs improvement Policies and procedures that allow for feedback including, grievances, moderation, re-assessment A statement relating to competency levels and remuneration levels appropriate to those skills attained Slide 6 Document and circulate assessment system features and procedures to relevant stakeholders (cont’d) There are many different parts to an assessment system and they may include: Outline country specific occupational health and safety regulations, employment requirements and equal opportunity legislation Procedures for recognising previous educational qualifications or experience gained at other training organisations or employment contexts Information about where assessment materials are located and where assessment can take place Slide 7 Document and circulate assessment system features and procedures to relevant stakeholders (cont’d) Some of the relevant stakeholders may include: Department heads, team leaders, managers and the business owners Participants and candidates from both current learners and past learners Trainers and assessors. External assessors and trainers may also need to be informed of the assessment system especially if there is an external audit process or certification requirements Slide 8 Document and circulate assessment system features and procedures to relevant stakeholders (cont’d) Some of the relevant stakeholders may include: Technical experts who may be required to provide feedback about the relevance and currency of skills and assessment Training and assessment coordinators Government and industry bodies related to the assessment components Slide 9 Establish procedures for keeping relevant stakeholders informed about key features of the assessment system There are several ways to keep stakeholders informed about assessment system features including Scheduling regular meetings that relate directly to assessment issues Special meetings which may be necessary at various times due to urgent issues or feedback Regular staff meetings having a standing agenda item included that allows for feedback and comments regarding assessment issues Slide 10 Establish procedures for keeping relevant stakeholders informed about key features of the assessment system There are several ways to keep stakeholders informed about assessment system features including The organisation of a dedicated newsletter or fact sheet that provides updates on assessment system changes or feedback Developing an internal network of assessors who can communicate directly with each other and discuss particular issues that may be relevant to the assessment system delivery and effectiveness Slide 11 Element 2: Provide support for enterprise assessors Performance criteria for this unit include: Verify assessors meet required competency standards Identify required training for assessors Provide for moderation of assessments Provide feedback to assessors on their performance Facilitate networking amongst assessors Slide 12 Verify assessors meet required competency standards In order for assessors to be able to assess effectively, it is critical that they have the necessary skills and experience required to assess participants. If the assessors have limited experience in the assessment item, then the assessment may not be a true reflection of the participant’s ability in that skill. It is important then to check all assessors experience and qualifications to ensure that they have the necessary skills to be able to conduct the assessment properly Slide 13 Verify assessors meet required competency standards cont’d The rules of evidence: Valid Sufficient Currency Authentic Slide 14 Identify required training for assessors After checking qualifications and experience of assessors, it may be necessary to also observe and interview them. If the assessors do not meet the required competency standards, then it may necessary to train them to fill any gaps in their skills. Slide 15 Identify required training for assessors Checking competency: Skill Knowledge Attitude Slide 16 Provide for moderation of assessments Moderation is a process where you can compare your own assessment activities with that of other people and evaluate your methods and procedures. It is concerned in particular with bringing assessment judgements and standards into alignment, thus ensuring the same standards are applied to all assessment results within the same unit of competency. Slide 17 Provide feedback to assessors on their performance The competency standard being assessed Critical evidence required Evidence collected Does it relate to the standards being assessed? (Valid) Is there enough of it? (Sufficient) Is it up Is it the to date? candidates own (Current) work? (Authentic) Slide 18 Provide feedback to assessors on their performance cont’d Analysing their assessments, including results and materials/tools as part of the assessment process Developing feedback instruments for candidates to complete following assessments Providing observation sessions while assessors conduct assessments Slide 19 Provide feedback to assessors on their performance cont’d Identifying specific foci for evaluation and feedback Encouraging self-evaluation as part of the feedback process Recognising effort and positive outcomes Supporting assessors where feedback identifies negative outcomes Slide 20 Facilitate networking amongst assessors Networks allow assessors to: Learn, discuss and share information about assessment issues Promote the best standards for assessment Explore ways to improve the assessment system Learn from external assessment agencies Keep informed about new changes or standards Keep engaged with their assessment responsibilities Slide 21 Facilitate networking amongst assessors (cont’d) Networks can be developed in many different ways and may include: Formal associations Informal groups Internal organisations within a company External agencies linking groups of assessors Meetings both regular and as required Slide 22 Facilitate networking amongst assessors (cont’d) Networks can be developed in many different ways and may include: Seminars and workshops Conferences Notice boards, handouts, newsletters Electronically through websites, emails, blogs Slide 23 Element 3: Manage the assessment record keeping system Performance criteria for this element include: Identify and develop the internal records required to support the assessment system Describe the requirements for completing the assessment records File the assessment records Review and up-date the assessment record keeping system Slide 24 Identify and develop the internal records required to support the assessment system What information needs to be recorded? Slide 25 Identify and develop the internal records required to support the assessment system Internal records could include: Personal information relating to candidates Evidence that required training has been delivered prior to assessment being attempted Assessment tools and assessment items Copies of relevant training curricula/training plans Training rolls Evidence captured as part of the assessment process Documentation used to record the resources used and costs involved as part of the assessment process Slide 26 Describe the requirements for completing the assessment records Describing requirements for completing records may include: Demonstration where an assessor trainer or supervisor completes a form with the assessor Written guidelines that are distributed to all assessors Providing examples or templates that show how to complete the records Online tutorials with examples FAQ sheets Monitoring the compilation of records completed by assessors on an ongoing basis where feedback can be provided if required Slide 27 File the assessment records Who needs access to the assessment records Assessors Managers External agencies Training providers and trainers Candidates or staff Administrators Assessment coordinators Slide 28 File the assessment records (cont’d) Why do they need access to the records?: To update assessment records To evaluate assessment procedures and policies To provide feedback to candidates, staff, managers, external agencies, training providers and trainers To check learning and qualification progress To check work performance or promotion potential To identify gaps in training or skills Slide 29 Review and up-date the assessment record keeping system A review mechanism should include: Dates and time of review Items for review Data/information/feedback gathered Issues or matters for attention Who conducted the review Recommendations for improvement Date for next review A mechanism for communicating updates or changes to the assessment record keeping system to relevant stakeholders Slide 30 Review and up-date the assessment record keeping system Improved Assessment System Conduct Consultations with Stakeholders Ongoing Evaluation Review Cycle Implement Improvements Gather Evidence Develop Implementation Strategy Slide 31 Element 4: Maintain quality assurance procedures Performance criteria for the element include: Define quality assurance procedures for the assessment system Undertake internal audits of the assessment system Use the results of quality assurance audits to improve the assessment system Review the applicability of the existing quality assurance procedures to the dynamic nature of the assessment system Slide 32 Define quality assurance procedures for the assessment system Validation is a quality review process. It involves checking that the assessment tool produces valid, reliable, sufficient, current and authentic evidence to enable reasonable judgments to be made as to whether the requirements of the relevant aspects of the training package or accredited course have been met. It includes reviewing and making recommendations for future improvements to the assessment tool, process and/or outcomes. Slide 33 Define quality assurance procedures for the assessment system (cont’d) There are many reasons why organisations need to review including: Confirming the credibility and recognition of certification and awards Supporting the industry and community recognition of training participants Providing the best service for stakeholders - learners and industry Ensuring assessments reflect changes in current industry requirements Improving the validity, reliability, flexibility and fairness of assessments Slide 34 Define quality assurance procedures for the assessment system (cont’d) There are many reasons why organisations need to review including: Providing greater justice for participants for assessment with improved relevance, transparency and support in assessment processes Ensuring more meaningful feedback and guidance for students after they have been assessed Supporting ongoing organisational quality assurance Enhancing the reputation and recognition of the organisation and its services Slide 35 Define quality assurance procedures for the assessment system (cont’d) There are many reasons why organisations need to review including: Minimising risks associated with the assessment process Improving the management of assessment systems Informing assessor selection and guiding professional development Slide 36 Define quality assurance procedures for the assessment system (cont’d) There are certain areas that need to be outlined in implementing quality assurance procedures. Systematically collecting and analysing data Ensuring that data is relevant and sufficient Data sources relevant to improving training and assessment Demonstrating improvements Slide 37 Undertake internal audits of the assessment system In developing quality assurance procedures for the assessment system there are five targets for validation that can be identified which should be included in an internal audit. They include: Assessment systems Assessment processes Assessment methods and tools Assessment evidence Assessment judgments Slide 38 Undertake internal audits of the assessment system cont’d Section for audit Specific area for audit Standards Result and recommendations Risk Assessment systems Participant personal details completed Administrative procedures Recording assessment results Assessment records for each participant completed and up to date. Compliant: All samples checked were completed correctly. Maintain regular monthly checks. Low Non-compliant: All but one sample checked was up to date and fully completed. Participant 1452 sample was missing assessor name. This should be completed and regular checks continued. Medium Slide 39 Use the results of quality assurance audits to improve the assessment system There are three fundamental processes to identify opportunities for continuous improvement. These are: Reviewing Moderating Evaluating Slide 40 Review the applicability of the existing quality assurance procedures to the dynamic nature of the assessment system It is important to ensure that quality assurance procedures are responsive to the dynamic nature of the assessment system. It is necessary to check: That all aspects of the assessment system are being checked and evaluated regularly The quality assurance procedures are flexible enough to still be able to review the system even if change occurs Slide 41 Element 5: Report on the contribution made by the assessment system to enterprise training and performance Performance criteria for this element include: Identify the outcomes that the assessment system has produced Review the assessment system Make recommendations for change to the assessment system Produce and circulate a report Revise the existing assessment system on the basis of responses to the report Slide 42 Identify the outcomes that the assessment system has produced It is important to analyse the results or outcomes and identify: Whether the assessment system has been effective in achieving the desired outcomes of the system as a whole The benefits that the assessment system has brought to the organisation by noting the new skills attained, the amount of certificates issued, the increased opportunities and performance that the system has provided to the enterprise Slide 43 Identify the outcomes that the assessment system has produced It is important to analyse the results or outcomes and identify: Increased professional development opportunities that have occurred due to the implementation of the assessment system Whether there were some areas of strength within the organisation that had not previously been acknowledged Increased staff engagement, productivity or workplace satisfaction Slide 44 Review the assessment system The review should analyse all areas of the system including: Assessment systems Assessment processes Assessment methods and tools Assessment evidence Assessment judgments Slide 45 Make recommendations for change to the assessment system Recommendations for change could include: Whether to continue with the existing assessment system with minor alterations or to cease with the current system in favour of an external provider administering an assessment system Making recommendations relating to staff resources, training, payment, costs, assessment content or administration procedures Expanding the existing system or reducing the scope of the existing assessment system Recommending changes to location and timing of the assessment system Slide 46 Produce and circulate a report A report should include: Introduction Main body Appendices, references and glossary Slide 47 Produce and circulate a report (cont’d) It is important to identify all of the stakeholders they may need to receive the report and these could be: Assessors Managers External agencies Training providers and trainers Candidates or staff Administrators Assessment coordinators Slide 48 Revise the existing assessment system on the basis of responses to the report Revisions to the assessment system could include: Revising policies and procedures relating to the assessment system Altering the assessment tools to make them more relevant and appropriate Revising the assessors that have been used which may mean increasing or decreasing numbers, choosing assessors with different skill sets and experience, or providing more training for current assessors Slide 49 Revise the existing assessment system on the basis of responses to the report (cont’d) Revisions to the assessment system could include: Revising the assessment times and locations to suit organisational needs Communicating any changes that have been made to the assessment system and making sure that all relevant stakeholders are informed of these changes or updates Slide 50