Assistant Test User: Educational proforma v0.1 May 2011 EDUCATIONAL TEST USER STANDARDS GUIDANCE FOR ASSESSORS FOR THE QUALIFICATION – ASSISTANT TEST USER: EDUCATIONAL Introduction This document contains the module sets and individual modules for the British Psychological Society’s Assistant Test User: Educational qualification in psychological testing. It should be used in conjunction with the Assessors’ Handbook by Chartered Psychologists applying to the Society to become a Verified Assessor for the Assistant Test User: Educational qualification in psychological testing. Separate proformas are available for each of the qualifications offered by the Society, and can be downloaded from the Psychological Testing Centre’s website at www.psychtesting.org.uk. 1|Page Assistant Test User: Educational proforma v0.1 May 2011 How to use this proforma Assessors should this use proforma to help them develop their assessment materials and as part of their submission of materials for verification purposes. They should also complete their details in the spaces below: Assessor’s details Name: Click here to enter text. Contact number: Click here to enter text. Company/organisation: Click here to enter text. Email: Click here to enter text. For each module in the Assistant Test User: Educational qualification in psychological testing, a description is given which provides an overview of the module contents and the most appropriate strategies for assessment. This is followed by descriptions of the competencies that test users must demonstrate in order to be affirmed as competent on the module. Alongside each competency there is detailed guidance for Assessors. This guidance is a development of the previous guidance for Assessors at Level A and Level B, and has had extensive input from Verifiers and members of the Psychological Testing Centre and Committee on Test Standards. As such, it draws on almost 20 years experience of assessing test users for the Society’s qualifications whilst also benefitting from an extensive update and review to reflect recent developments and current practice in psychometric testing. Alongside the guidance for assessors is a column headed ‘reference’. For each of the competencies, Assessors must provide a reference to where in their assessment materials each specific competency is assessed. When requested by your Verifiers, this completed proforma should be sent to them along with your assessment materials and model answers. Further details of the verification process are given in the Assessors’ Handbook. Details of the modules in the Assistant Test User: Educational qualification in psychological testing The table below outlines the module sets and individual modules in which test users must demonstrate competence for the award of the Assistant Test User: Educational qualification in psychological testing. Modules are grouped into ‘module sets’ for the purpose of registration and pricing of the qualifications. In practice this means that test users cannot register separate modules but only module sets, though in some cases a module set may only contain one module. The columns in the table below are as follows: Ref#: Unique module number 2|Page Assistant Test User: Educational proforma v0.1 May 2011 Title: Module name Category: Psychological knowledge; Psychometrics; or Practitioner skill Specificity: Whether the module is context-related and therefore would need to be evidenced separately for multiple domains or instruments. o Generic: The module is only required once for a qualification, regardless of domain o Domain Specific: The module would have to be re-assessed for different domain-related qualifications (e.g. Educational / Occupational) o Instrument specific: The module would have to be re-assessed for different instruments or instrument categories within domains. Assistant Test User: Educational Test Administration Prior registration requirements: None Overview of role: Assistant Test Users: Are able to administer and score specific tests under the supervision of a person qualified at Test User level or above. Have awareness of broader issues related to testing and test use, of limitations and value of using tests, and know when to seek more expert help. Are not able to make choices about which tests should be used or provide interpretations of test scores beyond those provided in standard reports. An Assistant Test User will be an individual who uses specific tests in well-defined and constrained contexts, such as testing groups of children and / or individuals to understand their strengths and specific learning needs. Choice of tests and details of how they are to be used and applied is outside the An Assistant Test User’s responsibility. Approximate European Qualification Framework (EQF) Level: 4 3|Page Assistant Test User: Educational proforma v0.1 May 2011 Ref# Title Category Specificity Module Set: 4B 102 Introduction to Testing: Educational Psychological Knowledge Domain specific 103 Administering tests to one or more candidates, part 1 Practitioner Skill Domain specific 105 Administering tests to one or more candidates, part 2: Educational Practitioner Skill Domain specific 106 Maintaining security and confidentiality of the test materials and the test data Practitioner Skill Domain specific The following tables show the modules and associated competencies for the Assistant Test User: Educational qualification in psychological testing. As part of their submission to the Society for verification, Assessors should complete the ‘Assessor’s reference’ column, identifying where in their assessment materials each competency is assessed. The following information is shown in each table: Column 1 is competency reference Column 2 contains the CCET reference Column 3 contains the text from the revised Level A/B standards (2005). Column 4 contains the guidance for Assessors Column 5 gives space for Assessors to enter a reference to where the competency is covered in their assessment materials NOTE: The ordering of the modules has no particular significance. It is not related to either importance or the order in which assessment might be carried out. 4|Page Assistant Test User: Educational proforma v0.1 May 2011 ASSISTANT TEST USER PSYCHOLOGICAL KNOWLEDGE Ref CCET Module 4.102. Introduction to testing: Educational Guidance: Educational Reference Overview of assessment requirements: Assessors should have evidence that test users understand the difference between tests of ability, aptitude and attainment and between formative and summative assessment. Test users should also be able to demonstrate (giving specific examples) that testing is a subset of the overall process of assessment and that testing may not always be the most appropriate means of obtaining relevant data. Test users should appreciate the impact of technology on modes of test administration and be aware of the benefits and issues associated with each of these. 102.1 1.6 102.2 5|Page The test user can: Distinguish between tests of ability, aptitude and attainment. Distinguish between open, controlled, supervised and managed modes of test administration Can describe in general terms the distinction between tests which are intended to estimate an underlying cognitive function and hence predict what someone might be able to learn or do in the future and those which measure their current level of knowledge understanding or skill, and can provide examples of the appropriate use of each. As defined by the ITC guidelines on CBT and the Internet, open mode is free access (e.g. on the web); controlled mode requires the test taker to be identified (e.g. via unique login and password) often referred to as remote administration; supervised requires the presence of a test administrator; managed involves the use of a secure test centre with appropriately trained staff. In educational settings in the classroom teachers might be in the room but unable to attend to each pupil individually, so leaving scope for unauthorised help being used. It is important to ensure test users recognise the extent to which pupils’ behaviour is under direct observation/control during test sessions. Click here to enter text. Click here to enter text. Guidance for Assessors Master Document v0.1 102.3 1.1 Demonstrate an understanding of the purpose of testing within the overall context of assessment. 102.4 1.7 Explain the difference between formative and summative uses of tests, giving examples of each type. 6|Page Demonstrate an understanding of how assessment is undertaken in broad terms, that it involves a process and is not simply ’client centred’. Also show understanding that psychometric testing is only a possible part of this process and show appreciation for its usefulness and limitations. Demonstrate an understanding of how test results can be used to inform future intervention and to indicate attainment to date. Show understanding of how test results can be used for both purposes in relevant circumstances. Click here to enter text. Click here to enter text. Guidance for Assessors Master Document v0.1 ASSISTANT TEST USER PRACTITIONER SKILLS Ref CCET Module 4.103. Administering tests to one or more candidates, part 1 Guidance: Educational Reference Overview of assessment requirements: This module should be assessed through the practical administration of tests in a realistic context. When conducted outside of the workplace Assessors should establish realistic situations so that test users can demonstrate the ability to prepare the test taker, organise the test setting and materials, and check and take account of any special considerations that might affect the test taker’s performance. Test users should also demonstrate the practical skills of test scoring, converting raw scores into standardised scores and making appropriate checks to ensure the accuracy of these. 103.1 103.2 7|Page Generic The test user can: Ensure adequate preparation by studying the relevant sections of the test user manual and other relevant documents and becoming familiar with all relevant materials and procedures in advance of the session. Ensure that all candidates receive appropriate briefing and preparation materials a suitable time prior to the testing session. The requirement here is for the test user to show in discussion, writing and practice that they have read and understand the administration requirements and instructions of specific tests. Click here to enter text. Demonstrate by example appropriate briefing and preparation information for relevant tests and contexts. Test users should demonstrate awareness of the need to provide sufficient information for candidates with specific disabilities and to decide whether they may need an accommodation. Teachers and EPs generally take on the responsibility for deciding on accommodations for disability, not the pupils. Advance information is only appropriate for some educational settings. One needs to refer to the age and ability of the pupils. For individual testing, parental permission is sought to test pupils but it is generally up to schools as to whether or not to inform parents of group testing. Young children are not Click here to enter text. Guidance for Assessors Master Document v0.1 normally told in advance but older ones (secondary) are. When administering supervised or managed mode assessments: 103.3 5,2; 5.3, 5.21 - ensure that the location is suitable for the duration and type of assessment, equipment required and numbers of candidates 103.4 5.1 - ensure all necessary materials are available in sufficient quantities and that materials are in good condition and equipment in working order 103.5 103.6 8|Page - ensure that any necessary test accommodations for disabled candidates are available, well understood and appropriately provided 5.4 - brief candidates on the purpose of the test session Supervised and managed modes and those where a test administrator is present during the test session. Essentially they require the Assistant Test User to ensure that all the practical administrative requirements of the test are followed to ensure that the results when obtained are both reliable and valid. Demonstrate the ability to arrange seating and desk space appropriately for the test whether for individuals or groups. Inform the candidates of the time and place well in advance and ensure they are adequately prepared – where relevant – for what they will be required to do and why. Ensure that those doing the test are not unduly disturbed or distracted and that the ambient qualities of the room are appropriate and suitable. Ensure, where re-usable materials are being used, that they are carefully checked for marks or notes which may have been made by previous candidates. Ensure that any equipment (e.g. computers) is operating correctly and sufficient test materials are available for use by the candidate. Where appropriate, candidates should be given the opportunity to familiarise themselves with response mechanisms or other special test equipment. Implement requirements as arranged by the holder of a Test User level qualification or other suitably responsible and qualified test user supervising the session. Click here to enter text. Apply appropriate rules which might include some or all of following: not talking to or colluding with others during the Click here to enter text. Click here to enter text. Click here to enter text. Click here to enter text. Guidance for Assessors Master Document v0.1 and key rules and put them at their ease while maintaining an appropriate atmosphere session, not having any recording devices, including mobile phones - which should be turned off and left with the test administrator. Heed any advice provided by the test manual on the content & wording of the briefing. Provide the candidates with sufficient time to work through example test items. Make careful checks to ensure proper use of the answer sheet and response procedures. Explain any time limits and ensure that during the test candidates maintain silence to avoid distracting others. Make clear that once the test or timed subtest has begun no further questions can be answered. Adhere strictly to test-specific instructions concerning pacing and timing. Deal appropriately with any questions which arise without compromising the purpose of the test. Ensure that any scoring required during the testing process is accurate whilst not in any way putting the client off, for example by showing high level of failure Click here to enter text. Ensure that appropriate invigilation of the test session is maintained to identify candidates who are confused, not following test instructions, etc, and avoid issues such as cheating and recording information about the test. Observe and record any notable issues to help interpret data when it comes to report writing or feedback. Ensure that the candidate’s personal details have been recorded, together with relevant details of what assessment instruments were used, what accommodations were made for any special needs and whether accommodations met their needs. Record any other relevant information, such as any particular problems which arose during the session which might have affected a candidate’s performance. Click here to enter text. 103.7 5.9, 5.10 - follow standard test procedures and instructions as specified in the user manual 103.8 5.26 - 103.9 6.8, 5.23 103.10 5.5 9|Page use appropriately discreet scoring procedures so that the test taker does not become unduly conscious of failure - monitor behaviour during the test to identify any issues or problems that may arise - ensure that all necessary information has been logged and recorded including details of any provisions that were made for any special needs Click here to enter text. Click here to enter text. Guidance for Assessors Master Document v0.1 103.11 5.12 - check and collect all relevant materials at the end of the test session, and ensure they are stored securely. Collect all materials when each test is completed, check that all materials have been recovered and lock test materials away in a secure place. For paper-and-pencil tests this includes test questions and answer sheets. For all tests this should include any notes or records made by the candidate during the test session. Visually check paper answer sheets for ambiguous markings which could be obscured by scoring keys or cause problems with machine scoring systems. Click here to enter text. 103.12 5.18 Demonstrate accurate use of different hand-scoring keys and ‘self-scoring’ forms for paper-and pencil-testing. Demonstrate the accurate application of commonly used forms keys. Click here to enter text. 103.13 2.7, 2.8 Convert raw scores into percentile equivalents and various standard scores. Click here to enter text. 103.14 5.175.20 Make appropriate records of candidates’ raw scores and score conversions or check the accuracy of these records where this is process is automated. Demonstrate the ability to accurately convert raw scores using norm tables. Standardised scores should include those based on percentiles and z-score conversions (T-scores, Stens etc). Score conversions may be checkable but other aspects may not always be able to be checked. Examples should ensure the test user can spot obvious errors in scoring. 10 | P a g e Click here to enter text. Guidance for Assessors Master Document v0.1 ASSISTANT TEST USER PRACTITIONER SKILLS Ref CCET Module 4.105. Administering tests to one or more candidates, part 2: Educational Guidance: Educational Reference Overview of assessment requirements: Evidence for this module should be collected by allowing test users to demonstrate skills in use of individual tests in an educational setting. Test users should be able to demonstrate the ability to build rapport with the candidate and administer the test according to the instructions laid down in the manual. In the case of individual testing the test user can: Establish a satisfactory rapport with the test taker prior to the commencement of testing and maintain this throughout the session. 105.1 5.24 105.2 5.25 Prompt the test taker where appropriate in accordance with the test instructions and without invalidating the test item. 105.3 5.26 Use appropriately discreet scoring procedures so that the test taker does not become unduly conscious of failure. 11 | P a g e The test user should demonstrate that he or she can establish and maintain a suitable relationship with the candidate such that the candidate performs to an optimum level of performance but in so doing so the test user must ensure that standard instructions are used and that the test remains valid and reliable The test user should ensure that where it is appropriate the candidate’s questions about the test are answered in such a way as to ensure the test is not invalidated in any way. Where doubt exists the test user should normally explain that a question cannot be answered because to do so would invalidate the test. Test user to score the test accurately but ensure that the candidate does not have inappropriate sight of the scoring and is not unduly discouraged by any wrong answers Click here to enter text. Click here to enter text. Click here to enter text. Guidance for Assessors Master Document v0.1 105.4 5.27 Follow discontinuation procedures in line with the guidelines in the test manual while ensuring that the test taker does not become disheartened through obvious failure. Demonstrate compliance with test instructions regarding discontinuation. If judgment is such that a test has to be curtailed before discontinuation point is reached the test user must have and demonstrate a suitable rationale for this and make appropriate reference and attach suitable caution to results in feedback. Click here to enter text. ASSISTANT TEST USER PRACTITIONER SKILLS Ref CCET Module 4.106. Maintaining security and confidentiality of the test materials and the test data Guidance: Educational Reference Overview of assessment requirements: Assessors should ensure test users demonstrate a clear understanding of the ethics of testing and that they will act in an ethical manner. Test users should be able to demonstrate their ability to provide candidates and other stakeholders where appropriate with a clear indication of how test results will be used and show how they will make suitable arrangements for the storage of test materials and data. 106.1 7.1 106.2 7.3, 7.5 12 | P a g e Does the test user: Ensure that best practice is followed in giving clear descriptions to the candidate(s), or relevant others, prior to the assessment concerning: how the results are to be used; who will be given access to them; for how long they will be retained. Ensure that all test data are stored in accordance with current legislation, in a secure place and access is not given to unauthorised personnel. Ensure that candidates receive clear and appropriate information about use and storage of their test results and access to them, so that informed consent is obtained – either in writing or by clicking consent on computer administration. In some cases consent may be required from relevant others, such as parents or guardians Ensure understanding of relevant aspects of Privacy, Digital Rights Management, EOC Legislation in the country of use and candidate's rights. Test Users must know how and where data is stored, if in a digital environment. If Test Users delegate some part of the process to someone else, they must be competent to exercise responsibility. Click here to enter text. Click here to enter text. Guidance for Assessors Master Document v0.1 106.3 7.4 Ensure that all test materials are kept in a secure place which is not accessible to people other than authorised test users. 106.4 7.6 Ensure that potential test candidates are not provided with prior access to test materials other than those specifically designed to help candidates prepare for their assessment. Ensure appropriate measures and processes are in place so that all paper-based test materials are stored securely and access to computer-based tests is restricted to authorised test users. Ensure that where available, candidates are given access to appropriate preparation materials but that security policies and procedures prevent candidates accessing actual test materials prior to testing. Click here to enter text. Click here to enter text. The British Psychological Society’s Psychological Testing Centre, St Andrews House, 48 Princess Road East, Leicester, LE1 7DR Tel: 0116 252 9530 Fax: 0116 227 1314 Email: enquiry@psychtesting.org.uk Web: www.psychtesting.org.uk Incorporated by Royal Charter. Registered Charity No 229642 13 | P a g e