Assistant Test User Occupational – Guidance for Assessor Form – April 2015 OCCUPATIONAL TEST USER STANDARDS GUIDANCE FOR ASSESSORS FOR THE QUALIFICATION: ASSISTANT TEST USER: OCCUPATIONAL Introduction This document contains the module sets and individual modules for the British Psychological Society’s (BPS) Assistant Test User: Occupational qualification in psychological testing. It should be used in conjunction with the Assessors’ Handbook by Chartered Psychologists applying to the BPS to become a Verified Assessor for the Assistant Test User: Occupational qualification in psychological testing. Separate forms are available for each of the qualifications offered by the BPS, and can be downloaded from the Psychological Testing Centre’s website at www.psychtesting.org.uk. How to use this form Assessors should this use form to help them develop their assessment materials and as part of their submission of materials for verification purposes. They should also complete their details in the spaces below: 1 Assistant Test User Occupational – Guidance for Assessor Form – April 2015 Assessor’s details Name: Click here to enter text. Company/organisation: Click here to enter text. For each module in the Assistant Test User: Occupational qualification in psychological testing, a description is given which provides an overview of the module contents and the most appropriate strategies for assessment. This is followed by descriptions of the competencies that test users must demonstrate in order to be affirmed as competent on the module. Alongside each competency there is detailed guidance for Assessors. This guidance is a development of the previous guidance for Assessors at Level A and Level B, and has had extensive input from Verifiers and members of the Psychological Testing Centre and Committee on Test Standards. As such, it draws on almost 20 years experience of assessing test users for the BPS’s qualifications whilst also benefitting from an extensive update and review to reflect recent developments and current practice in psychometric testing. Alongside the guidance for assessors is a column headed ‘Reference’. For each of the competencies, Assessors must provide a reference to where in their assessment materials each specific competency is assessed. When requested by your Verifiers, this completed form should be sent to them along with your assessment materials and model answers. Further details of the verification process are given in the Assessors’ Handbook. Details of the modules in the Assistant Test User: Occupational qualification in psychological testing The table below outlines the module sets and individual modules in which test users must demonstrate competence for the award of the Assistant Test User: Occupational qualification in psychological testing. Modules are grouped into ‘module sets’ for the purpose of registration and pricing of the qualifications. In practice this means that test users cannot register separate modules but only module sets, though in some cases a module set may only contain one module. The columns in the table below are as follows: Ref#: Unique module number Title: Module name Category: Psychological knowledge; Psychometrics; or Practitioner skill Specificity: Whether the module is context-related and therefore would need to be evidenced separately for multiple domains or instruments. 2 Assistant Test User Occupational – Guidance for Assessor Form – April 2015 o o o Generic: The module is only required once for a qualification, regardless of domain Domain Specific: The module would have to be re-assessed for different domain-related qualifications (e.g. Educational / Occupational) Instrument specific: The module would have to be re-assessed for different instruments or instrument categories within domains. Assistant Test User: Occupational Prior registration requirements: None Overview of role: Assistant Test Users: Are able to administer and score specific tests under the supervision of a person qualified at Test User level or above. Have awareness of broader issues related to testing and test use, of limitations and value of using tests, and know when to seek more expert help. Are not able to make choices about which tests should be used or provide interpretations of test scores beyond those provided in standard reports. An Assistant Test User will be an individual who uses specific tests in well-defined and constrained contexts, such as routine recruitment and selection procedures or standardised assessment for staff development programmes and who operates within organisational policies and directives on testing and test use. Choice of tests and details of how they are to be used and applied is outside the Assistant Test User’s responsibility. Approximate European Qualification Framework (EQF) Level: 4 Ref# Title Category Specificity Psychological Knowledge Domain specific Module Set: 4A 101 Introduction to Testing: Occupational 3 Assistant Test User Occupational – Guidance for Assessor Form – April 2015 103 Administering tests to one or more candidates, part 1 Practitioner Skill Domain specific 104 Administering tests to one or more candidates, part 2: Occupational Practitioner Skill Domain specific 106 Maintaining security and confidentiality of the test materials and the test data Practitioner Skill Domain specific The following tables show the modules and associated competencies for the Assistant Test User: Occupational qualification in psychological testing. As part of their submission to the BPS for verification, Assessors should complete the ‘Assessor’s reference’ column, identifying where in their assessment materials each competency is assessed. The following information is shown in each table: Column 1 is competency reference Column 2 contains the original Level A/B unit and element references (i.e. pre 2000) Column 3 contains the Level A/B unit and element references from the 2005 revised level A/B standards Column 4 contains the text from the revised Level A/B standards (2005) Column 5 contains the guidance for Assessors Column 6 gives space for Assessors to enter a reference to where the competency is covered in their assessment materials NOTE: The ordering of the modules has no particular significance. It is not related to either importance or the order in which assessment might be carried out. 4 Assistant Test User Occupational – Guidance for Assessor Form – April 2015 ASSISTANT TEST USER PSYCHOLOGICAL KNOWLEDGE Ref LA/B LA/B rev Module 4.101. Introduction to Testing: Occupational Guidance: Occupational Reference Overview of assessment requirements: Assessors should have evidence that test users show knowledge of the difference between the main types of tests (maximum and typical performance) and related assessments used in occupational assessment, and are able to distinguish between the different types of maximum performance test. Test users should appreciate the impact of technology on modes of test administration and be able to describe the benefits and issues associated with each of these. The test user can: 101.1 101.2 A1.2 A1.3 Distinguish between tests of ability, aptitude and attainment. A10.1 Distinguish between open, controlled, supervised and managed Describe in general terms the distinction between tests which describe what someone is currently capable of doing, those that are intended to predict what someone will be able to learn or do in the future and those which measure their current level of knowledge understanding or skill. Test users should provide examples of the appropriate use of each type of test. Explain the four modes of testing as defined by the ITC guidelines on CBT and the Internet: open 5 Verifier’s Notes (Assessors, please leave this column blank) Methods of Assessment (Assessors please indicate your method of assessment and where this is evidenced in your portfolio, e.g. Report 1, p.34, para 3 -6) Click here to enter text Click here to enter text Click here to enter text Click here to enter text Assistant Test User Occupational – Guidance for Assessor Form – April 2015 modes of test administration. 101.3 A10.2 Describe the main issues associated with administering different types of tests in each mode. 101.4 A10.4.4 Describe the potential problems of collusion in remote assessment (i.e. in open or controlled mode). A1.2 Describe the distinction between tests assessing ‘maximum’ performance and those assessing ‘typical’ performance. 101.5 A1.1 mode is free access (e.g. on the web); controlled mode requires the test taker to be identified (e.g. via unique login and password; often referred to as remote administration); supervised mode requires the presence of a test administrator; managed mode involves the use of a secure test centre with appropriately trained staff. Describe and evaluate the potential benefits and problems with the different modes of administration, including issues of cost, accessibility, candidate authentication and collusion. Test users should also be able to differentiate between issues of control associated with ability tests (timing, item security etc) and self-report inventories. Describe methods for checking candidate behaviour remotely (e.g. forensic data analysis) and tactics for follow-up supervised assessment where candidate short-listing is concerned. Distinguish between tests with right-wrong answers (ability, aptitude, attainment measures) and those which ask people to rate or select options (self- and other-report measures like personality, attitude, interests, 360 6 Click here to enter text Click here to enter text Click here to enter text Click here to enter text Click here to enter text Click here to enter text Assistant Test User Occupational – Guidance for Assessor Form – April 2015 degree, climate measures and organisational surveys), identifying the key characteristics of each and naming examples of the main types of test. 101.6 A1.5 A1.7 Describe the range of tests used in work and organizational psychology. Identify and distinguish between the main types of test used in work and organisational assessment settings (e.g. work sample, ability, interests, personality and performance tests). 7 Click here to enter text Click here to enter text Assistant Test User Occupational – Guidance for Assessor Form – April 2015 ASSISTANT TEST USER PRACTITIONER SKILLS Ref LA/B LA/B rev Module 4.103. Administering tests to one or more candidates, part 1 Guidance: Occupational Reference Overview of assessment requirements: This module should be assessed through the practical administration of tests in a realistic context. When conducted outside of the workplace Assessors should establish realistic situations so that test users can demonstrate the ability to prepare the test taker, organise the test setting and materials, and check and take account of any special considerations that might affect the test taker’s performance. Test users should also demonstrate the practical skills of test scoring, converting raw scores into standardised scores and making appropriate checks to ensure the accuracy of these. The test user can: 103.1 New Ensure adequate preparation by studying the relevant sections of the test user manual and other relevant documents and becoming familiar with all relevant materials and procedures in advance of the session. The requirement here is for the test user to show in discussion, writing and practice that they have read and understand the administration requirements and instructions of specific tests. 8 Methods of Assessment (Assessors please indicate your method of assessment and where this is evidenced in your portfolio, e.g. Report 1, p.34, para 3 -6 Click here to enter text Verifier’s Notes (Assessors, please leave this blank) Click here to enter text Assistant Test User Occupational – Guidance for Assessor Form – April 2015 103.2 103.3 A10.3 Ensure that all candidates receive appropriate briefing and preparation materials a suitable time prior to the testing session. A5.1 A10.5 When administering supervised or managed mode assessments: A5.1 A10.5.1 - ensure that the location is suitable for the duration and type of assessment, equipment required and numbers of candidates Demonstrate by example appropriate briefing and preparation information for at least two different tests and contexts. Test users should demonstrate awareness of the need to provide sufficient information for candidates with specific disabilities to decide whether they may need to request an accommodation. Supervised and managed modes and those where a test administrator is present during the test session. Essentially they require the Assistant Test User to ensure that all the practical administrative requirements of the test are followed to ensure that the results when obtained are both reliable and valid. Demonstrate the ability to arrange a suitable venue for the test session, with due regard for the number of candidates and time needed for testing. Arrange seating and desk space appropriately for the test whether for individuals or groups. Inform the 9 Click here to enter text Click here to enter text Click here to enter text Click here to enter text Click here to enter text Click here to enter text Assistant Test User Occupational – Guidance for Assessor Form – April 2015 candidates of the time and place well in advance and ensure they are adequately prepared – where relevant – for what they will be required to do and why. 103.4 103.5 A5.3 A10.5.2 - ensure all necessary materials are available in sufficient quantities and that materials are in good condition and equipment in working order A10.5.3 - ensure that any necessary test accommodations for disabled candidates are available, well understood and appropriately provided Ensure, where re-usable materials are being used, that they are carefully checked for marks or notes which may have been made by previous candidates. Ensure that any equipment (e.g. computers) is operating correctly and sufficient test materials are available for use by the candidate. Where appropriate, candidates should be given the opportunity to familiarise themselves with response mechanisms or other special test equipment. Implement requirements and adjustments as arranged by the holder of a Test User level qualification or other responsible test user supervising the session. 10 Click here to enter text Click here to enter text Click here to enter text Click here to enter text Assistant Test User Occupational – Guidance for Assessor Form – April 2015 103.6 A5.5 A10.5.4 - brief candidates on the purpose of the test session and key rules and put them at their ease while maintaining an appropriate atmosphere 103.7 A5.12 A10.5.6 - follow standard test procedures and instructions as specified in the user manual Deliver an appropriate informal briefing on the purpose of the test session and put candidates at their ease. Include appropriate rules which might be some or all of following: not talking to or colluding with others during the session, not having any recording devices, including mobile phones - which should be turned off. Heed any advice provided by the test manual on the content & wording of the briefing. Provide the candidates with sufficient time to work through example test items. Make careful checks to ensure proper use of the answer sheet and response procedures. Explain any time limits and ensure that during the test candidates maintain silence to avoid distracting others. Make clear that once the test or timed subtest has begun no further questions can be answered. Adhere strictly to test-specific instructions concerning pacing and timing. Deal appropriately with any questions which 11 Click here to enter text Click here to enter text Click here to enter text Click here to enter text Assistant Test User Occupational – Guidance for Assessor Form – April 2015 arise without compromising the purpose of the test. 103.8 - 103.9 A10.5.7 103.10 A5.6 A10.5.8 use appropriately discreet scoring procedures so that the test taker does not become unduly conscious of failure - monitor behaviour during the test to identify any issues or problems that may arise - ensure that all necessary information has been logged and recorded including details of any provisions that were made for any special needs n/a for Assistant Test User: Occupational Click here to enter text Click here to enter text Ensure that appropriate invigilation of the test session is maintained to identify candidates who are confused, not following test instructions, etc, and avoid issues such as cheating and recording information about the test. Observe and record any notable issues to help interpret data when it comes to report writing or feedback. Ensure that the candidate’s personal details have been recorded, together with relevant details of what assessment instruments were used, what accommodations were made for any special needs and whether Click here to enter text Click here to enter text Click here to enter text Click here to enter text 12 Assistant Test User Occupational – Guidance for Assessor Form – April 2015 103.11 A5.15 A10.5.9 103.12 A5.21 A10.6 - check and collect all relevant materials at the end of the test session, and ensure they are stored securely. Demonstrate accurate use of different hand-scoring keys and ‘self-scoring’ forms for paper-and pencil-testing. accommodations met their needs, using an appropriate method such as a test log. Record any other relevant information, such as any particular problems during the session which might have affected a candidate’s performance. Collect all materials when each test is completed, check that all materials have been recovered and lock test materials away in a secure place. For paperand-pencil tests this includes test questions and answer sheets. For all tests this should include any notes or records made by the candidate during the test session. Visually check paper answer sheets for ambiguous markings which could be obscured by scoring keys or cause problems with machine scoring systems. Demonstrate the accurate application of scoring keys or the use of self-scoring answer sheets to accurately produce raw scores for the test. 13 Click here to enter text Click here to enter text Click here to enter text Click here to enter text Assistant Test User Occupational – Guidance for Assessor Form – April 2015 103.13 A2.11 A4.5 Convert raw scores into percentile equivalents and various standard scores. 103.14 Make appropriate records of candidates’ raw scores and score conversions or check the accuracy of these records where this is process is automated. A10.7 Demonstrate the ability to accurately convert raw scores using norm tables. Standardised scores should include those based on percentiles and z-score conversions (T-scores, Stens etc). Score conversions may be checkable but other aspects may not always be able to be checked. Examples should ensure the test user can spot obvious errors in scoring. 14 Click here to enter text Click here to enter text Click here to enter text Click here to enter text Assistant Test User Occupational – Guidance for Assessor Form – April 2015 ASSISTANT TEST USER PRACTITIONER SKILLS Ref LA/B LA/B rev Module 4.104. Administering tests to one or more candidates, part 2: Occupational Guidance: Occupational Reference Overview of assessment requirements: Evidence for this module should be collected by allowing test users to demonstrate skills in appropriately preparing candidates for completing a test under controlled mode, including providing all necessary information in a suitable format to gain ‘buy-in’ and establish an ‘honesty contract’. The test user can: 104.1 104.2 Methods of Assessment (Assessors please indicate your method of assessment and where this is evidenced in your portfolio, e.g. Report 1, p.34, para 3 -6 A10.4 When administering controlled mode Controlled mode is where the candidate is not Click here to assessments: under direct supervision, but they have been enter text provided with a unique (generally one-time) login, or provided with test materials to complete in their own time. A10.4.2 - ensure that candidates are Present candidates with a clear explanation Click here to sent the necessary about the purpose of the assessment, how enter text information and log in details the data will be treated, privacy and in good time confidentiality, and what provisions are available for letting the test user know if internet access is a problem for the candidate. A10.4.3 - ensure that candidates Brief candidate in a way that establishes an Click here to understand what is required appropriate honesty contract with them in enter text 15 Verifier’s Notes (Assessors, please leave this blank) Click here to enter text Click here to enter text Click here to enter text Assistant Test User Occupational – Guidance for Assessor Form – April 2015 of them and what will be happen after completion of the assessment and that they have a means of raising queries. selection situations and explains that scores obtained in unsupervised situations may be checked later in the process. ASSISTANT TEST USER PRACTITIONER SKILLS Ref LA/B LA/B rev Module 4.106. Maintaining security and confidentiality of the test materials and the test data Guidance: Occupational Reference Overview of assessment requirements: Assessors should ensure test users demonstrate a clear understanding of the ethics of testing and that they will act in an ethical manner. Test users should be able to demonstrate their ability to provide candidates and other stakeholders where appropriate with a clear indication of how test results will be used and show how they will make suitable arrangements for the storage of test materials and data. Does the test user: 106.1 A7.1 A13.1 Ensure that best practice is followed in giving clear descriptions to the candidate(s), or relevant others, prior to the assessment concerning: how the results are to be used; who will be given access to them; for how Ensure that candidates receive clear and appropriate information about use and storage of their test results and access to them, so that informed consent is obtained – either in writing, verbally or by clicking consent on computer administration. 16 Methods of Assessment (Assessors please indicate your method of assessment and where this is evidenced in your portfolio, e.g. Report 1, p.34, para 3 -6 Click here to enter text Verifier’s Notes (Assessors, please leave this blank) Click here to enter text Assistant Test User Occupational – Guidance for Assessor Form – April 2015 long they will be retained. 106.2 A7.3 A13.2 Ensure that all test data are stored in accordance with current legislation, in a secure place and access is not given to unauthorised personnel. 106.3 A7.4 A13.3 Ensure that all test materials are kept in a secure place which is not accessible to people other than authorised test users. 106.4 A7.5 A13.5 Ensure that potential test candidates are not provided with prior access to test materials other than those specifically designed to help candidates prepare for their assessment. Ensure understanding of relevant aspects of Privacy, Digital Rights Management, EOC Legislation in the country of use and candidate's rights. Test Users must know how and where data is stored, if in a digital environment. If Test Users delegate some part of the process to someone else, they must be competent to exercise responsibility. Ensure appropriate measures and processes are in place so that all paper-based test materials are stored securely and access to computer-based tests is restricted to authorised test users. Ensure that where available, candidates are given access to appropriate preparation materials but that security policies and procedures prevent candidates accessing actual test materials prior to testing. Click here to enter text Click here to enter text Click here to enter text Click here to enter text Click here to enter text Click here to enter text The British Psychological Society’s Psychological Testing Centre, St Andrews House, 48 Princess Road East, Leicester, LE1 7DR Tel: 0116 252 9530 Fax: 0116 227 1314 Email: enquiry@psychtesting.org.uk Web: www.psychtesting.org.uk Incorporated by Royal Charter. Registered Charity No 229642 17