© Copyright 2015 Rail Safety and Standards Board Limited Issue Three: March 2015 Rail Industry Standard RSSB Block 2 Angel Square 1 Torrens Street London EC1V 1NY Rail Industry Standard for Train Driver Selection Published by: RIS RIS-3751-TOM Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Issue record Issue Date Comments One 02 August 2008 Original document Two 01 June 2013 Revised psychometric assessment process Three 07 March 2015 Clarification of the risks the psychometric assessment process helps mitigate, clarification of guidance for candidates with previous driving competence on non-GB mainline networks, incorporation of ATOC ACOP006 following ATOC ceasing to oversee the Rail Assessment Centre Forum (RACF) and numerous editorial changes. Superseded or replaced documents The following Railway Group documents are superseded or replaced, either in whole or in part as indicated: Superseded documents Sections superseded Date when sections are superseded RIS-3751-TOM Train Driver Selection Issue one All 30 September 2013 RIS-3751-TOM Train Driver Selection Issue two All 06 June 2015 Supply The authoritative version of this document is available at www.rgsonline.co.uk. Uncontrolled copies of this document can be obtained from Communications, RSSB, Block 2, Angel Square, 1 Torrens Street, London EC1V 1NY, telephone 020 3142 5400 or e-mail enquirydesk@rssb.co.uk. Other Standards and associated documents can also be viewed at www.rgsonline.co.uk. Page 2 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Contents Section Description Part 1 1.1 1.2 1.3 1.4 1.5 1.6 Introduction Purpose of this document Application of this document Health and safety responsibilities The structure of this document Copyright Approval and authorisation of this document 4 4 4 5 5 5 5 Part 2 2.1 2.2 2.3 Requirements for railway undertakings Selection criteria Overview of the selection process Qualifications of people appointed to administer and design psychometric assessment processes for train drivers Assessment of candidates with previous driving competence Delivery of the assessment processes Results of the psychometric assessment process Re-assessments for failed candidates Review of the psychometric assessment process Audit of the psychometric assessment process Transfer of information 6 6 7 2.4 2.5 2.6 2.7 2.8 2.9 2.10 Page 7 8 9 9 10 10 11 12 Appendices Appendix A Standardised train driver psychometric assessment process 14 Appendix B Transfer of safety information – candidate with previous experience as a train driver 16 Appendix C Examples of major life events 26 Appendix D Audit protocol for the train driver psychometric assessment process 27 Appendix E Recommended qualifications, competences and experience for internal and external auditors 64 Definitions 65 References 67 Tables Table 1 Table 2 Table 3 RSSB Driver selection criteria and psychometric assessment methods 15 Records covered by a transfer request of available records relevant to an individual’s employment as a train driver 18 Audit criteria, example indicators, timescales for actions to be completed and requirements for audit 28 Page 3 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Part 1 Introduction 1.1 Purpose of this document 1.1.1 This Rail Industry Standard (RIS) provides requirements and guidance on the selection process for train driver candidates who will drive on the GB mainline network. It has been developed for any member of RSSB to adopt if they wish to do so. 1.1.2 It includes requirements and guidance for the standardised psychometric assessment of train driver candidates. This is part of the overall selection process of train drivers. 1.1.3 In addition it contains requirements and guidance that relate to the transfer of information about train drivers when they change employment between railway undertakings. 1.1.4 The document is set out in the form of requirements followed by guidance. Railway undertakings may wish to use the guidance during the development and management of their train driver selection process. 1.2 Application of this document 1.2.1 A member of RSSB may choose to adopt all or part of this document through company procedures or contract conditions. Where this is the case the member of RSSB will specify the nature and extent of application. 1.2.2 This standard outlines an agreed industry method for the psychometric assessment process for candidates for employment as train drivers who will drive on the GB mainline network. Railway undertakings may wish to conduct psychometric assessments as part of the selection of train drivers according to this standard. 1.2.3 The psychometric assessment process outlined in this standard has been designed and evaluated as part of the RSSB research programme (T340, T628 and T948) by qualified occupational psychologists and provides a robust assessment against a minimum standard of the key aptitudes required for safe train driving. 1.2.4 Railway undertakings that use an alternative psychometric assessment process to that defined in this standard may wish to appoint an occupational psychologist chartered by the British Psychological Society (BPS) or a holder of an equivalent qualification to provide advice on the content of the alternative process. 1.2.5 The psychometric assessment process can be administered within a railway undertaking or an external assessment centre or an assessor can be appointed to administer the process. 1.2.6 Railway undertakings that appoint an external assessment centre to administer the psychometric assessment may wish to check that the assessments will be carried out according to this standard. 1.2.7 The Rail Assessment Centre Forum (RACF) is a group of assessment centre representatives who have undertaken to follow this standard and share good practice. Membership of this forum provides assurance that an assessment centre conducts the psychometric assessment process according to this standard. 1.2.8 For railway undertakings that use the services of an external assessment centre further information on procedures and methods of assessment is provided in BS ISO 106671:2011 Assessment service delivery — Procedures and methods to assess people in work and organizational settings. Part 1: Requirements for the client. Page 4 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 1.2.9 The Train Driving Licences and Certificates Regulations (TDLCR, 2010) and accompanying Office of Rail Regulation (ORR) position statements1 also contain requirements that are relevant to the selection of train drivers. This includes exceptions from the regulations, requirements relating to what criteria must be assessed during selection, the requirement for assessors to be registered with the ORR and for certain documentation to be produced when train drivers change employer. 1.3 Health and safety responsibilities 1.3.1 Users of documents published by Rail Safety and Standards Board Limited (RSSB) are reminded of the need to consider their own responsibilities to ensure health and safety at work and their own duties under health and safety legislation. RSSB does not warrant that compliance with all or any documents published by RSSB is sufficient in itself to ensure safe systems of work or operation or to satisfy such responsibilities or duties. 1.4 The structure of this document 1.4.1 This document is set out as a series of requirements, followed by guidance. The guidance is indicated by prefixing the paragraph number with the letter ‘G’. 1.5 Copyright 1.5.1 Copyright in the Railway Group documents is owned by Rail Safety and Standards Board Limited. All rights are hereby reserved. No Railway Group document (in whole or in part) may be reproduced, stored in a retrieval system, or transmitted, in any form or means, without the prior written permission of Rail Safety and Standards Board Limited, or as expressly permitted by law. 1.5.2 RSSB members are granted copyright licence in accordance with the Constitution Agreement relating to Rail Safety and Standards Board Limited. 1.5.3 In circumstances where Rail Safety and Standards Board Limited has granted a particular person or organisation permission to copy extracts from Railway Group documents, Rail Safety and Standards Board Limited accepts no responsibility for, nor any liability in connection with, the use of such extracts, or any claims arising therefrom. This disclaimer applies to all forms of media in which extracts from Railway Group documents may be reproduced. 1.6 Approval and authorisation of this document 1.6.1 The content of this document was approved by Traffic Operation and Management Standards Committee on 03 February 2015. 1.6.2 This document was authorised by RSSB on 05 February 2015. 1 ORR (2014) Guide to the Train Driving Licences and Certificates Regulations RSSB Page 5 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Part 2 Requirements for railway undertakings 2.1 Selection criteria 2.1.1 Railway undertakings shall apply valid selection criteria to select suitable persons for train driving work. 2.1.2 Railway undertakings shall derive the selection criteria from: 2.1.3 a) The physical demands of the operation on train drivers. b) The aptitudes and mental abilities identified as necessary to ensure competent train driving, after the appropriate training and development has taken place. Railway undertakings shall include the selection criteria as defined in Table 1 of Appendix A of this document. Railway undertakings are required to include the selection criteria defined in Table 1 of Appendix A because they represent the core set of cognitive, psychomotor and behavioural aptitudes required to have the potential to become a safe train driver and were developed and validated by occupational psychologists at RSSB with considerable input from operations subject matter experts. The selection criteria defined in Table 1 of Appendix A do not include the physical capabilities that are required for train driving. The physical capabilities required to be assessed during the selection process are to be defined by the railway undertaking with reference to GO/RT3451 Train Drivers – Suitability and Medical Fitness Requirements. The selection criteria and assessment methods defined in Table 1 of Appendix A may also be used as part of the selection process for other railway roles that have tasks or characteristics in common with train driving, for example, shunters. Railway undertakings may wish to review the particular role in question to determine what cognitive, psychomotor and behavioural aptitudes are required for that role and what levels of aptitude or skill are required. Based on this, the railway undertaking can choose which of the criteria and methods in Table 1 of Appendix A to apply and how they should be applied. Physical demands To help meet the requirement set out in 2.1.2, a railway undertaking may wish to assess the physical demands placed on train drivers by their operations, identifying the factors which influence the physical requirements for train driving. The key factors influencing physical requirements include: a) Position of steps, door handles or handrails on rail vehicles in relation to the railway infrastructure. b) Operational requirements to access and operate equipment inside or outside trains or on the railway infrastructure. c) Design of driving cabs, including sight lines. The selection criteria based on these factors represents the minimum physical requirements necessary for performance of train driving tasks without increased risk to the safety of train drivers or to train operations. Page 6 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection The railway undertaking may wish to inform the occupational physicians who conduct or supervise medical assessments of train drivers of these physical requirements. In addition the railway undertakings may wish to obtain advice from occupational physicians and ergonomists on defining and applying physical requirements and on compliance with disability discrimination legislation. Cognitive, psychomotor and behavioural aptitudes In addition to the selection criteria detailed in Appendix A, a railway undertaking can assess candidates against any additional selection criteria that are relevant to their own organisation and operation. The selection criteria detailed in Appendix A are consistent with the criteria required to be assessed to obtain a train driver licence under the Train Driving Licences and Certificates Regulations (TDLCR 2010). In addition to the selection criteria in Appendix A, candidates for employment as cross-border drivers are to be assessed for language skills to the standard required in TDLCR 2010. 2.2 Overview of the selection process 2.2.1 Railway undertakings shall use a selection process which includes: a) A psychometric assessment process as defined in Table 1 of Appendix A of this document for all candidates who have no previous experience as a train driver. b) Any evidence from the candidate which might affect a conclusion on the likelihood of future safe performance. The railway undertaking is required to include the psychometric assessment process in Table 1 of Appendix A because it was designed to cover all of the key aptitudes required for safe train driving and help mitigate the risk of a railway undertaking selecting candidates who do not possess, to the standard required, the core set of cognitive, psychomotor and behavioural aptitudes required to have the potential to become a safe train driver. The Multi-Modal Interview (MMI) assesses oral communication and not written communication, because written communication is considered to be important in train driving but not safety-critical so it is not included in the core psychometric assessment process. If the railway undertaking wishes to assess written communication, the Written Communication Test (WCT) provides a structured way to do this. 2.3 Qualifications of people appointed to administer and design psychometric assessment processes for train drivers 2.3.1 Railway undertakings shall appoint persons qualified to administer and interpret the psychometric assessment methods used. Rail undertakings may wish to appoint persons to manage the psychometric process who have the necessary qualifications, expertise and experience in the: RSSB a) Administration of the psychometric assessment methods. b) Conduct of interviews. c) Interpretation of results. d) Feeding back of results to candidates and to recruiting railway undertakings. Page 7 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection People appointed to administer the psychometric assessment process are to hold the ‘Test User: Occupational, Ability’ British Psychological Society (BPS) qualification. This qualification was formerly known as the British Psychological Society Accredited Level A qualification. The qualifications to administer and interpret psychometric assessments may vary according to the requirements of each individual assessment publisher. RSSB is the test publisher of the Situational Judgement Exercise, MultiModal Interview and Written Communication Test and as such persons who use these methods are to be appropriately trained and certificated by RSSB, or by an individual or organisation that RSSB has authorised, to provide training and certification. Railway undertakings that use any additional selection criteria and psychometric assessment methods other than those outlined in Appendix A may wish to appoint a person with the BPS ‘Test User: Occupational, Ability’ qualification to select appropriate psychometric assessment methods. 2.4 Assessment of candidates with previous driving competence 2.4.1 There are no requirements associated with this section, rather guidance is provided for the railway undertaking to consider. A selection process including the psychometric assessment process does not have to be used for candidates with previous competence as a train driver on the GB mainline network. The extent to which the selection process is modified for such candidates, and the use of psychometric assessment methods in each individual case, depends on the information available to the recruiting railway undertaking. The recruiting railway undertaking may wish to consider information on the following factors when deciding on the selection process to be applied: a) The amount of time spent as a driver after initial training and certification. b) The candidate’s safety record, confirmed by the current and previous employer (see Appendix B of this document). c) Recent competence assessment and monitoring records confirmed by the current and previous employer (see Appendix B of this document). d) Differences between previous driving experience and the railway undertaking’s own operations. This might be relevant to the choice of psychometric assessment methods (for example, suburban operations compared with long-distance high-speed passenger, freight working compared with passenger, etc). e) The period of time since the candidate last worked as a train driver and the reason for ceasing train driving work. f) Any evidence of major life events which might affect safe performance in future, for examples see Appendix C of this document. The recruiting railway undertaking may wish to document the psychometric assessment process adopted for a candidate with train driving experience, together with reasons for differences between it and that used for other candidates. Page 8 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection A railway undertaking may wish to use a selection process including the psychometric assessment process for candidates with previous competence as a train driver on non-GB mainline networks, unless they have previously been assessed to the standard specified in this document. 2.5 Delivery of the assessment processes 2.5.1 There are no requirements associated with this section, rather guidance is provided for the railway undertaking to consider. Assessors may wish to consider what preparation materials are appropriate for each assessment method and consider providing candidates, at least seven days in advance of the assessment, with the following: a) A description of the assessment methods and interview that will be used. b) Practice materials with correct responses. c) Advice on personal preparation for the process. Further guidance on the delivery of psychometric assessment processes is provided in BS ISO 10667-2:2011 Assessment service delivery — Procedures and methods to assess people in work and organizational settings. Part 2: Requirements for service providers. The Train Driver Assessment Procedures Manual provides detailed instructions on how to administer the psychometric assessment process. It is maintained jointly by the Rail Assessment Centre Forum and RSSB and is currently available to Rail Assessment Centre Forum members on request from the Forum. 2.6 Results of the psychometric assessment process 2.6.1 Railway undertakings shall make a judgement about whether a candidate passes or fails the psychometric assessment process based on the agreed industry scoring criteria that are published in the Train Driver Assessment Procedures Manual. 2.6.2 Railway undertakings shall record each assessment score and the outcome from the assessment of each selection criterion for each candidate. To facilitate the final selection decision the railway undertaking may wish to provide the recruiting manager with guidance on the scores achieved. Railway undertakings may wish to provide feedback to each candidate regarding their performance on the psychometric assessment process. The results of a psychometric assessment that was conducted according to this rail industry standard are to be regarded as valid for a period of five years from the date of assessment, unless the candidate has had a major life event (for examples see Appendix C of this document) that may have caused a decrease in cognitive ability or a change in behavioural tendencies. Railway undertakings may wish to put appropriate processes in place to detect the occurrence of a major life event in the time between assessment and recruitment. If the incidence of such an event is suspected, the railway undertaking may wish to re-assess the candidate prior to employment, with the assessment delayed until a time when the candidate feels they have regained their performance. A copy of the candidate’s scores and the outcome of the assessment are to be given to the railway undertaking which commissioned the assessment and retained on the candidate’s personal file for future reference. RSSB Page 9 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection For candidates who have passed the psychometric assessment process, the final selection decision is a managerial one, for which the results of the psychometric assessment process provide evidence. 2.7 Re-assessments for failed candidates 2.7.1 Railway undertakings shall not re-assess a candidate who has failed the psychometric assessment process within six months. 2.7.2 Railway undertakings shall not re-assess a candidate who has failed the psychometric assessment process more than once unless there are mitigating circumstances that have been considered and documented by the railway undertaking. When a candidate is re-assessed, the process needs to cover only the elements which the candidate failed at the first assessment, if: a) It is carried out within five years of the first assessment. b) There is no evidence of a major life event having occurred since the first assessment, for examples see Appendix C of this document. Any candidate who failed a previous version of the psychometric assessment process is considered by the industry Driver Selection Governance Group to be eligible to be assessed up to twice using the process outlined in Appendix A. 2.8 Review of the psychometric assessment process 2.8.1 Railway undertakings shall review the psychometric assessment process periodically to evaluate the suitability and effectiveness of the selection criteria and associated assessment methods for their organisation. 2.8.2 Railway undertakings shall consider the following elements within the psychometric assessment process review: a) The impact of any changes to train driving work and the skills required of train drivers on the selection criteria. b) Developments in psychometric assessment methodology. c) The reliability, validity and fairness of the psychometric assessment methods used. The suitability and effectiveness of psychometric assessment processes can be affected by changes in the role being selected for and changes in the candidate population. Railway undertakings are therefore required to periodically review their psychometric assessment processes to check that they are working effectively and remain fit for the purpose they are intended. Examples of changes to train driving work which could trigger a review of the psychometric assessment process include: a) The introduction of different working practices such as driver only operation or operations across international borders. b) Technical changes, such as high-speed operation or new signalling systems affecting the interface between the train driver and the signalling system. The psychometric assessment process outlined in Appendix A is subject to an evaluation performed by RSSB every five years on a national basis. This evaluation covers the suitability of the selection criteria and the reliability, validity and fairness of the assessment methods. Page 10 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Railway undertakings who are following the psychometric assessment process in this standard can use the RSSB evaluation to inform their review. This would cover requirements b) and c) of 2.8.2. However, railway undertakings may also wish to take into consideration the characteristics of their own organisation and operation. 2.9 Audit of the psychometric assessment process 2.9.1 Railway undertakings shall arrange for the audit of their psychometric assessment process to check the reliability and consistency of the process and compliance with the procedures at the assessment centre. 2.9.2 Railway undertakings shall appoint persons qualified to judge the reliability and consistency of the psychometric assessment process to carry out audits. 2.9.3 Railway undertakings shall involve persons who are independent of the provider of the psychometric assessment process in their audit arrangements. In addition to the involvement of an independent person in the audit of the railway undertaking’s psychometric assessment process, the assessment provider (usually an assessment centre) may wish to carry out internal audits of its assessment practice and compliance with its procedures. An external audit involves an independent evaluation of the railway undertaking’s psychometric assessment process and is in direct contrast with an internal audit performed by someone from within the assessment provider (usually an assessment centre). An independent auditor is to have the required audit competence and not be a stakeholder in, nor have a dependency on, the outcome of the audit. The same audit protocol can be applied to external and internal audits, with the external audit undertaken in more detail. The railway undertaking may wish to use the audit protocol outlined in Appendix D, for external and internal audits, as it has been designed to provide checks that the psychometric assessment process is effective in meeting the requirements of this document and contributes to the railway undertaking’s safety management system. Where an external assessment centre supplies psychometric assessment services to a railway undertaking, the railway undertaking may wish to check that the contract provides for internal and external audit and for audit results to be made available to the railway undertaking. To provide satisfactory checks that the psychometric assessment process is effective in meeting the requirements of this document, and contributes to the railway undertaking’s safety management system, internal and external audits can be undertaken every two years, alternating so that the frequency of audits is every year. Railway undertakings may wish the scope of the audit, such as that outlined in Appendix D, to include: RSSB a) Use of industry standard selection tools. b) Candidate and assessor preparation. c) Training and certification of assessors. d) Administration of assessment methods (including adequacy of the premises where they are conducted), scoring and recording of results and processes for making and recording decisions based on the results of the assessment methods used. e) Confidentiality and security of selection materials and records. Page 11 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection f) Storing and recording of results. So far as is reasonably practicable, the railway undertaking may wish for more than one method of auditing to be used, including: a) Observation of test administration and scoring. b) Review of documentation. c) Interviews with assessors to ensure compliance with the relevant competence requirements and conditions of certification (where applicable). Railway undertakings may wish their psychometric assessment process to be audited against criteria, such as those outlined in Appendix D, with ratings provided for each criteria. To judge the reliability and consistency of the psychometric assessment process, a railway undertaking may wish to appoint internal and external auditors who have the necessary qualifications, competences and experience, such as those outlined in Appendix E to undertake their tasks competently and who: a) Understand the selection requirements for train driving. b) Understand the risks that those requirements are intended to control. c) Possess knowledge of the psychometric assessment process being audited and of the audit procedure itself. Railway undertakings may wish to comprehensively document and communicate to relevant parties audit findings and ensure corrective actions are undertaken. 2.10 Transfer of information 2.10.1 Railway undertakings shall, if a candidate discloses previous experience as a train driver, request information relevant to competent train driving performance from a railway undertaking named as a previous employer. 2.10.2 Railway undertakings shall co-operate with another railway undertaking which requests information to facilitate its compliance with section 2.10.1. Previous train driving competence The recruiting railway undertaking may wish to obtain the information to meet this requirement before the candidate is permitted to drive trains by the recruiting railway undertaking. Information from the candidate When a candidate indicates that they are, or have been, employed in a train driving role in the rail industry, the railway undertaking is to request information, in as much detail as possible, which can be passed to the current or former employer for verification. A recommended format for recording this information ready for verification can be found in Appendix B of this document. When requesting information from the candidate, the railway undertaking may wish to consider: a) The results of any assessment methods used for selection as a train driver. Page 12 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection b) The most recent competency records that demonstrate the candidate’s competency. c) Incidents and accidents for which the candidate was found to be wholly or partly responsible. d) Records of any special monitoring arrangements in place at the time the candidate left that railway undertaking. e) Route and traction competence current at the time the candidate left that railway undertaking. f) Temporary or permanent restrictions or exclusions placed upon the candidate’s work as a train driver because of a safety of the line incident, medical reasons or other cause. Railway undertakings may wish to check whether the candidate has worked previously in any other safety-critical role within the rail industry. The requirements in this document for co-operation in the confirmation and transfer of information are limited to train driving. However, specific information about the safety record of candidates who have worked in other safety-critical roles could be requested as part of the normal referencing process. The information provided by a current and / or previous employer is to be, so far as is reasonably practicable, sufficient for the recruiting railway undertaking to confirm the accuracy of information provided by the candidate and to fill in any gaps. The accuracy and value of competence information reduces over time, and it is advisable to treat cautiously any information more than two years old that relates to performance. It is inappropriate to use out-of-date competence records as a justification for curtailing selection procedures. Retention of records Railway undertakings may wish to retain records (or retain access to records if held elsewhere, for example by an external assessment centre) for such period as is necessary to assist them in responding to requests from other railway undertakings. The type of information a railway undertaking may wish to retain includes: a) Results of psychometric assessments. b) Competence assessment records; the most recent competency records that demonstrate the train driver’s competence. c) Route and traction knowledge; current at the time the candidate left the company. d) Training records; details of training received, dates and trainee performance. e) Safety records; details of any accidents, incidents or near misses for which the candidate was found to be wholly or partly responsible. f) Occupational medical records of conditions that have affected a candidate’s ability to undertake their duties as a train driver. Data protection Railway undertakings may wish to obtain expert advice on data protection, liabilities arising from the provision of references or any aspect of employment law, as these matters are not within the scope of this document. RSSB Page 13 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Appendix A Standardised train driver psychometric assessment process A.1 Driver selection criteria and psychometric assessment methods Selection criteria Sub-criteria and definitions Psychometric assessments Attention Selective attention - The ability to differentiate between different sources of information and attend selectively to them, for example distinguishing and attending to alarms (selective attention). Paper Group Bourdon and Test of Everyday AttentionOccupational (TEA-Occ) Divided attention - The ability to switch attention between sources of information, for example lineside information and in-cab information and perform different tasks in parallel, for example making train announcements while on the move. Vigilance The ability to attend and respond to stimuli which occur relatively infrequently and over extended periods of time. WAFV Vigilance test Memory The ability to learn, recall and apply job related information in appropriate time limits, for example learn new information in training; remembering instructions from signallers; applying specific rules and procedures. TRP1 Trainability for Rules and Procedures Test (part 1) Reasoning The ability to solve problems and make decisions, for example fault diagnosis; understanding and interpreting information from instrumentation. TRP2 Trainability for Rules and Procedures test (part 2) Perception The ability to anticipate elements in a traffic environment and make a correct decision about how to respond given the speed and distances involved, for example identifying a landmark cue before a station and starting to decelerate. ATAVT Adaptive Tachistoscopic Traffic Test Reaction time A quick and adequate response to simple and complex visual and acoustic stimuli and the associated quality of performance. WAFV Hand coordination The ability to make appropriate and controlled movements in response to decisions about complex stimuli. 2HAND Two hand coordination test Communication The ability to read, listen, understand and respond appropriately, and effectively convey information orally and in writing. Multi-Modal Interview (MMI) Foreign language skills (only applicable to crossborder drivers operating in Europe) - The ability to communicate in the language indicated by the infrastructure manager concerned on critical safety issues. Their language skills must be such that they: can communicate actively and effectively in routine, adverse and emergency situations; can cope with practical situations involving an unforeseen element; can describe; can keep a simple conversation going. Page 14 of 67 RIS-3751-TOM Issue Three : March 2015 ATAVT Optional: Written Communication Test RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Selection criteria Sub-criteria and definitions Psychometric assessments Behavioural aptitudes No established occupational psychological deficiencies, particularly in operational aptitudes or any relevant personality factor. Situational Judgement Exercise (SJE) and MMI Table 1 Driver selection criteria and psychometric assessment methods A.1.1 The computerised Group Bourdon is not equivalent to the paper Group Bourdon and is not recommended for use as part of the train driver psychometric assessment process. A.1.2 Where more than one assessment method is specified for a selection criterion both assessment methods are to be used in combination, for example both the paper Group Bourdon and the TEA-Occ tests are to be used to assess attention. The use of multiple methods to assess a selection criterion provides a more reliable assessment than using one method alone. A.1.3 Sub-criteria and detailed definitions of the behavioural aptitudes criteria have been developed but are not included in this Rail Industry Standard due to the risk that candidates could use this information to get an unfair advantage during the psychometric assessment process. This information is available to railway undertakings and assessment centres in the Train Driver Assessment Procedures Manual. RSSB Page 15 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Appendix B Transfer of safety information – candidate with previous experience as a train driver B.1 Form for transfer of safety information – candidate with previous experience as a train driver Instructions for the use of this form 1. The relevant parts of sections 1 - 5 should be completed by the candidate. Any information that needs to be provided but does not fit on the form should be entered in the additional information section on the final page of the form. 2. The employer to which the candidate is applying (prospective employer) should make a copy of the form for each current and former employer identified by the candidate in section 2. The prospective employer should forward the form to each current or former employer. 3. The current or former employer(s) should check and confirm the details provided by the candidate and provide additional details or corrections as required. If the current or former employer cannot confirm some of the details then they should indicate this by using the ‘unable to confirm’ option. Any information that needs to be provided but does not fit on the form should be entered in the additional information section on the final page of the form. The current or former employer should sign the declaration in section 6 and return the completed form to the prospective employer at the specified address. Page 16 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Section 1. Transfer of safety information – Request for confirmation from current or former employer 1.1 Candidate’s name 1.2 Please declare employment status (circle appropriate box) Section 2 covers employment history, including names and location(s) of employment Currently employed 1.3 Do you hold a European Train Driving Licencing Directive License? (circle appropriate box) Yes No If Yes, please also complete either 1.4a or 1.4b depending on your circumstance If No, go to 1.5 1.4a If you hold a European Train Driving Licencing Directive License and are currently employed please provide the license number and category of certificate held. Please attach a copy of your license and certificate to this form. European Train Driving Licencing Directive License number: Category of European Train Driving Licencing Directive Certificate held: 1.4b If you hold a European Train Driving Licencing Directive License and are NOT currently employed please provide the license number. Please attach a copy of your license and a certified copy of the European Train Driving Certificate as per Office of Rail Regulation (ORR) Guide to the Train Driving Licences and Certificates Regulations Annex B5. This will have been issued by your previous employer European Train Driving Licencing Directive License number: Previously employed Self-employed 1.5 Name of prospective employer 1.6 Address for the form to be returned to 1.7 Date of form completion The person named above has applied for a train driver’s position with our company. This person has provided written consent for your company to disclose information regarding their competence and previous safety performance. A copy of the signed consent and the information provided by the candidate is attached. Please confirm within five working days whether the information supplied by the candidate is accurate. If the information is incorrect, or you are aware of omissions, please provide corrected information as far as you are aware of it. Please indicate if you cannot confirm any of the information provided by the candidate. Should the candidate be selected for employment as a train driver with this company, a request will be made for the transfer to this company of available records relevant to the individual’s employment as a train driver. Relevant records covered by such a request are provided in Table 2. RSSB Page 17 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Records relevant to the individual’s employment as a train driver Results of tests used in selecting the applicant as a train driver Results of competence assessment records: the most recent competency records that demonstrate the train driver’s competence Route and traction competence current at the time the applicant left your company Training records Incidents, accidents and near misses for which the applicant was found to be wholly or partly responsible Occupational medical records that have affected the person’s ability to undertake their duties as a train driver Restrictions or exclusions placed upon the applicant’s work as a train driver because of a safety of the line incident, medical reasons or other cause A current certified copy of the European Train Driving Certificate issued in accordance with Article 17 of Directive 2007/59/EC, by current employer or a certified copy of the European Train Driving Certificate as per Office of Rail Regulation (ORR) Guide to the Train Driving Licences and Certificates Regulations Annex B5, issued by previous employer Table 2 Records covered by a transfer request of available records relevant to an individual’s employment as a train driver Section 2. Employment details Please provide details of current and/or former employers where you have undertaken the role of train driver. 2.1 Name of employer 2.2 Job title and location 2.3 Start date of employment 2.4 End date of employment Section 3. Declaration to contact current and/or previous employers The information requested in sections 4 and 5 inclusive of this document is required to establish your competence and safety record as a train driver and will assist in the formation of future assessment and training plans if you join this company. The information is necessary for us to consider your suitability for employment as a train driver. An employer is not entitled to disclose information about an employee unless the employer is under a legal obligation to do so or the employee concerned has given his/her consent. Subject to your consent, your current or a former employer will be requested to confirm the accuracy of the information you have provided. Please sign below to indicate that you agree for this information to be sent to your current or former employer for confirmation. If you fail to provide details, or the accuracy of the information is incorrect, your application will not be progressed. If it is later found that you have provided inaccurate information, you will be liable to summary dismissal. Page 18 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Declaration Candidate’s signature Date of signature 3.1 I hereby give my permission for my current and/or former employer(s) to: a) Confirm the information I have given using records they may hold b) Provide my prospective employer with a copy of my records (as detailed in Table 2) in the event of my appointment 3.2 I understand that these records will then be transferred and form part of my employment history with my new employer Section 4. Competence This section requires you to provide details of your competence and recent competence assessments as a train driver. You do not need to complete this section if you provide one of the following: 1. A current certified copy of your European Train Driving Certificate issued in accordance with Article 17 of Directive 2007/59/EC. 2. A certified copy of the European Train Driving Certificate as per Office of Rail Regulation (ORR) Guide to the Train Driving Licences and Certificates Regulations Annex B5. Your previous employer will provide this. If you cannot provide one of the above please complete items 4.1 to 4.8. Candidate should complete ‘Details provided by the candidate’. Current or former employer should complete ‘Details provided by the former employer’. Details provided by the former employer Details provided by candidate Details are correct and complete Additional information (If applicable) Delete as appropriate 4.1 Date initially certified competent as a train driver Yes/No/ Unable to confirm 4.2 Date and details of any additional support or development in place currently or at the time of leaving your last employer Yes/No/ Unable to confirm RSSB Page 19 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.3 Traction/ Rolling stock on which you are competent (or were competent at the time you left your last employer) 4.4 Date of completion of last reassessment of competence 4.5 Results of last reassessment of competence Details are correct and complete Additional information (If applicable) Delete as appropriate Use one row per traction or rolling stock type Yes/No/ Unable to confirm Yes/No/ Unable to confirm Yes/No/ Unable to confirm Yes/No/ Unable to confirm Details provided by the former employer Details provided by candidate 4.6 Infrastructure/ routes on which you are competent (or were competent at the time you left your last employer) 4.7 Date of completion of last reassessment of competence 4.8 Results of last reassessment of competence Details are correct and complete Additional information (If applicable) Delete as appropriate Use one row per infrastructure/route Yes/No/ Unable to confirm Yes/No/ Unable to confirm Yes/No/ Unable to confirm Yes/No/ Unable to confirm Page 20 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Section 5. Safety Record In this section provide details of safety and operational incidents that you have been involved in during your railway employment as a train driver, or in other safety-critical railway operations work, and which was recorded on your personal record by your employer, whether or not formal disciplinary action was taken. Candidate should complete ‘Details provided by the candidate’. Current or former employer should complete ‘Details provided by the former employer’. 5.1 Signals passed at danger (SPADs) This section refers only to Category A1 SPAD event types as defined in GO/RT3119 Accident and Incident Investigation. Details provided by candidate Details provided by the former employer Details are correct and complete Additional information (If applicable) Delete as appropriate 5.1.1 Have you previously been involved in any SPAD incidents (including incidents where you were subsequently exonerated)? Yes/No If yes, complete 5.1.2 and 5.1.3 on a separate row for each SPAD incident Yes/No/ Unable to confirm Delete as appropriate 5.1.2 Date of SPAD 5.1.3 Severity Details are correct and complete Additional information (If applicable) Delete as appropriate Yes/No/ Unable to confirm Yes/No/ Unable to confirm Yes/No/ Unable to confirm RSSB Page 21 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 5.2 History of operational incidents Details provided by candidate Details provided by the former employer Details are correct and complete Additional information (If applicable) Delete as appropriate 5.2.1 Have you previously been involved in any of the following (including incidents where you were subsequently exonerated): excess speed (including TPWS interventions), TPWS activations, station overruns, failure to call, stop short and release of doors, wrong side release, collisions, derailments or other safety incidents? Yes/No If yes, complete 5.2.2 and 5.2.3 on a separate row for each incident Yes/No/ Unable to confirm Delete as appropriate 5.2.2 Date of incident 5.2.3 Description of the incident Details are correct and complete Additional information (If applicable) Delete as appropriate Yes/No/ Unable to confirm Yes/No/ Unable to confirm Yes/No/ Unable to confirm Yes/No/ Unable to confirm Yes/No/ Unable to confirm Page 22 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 5.3 Removal or restriction from train driving work Details provided by candidate Details provided by the former employer Details are correct and complete Additional information (If applicable) Delete as appropriate 5.3.1 Have you ever been removed from train driving, or had restrictions placed on you, because of a safety of the line incident, your health or other cause? Yes/No If yes, complete 5.3.2 – 5.3.4 on a separate row for each separate removal or restriction Yes/No/ Unable to confirm Delete as appropriate 5.3.2 Start date of removal or restriction 5.3.3 End date of removal or restriction 5.3.4 Details and cause of removal or restriction Details are correct and complete Additional information (If applicable) Delete as appropriate Yes/No/ Unable to confirm Yes/No/ Unable to confirm Yes/No/ Unable to confirm Yes/No/ Unable to confirm Yes/No/ Unable to confirm RSSB Page 23 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 5.4 Other information on fitness, competence or safety performance The candidate should use this section to provide any other relevant details about fitness, competence or safety performance that were not provided in the previous sections. Details provided by candidate Details provided by the former employer Details are correct and complete Additional information (If applicable) Delete as appropriate Yes/No/ Unable to confirm Section 6. Declaration by current or former employer I have verified the information supplied by the candidate and indicated (where necessary): a) Where information is factually incorrect according to our records. b) Omissions according to our records. c) Information which this company does not have supporting information to confirm. 6.1 Company 6.2 Name of person who completed the form 6.3 Position of person who completed the form 6.4 Signature of person who completed the form 6.5 Date of signature Page 24 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Use this section to provide any information that you cannot fit in the appropriate section of the form. Enter the reference number for the relevant question in the ‘number’ column and indicate if you are the candidate or the current or former employer in the ‘provided by’ column. Number RSSB Additional information Provided by Page 25 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Appendix C Examples of major life events C.1.1 There are situations which could impair cognitive performance in the short term, such as the death of a loved one, divorce, or illness. While these events are likely to cause considerable emotional stress to the individual, any decrement in performance would be temporary, and the individual would recover without any long term impact. C.1.2 There are other conditions that could cause long term, or permanent decrement to the individual’s cognitive performance and impact on the likelihood that they would be suitable for a safety-critical role. C.1.3 The list below (C1.4) provides examples of significant events or conditions that could lead to cognitive impairment and affect a candidate’s suitability. This is not an exhaustive list. Consideration of the types of events shown below may be used to decide whether reassessment using the psychometric assessment process is required. The list is to be used only for this purpose and not to eliminate or discriminate against candidates. C.1.4 Examples of significant events or conditions that could affect cognitive performance include: Serious illness (including Cancer, Bacterial Meningitis). Stroke. Significant accident or injury. Traumatic brain injury. Brain Tumour. Degenerative disorders (for example dementias). Progressive disorders (for example Multiple Sclerosis, Parkinson’s Disease). Toxic or metabolic disorders (for example alcohol related disorders, neurotoxins). Oxygen deprivation. Alcohol or substance abuse / long term use of medication. Epilepsy /seizures. Psychological illness such as: clinical / serious depression, clinically diagnosed anxiety, Post-Traumatic Stress Disorder. Motor Neurone Disorder. Sleep Disorder (for example Sleep Apnoea). C.1.5 If there is evidence that a significant event or condition has occurred, it is recommended that an assessment is completed by either, depending on the nature of the assessment, a medical professional / practitioner, psychologist registered by the Health and Care Professions Council or an individual suitably qualified to undertake such assessments, to determine the extent to which the significant event and / or condition may have affected the cognitive performance of the train driver. Page 26 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Appendix D Audit protocol for the train driver psychometric assessment process D.1 Before commencing the audit D.1.1 The auditor (internal or external) is to obtain a full copy of the last internal and external audit and review for specific previous non-compliances, which are to be investigated and confirmation of corrective action obtained at this audit. If any of these have not been satisfactorily addressed they are to be categorised as a C grade and dealt with accordingly as detailed in Section D4. D.1.2 The auditor is to obtain a list identifying all assessments carried out since the last equivalent audit, to identify candidate’s records to be checked (Section D2). D.1.3 The auditor is to obtain a list of approved assessor(s) for the assessment provider (usually an assessment centre), their competency records and sample signatures and use these to decide who will be interviewed as part of the audit and for cross-checking against assessment documentation. D.2 Sampling and sample size D.2.1 The minimum number of candidate records to be physically checked is six for an internal audit and 24 for external audits. Candidate records are to be selected at random by the auditor from all candidate records generated since the last equivalent (ie internal or external) audit. D.2.2 Depending on the extent and / or severity of non-compliances discovered (Section D4) from the first sample of candidate records, a second sample (minimum 12) can be taken to further define the extent of any problem. D.3 D.3.1 D.4 Audit duration An external audit is to be conducted in greater detail than an internal audit and as such it is recommended that an external audit is undertaken over two days, with one day for assessing how assessments are conducted and one day for interviews and assessing documentation. Audit rating, non-compliance and actions arising D.4.1 The Audit Form sets out standards of compliance for each audit area, termed compliance area (Section D6). The auditor is to rate each compliance standard being assessed using the audit rating described in Table 3. D.4.2 The compliance standards are drawn from a number of sources including: RIS-3751-TOM. The Train Driver Assessment Procedures Manual. Test manual: Multimodal Interview, Situational Judgement Exercise and Written Communication Test. General information on the Register of the British Psychological Society’s Qualifications in Test Use. D.4.3 RSSB The auditor is also to ensure that the relevant timescales for action to be completed and requirements for audit, detailed in Table 3, are adhered to. Page 27 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Audit rating Examples in respect of the non-compliance (including but not limited to) Timescale for actions to be completed and requirement for audit A - Meets specification Psychometric process meets the requirements as set out in this document. None B - Minor noncompliance with specification Erroneous database entries. To be rectified within one month of audit and specifically checked and confirmed at next equivalent audit as per schedule ie either internal or external. Some, but less than 2%, errors in grading / scoring with no effect on outcome. Administrative errors. Other non-serious noncompliances. C - Serious noncompliance with specification More than 2% errors in grading / scoring results with no effect on outcome. To be rectified within one month of audit and specifically checked and confirmed at next equivalent audit which is to be an external audit within 6 months whether such was scheduled or not. D - Serious noncompliance with specification with safety-critical implications Assessments or audits carried out, scored or checked by unqualified staff. The auditor is to immediately and separately notify the Chair of the Driver Selection Governance Group (DSGG) via Rail Assessment Centre Forum (RACF) Chair in writing. The assessment provider (usually an assessment centre) is to rectify the matter immediately and confirm that this has been done to the auditor and DSGG via the RACF Chair. Grading / scoring resulting in passing otherwise failed candidates. Failure to update database. A Director of the assessment provider is to confirm all non-compliant actions are addressed, in writing, via the RACF Chairperson. The specific areas of non-compliance identified are to be checked and confirmed at next audit, which is to be an external audit within three months. Further audits will be at the expense of the assessment provider. Table 3 D.4.4 D.5 Audit criteria, example indicators, timescales for actions to be completed and requirements for audit Following identification of non-compliance, the Chair of RACF will establish a recovery plan with the assessment provider and, when it has been actioned and the non-compliance remedied, they will inform all companies who use the audited assessment centre of this change in status. Audit findings D.5.1 The findings and recommendations of the audit are to be comprehensively documented, communicated to relevant parties and detail the areas of compliance, non-compliance and the corrective actions that will be undertaken to address non-compliance. D.5.2 A copy of the audit documentation is to be: Page 28 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Retained on file at the assessment centre. Sent to the Chair of the RACF. Distributed to all companies procuring a selection service for train driving from the assessment provider. D.6 Audit Form Instructions for the use of this form 1. This Audit Form provides the auditor with a comprehensive structure for undertaking an audit. 2. Section 1 requires general information including names of assessor(s), date of audit etc. 3. This is followed by Section 2 which requires a summary of identified noncompliances. This can be completed by the auditor at the end of the audit. 4. The framework is then split into three sections – Section 3: Housekeeping, Section 4: Delivery of Assessment Centre and Section 5: Historical Data. Each of these sections are structured as follows: a) Compliance area. b) Compliance standard that is being assessed. c) Auditor’s rating of each compliance standard - A, B, C or D. d) Source(s) of evidence. e) Recommended method of collecting evidence. f) Comments from the auditor and / or interview questions if non-compliance is detected. g) Date of audit. 5. Section 6 provides the assessor competence recording sheet. The auditor is required to record relevant certifications, registrations and qualifications for each assessor. 6. Section 7 provides the historical data check sheet. The auditor is required to record the details of the candidate records checked as part of the audit. 7. Work systematically through this Audit Form. Inability or refusal of the assessment provider to allow the auditor unhindered access to the selected records, documentation or staff is to be treated as a serious non-compliance and dealt with accordingly. 8. If no assessment activity is taking place at the time of an external audit then the audit is to cover Section 3 - Housekeeping and Section 5 – Historical Data until such time as Section 4 - Delivery of Assessment Centre can be carried out. RSSB Page 29 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Section 1. General information 1.1 Type of audit (circle appropriate box) External Internal 1.2 Auditor name First name Surname 1.4 Date of audit From To 1.5 Name of assessor(s) audited First name Surname 1.6 Audit decision (circle appropriate box) Audit acceptable Areas of non-compliance 1.7 Has audit certificate been issued (circle appropriate box)? Yes No 1.3 Name of assessment centre(s) being audited 1.8 If audit certificate has not been issued, what is the reason? 1.9 Date of next audit Page 30 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Section 2. Summary of non-compliance The auditor should complete this section if non compliances have been identified. 2.1 Compliance rating 2.2 Remedial action to be taken 2.3 By date 2.4 By whom 2.5 Date compliance achieved 2.1.1 Compliance area and relevant standard(s) assessed as B 2.1.2 Compliance area and relevant standard(s) assessed as C 2.1.3 Compliance area and relevant standard(s) assessed as D Rating Key: B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications RSSB Page 31 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Section 3. Housekeeping 3.1 Compliance area 3.2 Compliance standard 3.1.1 Suitable accommodation 3.1.2 Security/confidentiality of documentation, materials and electronic media 3.1.3 Working order of computer for Vienna Test System (WAFV, ATAVT, 2HAND) and Situational Judgement Exercise 3.1.4 Audio equipment in working order 3.3 Rating 3.4 Source(s) of evidence 3.5 Method of collecting evidence Suitable lighting/light/ temperature. Refreshments available. Low noise. Environment Inspection Test materials. Procedures manual. Security of passwords and access to RACF Database to authorised persons. Document Availability of spares. Mechanical soundness. Lead assessor Interview Audio equipment in good working order. Volume control adjustable. Observation Inspection 3.6 Comments/Questions/Notes (if non-compliant) 3.7 Date Audited Computer How do you ensure that the computer equipment is regularly maintained? Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. Page 32 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 3.1 Compliance area 3.2 Compliance standard 3.1.5 Train driver assessment procedures manual is complete and up to date 3.1.6 Rail Assessment Centre Forum meetings 3.1.7 Evidence of assessor competence 3.4 Source(s) of evidence 3.5 Method of collecting evidence Manual complete. Manual up to date. Manual contains example materials. Lead assessor Document Periodic intervals – evidence of attending 50%, ie two meetings per year Lead assessor Document Please provide evidence of attendance at Rail Assessment Centre Forum meetings in the last 12 months. Each assessor carrying out the Multimodal Interview (MMI) holds an MMI Certificate. Each assessor holds the ‘Test User: Occupational, Ability’ British Psychological Society (BPS) qualification’. Each assessor is registered with the ORR as a ‘psychologist’. Document Documents Use the sheet in Section 6 to record: 3.3 Rating 3.6 Comments/Questions/Notes (if non-compliant) 3.7 Date Audited a. Names of assessor(s) carrying out MMI and certification expiry date. b. Checks that assessor(s) hold the ‘Test User: Occupational, Ability’ British Psychological Society (BPS) qualification’. c. Checks that assessor(s) are on the ORR register as a ‘psychologists’. Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. RSSB Page 33 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 3.1 Compliance area 3.2 Compliance standard 3.1.8 Periodic monitoring of assessor competence Verbal evidence. Written evidence. 3.3 Rating 3.4 Source(s) of evidence 3.5 Method of collecting evidence 3.6 Comments/Questions/Notes (if non-compliant) Lead assessor Interview Document 3.1.9 Re-testing policy 3.1.10 Condition of test booklets Candidate retesting policy. Re-testing in accordance with the requirement set out in RIS-3751-TOM or other equivalent documented standard. Lead assessor Interview Document Inspection Sample of books are free of defacements and in good condition. Document 3.7 Date Audited How is assessor’s competence monitored over time? Provide documented evidence of competence. How regularly is the monitoring undertaken? Who undertakes it? Outline the re-test policy that you operate for train driver assessment. How do you implement the retest policy? How do you ensure that the assessment cannot be taken more than twice? Inspection Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. Page 34 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 3.1 Compliance area 3.2 Compliance standard 3.1.11 Storage and retention of candidate files RSSB Candidate files securely stored. Candidate files only contain original test material in line with the re-test policy. All ‘passes’ kept until the candidate achieves the age of retirement – stored using any system that allows sight of the assessor’s signature. All documentation relating to a ‘fail’ kept pending the outcome of the next external audit and then destroyed in line with the re-test policy. Candidate files ‘life expired’ contain only relevant documentation ie Candidate Details Questionnaire (CDQ) Overall Marking Sheet (OMS) Candidate Profile Form (CPF). 3.3 Rating 3.4 Source(s) of evidence 3.5 Method of collecting evidence Observation Inspection 3.6 Comments/Questions/Notes (if non-compliant) 3.7 Date Audited Page 35 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 3.1 Compliance area 3.2 Compliance standard 3.1.12 Liaison with relevant manager and feedback on candidate results Candidate Profile Form (CPF) provided to relevant manager. No results advised by any other means before the form is completed and signed. Raw scores and number of questions completed not stated. 3.3 Rating 3.4 Source(s) of evidence 3.5 Method of collecting evidence 3.6 Comments/Questions/Notes (if non-compliant) Lead assessor Interview 3.7 Date Audited What do you do if a manager contacts you and asks for results to be given verbally, before the CPF is completed? What do you do if a manager requests raw scores or the number of questions completed by a candidate? Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. Page 36 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Section 4. Delivery of assessment centre 4.1 Overall delivery 4.1.1 Compliance area 4.1.2 Compliance standard 4.1.1.1 Candidate preparation 4.1.1.2 Completion of test log 4.1.3 Rating 4.1.4 Source(s) of evidence 4.1.5 Method of collecting evidence 4.1.6 Comments/Questions/Notes (if non-compliant) Completion of CDQ. Photographic ID verification. Check RACF database to establish prior assessments. Preparation of individual candidate files (including OMS, CDQ and CPF). Check for special requirements. Treatment of dyslexic candidates according to RSSB guidance2. Lead assessor Document Inspection Interview Correct and complete. Document 4.1.7 Date Audited What would you do if a candidate had not received the practice material at least seven days prior to the assessment taking place? What would you do if a candidate has identified other issues that could affect their performance at the assessment? What do you do if you establish a candidate is not eligible to undertake the assessment? What do you do if a candidate claims they have dyslexia? Inspection Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. 2 Guidance provided within RSSB’s frequently asked questions (FAQ) for the implementation of the new train driver psychometric assessment process, as published on the RSSB website. RSSB Page 37 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.1.1 Compliance area 4.1.2 Compliance standard 4.1.1.3 Introducing the session Introduction covering: 4.1.3 Rating 4.1.4 Source(s) of evidence 4.1.5 Method of collecting evidence Lead assessor Observation 4.1.6 Comments/Questions/Notes (if non-compliant) 4.1.7 Date Audited Personal introductions and welcome. Fire instructions. Tea and coffee facilities. Arrangements for lunch. Smoking regulations. Switching off mobile phones. Location of toilets. Wearing glasses. Factors affecting performance. Programme for the day. Why assessment exercises are used. Test validity. Selection process after the assessment. Any questions. Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. Page 38 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.2 Administration of Group Bourdon test 4.2.1 Compliance area 4.2.2 Compliance standard 4.2.1.1 Test administration Correct materials available. Correct dissemination of materials. Adherence to Test Administration Instructions. Clear diction. Appropriate pacing of delivery. Interpersonal style. Deals with candidates’ questions. Vigilant / walks round room during test. Takes corrective action where appropriate. Accurate timing of session. Concludes test session effectively. 4.2.3 Rating 4.2.4 Source(s) of evidence 4.2.5 Method of collecting evidence 4.2.6 Comments/Questions/Notes (if non-compliant) Assessor Observation 4.2.7 Date Audited What corrective action do you take if a candidate disrupts a testing session? Why do you use the particular action that you have taken? Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. RSSB Page 39 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.2.1 Compliance area 4.2.2 Compliance standard 4.2.1.2 Scoring test 4.2.1.3 Recording test scores 4.2.3 Rating 4.2.4 Source(s) of evidence 4.2.5 Method of collecting evidence 4.2.6 Comments/Questions/Notes (if non-compliant) Scoring is conducted carefully and correctly. Scoring in line with the current Train Driver Assessment Procedures Manual (TDAPM). Scoring is doublechecked for accuracy. Assessor Observation How are errors picked up? What do you do to minimise the number of errors? Scores are transferred to the OMS correctly. Pass / fail is correctly calculated. Pass / fail is correctly entered in the relevant boxes provided on the OMS in line with the TDAPM. Group Bourdon Test Paper is stored within the candidate’s individual file. Assessor How are errors picked up? What do you do to minimise the number of errors? Inspection Observation Inspection 4.2.7 Date Audited Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. Page 40 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.3 Administration of Trainability for Rules and Procedures Test Part One 4.3.1 Compliance area 4.3.2 Compliance standard 4.3.1.1 Test administration Correct materials available including audio equipment. Correct dissemination of materials. Adherence to Test Administration Instructions. Clear diction. Appropriate pacing of delivery. Interpersonal style. Deals with candidates’ questions. Vigilant / walks round room during test. Takes corrective action where appropriate. Accurate timing of session. Concludes test session effectively. 4.3.3 Rating 4.3.4 Source(s) of evidence 4.3.5 Method of collecting evidence 4.3.6 Comments/Questions/Notes (if non-compliant) Assessor Observation 4.3.7 Date Audited What corrective action do you take if a candidate disrupts a testing session? Why do you use the particular action that you have taken? Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with Safety-critical implications. RSSB Page 41 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.3.1 Compliance area 4.3.2 Compliance standard 4.3.1.2 Scoring test 4.3.1.3 Recording test scores 4.3.3 Rating 4.3.4 Source(s) of evidence 4.3.5 Method of collecting evidence 4.3.6 Comments/Questions/Notes (if non-compliant) Scoring is conducted carefully and correctly. Scoring in line with the current Train Driver Assessment Procedures Manual (TDAPM). Scoring is doublechecked for accuracy. Assessor Observation How are errors picked up? What do you do to minimise the number of errors? Scores are transferred to the OMS correctly. Assessment outcome is correctly calculated. Assessment outcome is correctly entered in the relevant boxes provided on the OMS in line with the TDAPM. Answer Sheet is stored within the candidate’s individual file. Assessor How are errors picked up? What do you do to minimise the number of errors? Inspection Observation Inspection 4.3.7 Date Audited Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. Page 42 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.4 Administration of Trainability for Rules and Procedures Test Part Two 4.4.1 Compliance area 4.4.2 Compliance standard 4.4.1.1 Test administration RSSB Correct materials available. Correct dissemination of materials. Adherence to Test Administration Instructions. Clear diction. Appropriate pacing of delivery. Interpersonal style. Deals with candidates’ questions. Vigilant / walks round room during test. Takes corrective action where appropriate. Accurate timing of session. Concludes test session effectively. 4.4.3 Rating 4.4.4 Source(s) of evidence 4.4.5 Method of collecting evidence 4.4.6 Comments/Questions/Notes (if non-compliant) Assessor Observation 4.4.7 Date Audited What corrective action do you take if a candidate disrupts a testing session? Why do you use the particular action that you have taken? Page 43 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.4.1 Compliance area 4.4.2 Compliance standard 4.4.3 Rating 4.4.4 Source(s) of evidence 4.4.5 Method of collecting evidence 4.4.6 Comments/Questions/Notes (if non-compliant) 4.4.7 Date Audited Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. 4.4.1.2 Scoring test Scoring is conducted carefully and correctly. Scoring in line with the current Train Driver Assessment Procedures Manual (TDAPM). Scoring is doublechecked for accuracy. Page 44 of 67 RIS-3751-TOM Issue Three : March 2015 Assessor Observation Inspection RSSB How are errors picked up? What do you do to minimise the number of errors? Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.4.1 Compliance area 4.4.2 Compliance standard 4.4.1.3 Recording test scores Scores are transferred to the OMS correctly. Assessment outcome is correctly calculated. Assessment outcome is correctly entered in the relevant boxes provided on the OMS in line with the TDAPM. Answer Sheet is stored within the candidate’s individual file. 4.4.3 Rating 4.4.4 Source(s) of evidence 4.4.5 Method of collecting evidence 4.4.6 Comments/Questions/Notes (if non-compliant) Assessor Observation Inspection 4.4.7 Date Audited How are errors picked up? What do you do to minimise the number of errors? Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. RSSB Page 45 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.5 Administration of Test of Everyday Attention – Occupational (TEA-Occ) 4.5.1 Compliance area 4.5.2 Compliance standard 4.5.1.1 Test administration Correct materials available. Correct dissemination of materials. Adherence to Test Administration Instructions. Clear diction. Appropriate pacing of delivery. Interpersonal style. Deals with candidates’ questions. Vigilant during test. Takes corrective action where appropriate. Accurate timing of session. Concludes test session effectively. 4.5.3 Rating 4.5.4 Source(s) of evidence 4.5.5 Method of collecting evidence 4.5.6 Comments/Questions/Notes (if non-compliant) Assessor Observation 4.5.7 Date Audited What corrective action do you take if a candidate disrupts a testing session? Why do you use the particular action that you have taken? Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. Page 46 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.5.1 Compliance area 4.5.2 Compliance standard 4.5.1.2 Scoring test 4.5.1.3 Recording test scores 4.5.3 Rating 4.5.4 Source(s) of evidence 4.5.5 Method of collecting evidence 4.5.6 Comments/Questions/Notes (if non-compliant) Scoring is conducted carefully and correctly. Scoring in line with the current Train Driver Assessment Procedures Manual (TDAPM). Scoring is doublechecked for accuracy. Assessor Observation How are errors picked up? What do you do to minimise the number of errors? Scores are transferred to the OMS correctly. Grades are correctly calculated. Assessment outcome is correctly entered in the relevant boxes provided on the OMS in line with the TDAPM. Answer Sheet is stored within the candidate’s individual file. Assessor How are errors picked up? What do you do to minimise the number of errors? Inspection Observation Inspection 4.5.7 Date Audited Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. RSSB Page 47 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.6 Administration of Vienna Test System Tests (WAFV, ATAVT, 2HAND) 4.6.1 Compliance area 4.6.2 Compliance standard 4.6.1.1 Test administration 4.6.1.2 Scoring test 4.6.3 Rating 4.6.4 Source(s) of evidence 4.6.5 Method of collecting evidence 4.6.6 Comments/Questions/Notes (if non-compliant) Adherence to Test Administration Instructions. Effective organisation of the log in process. Clear diction. Appropriate pacing of delivery. Interpersonal style. Deals with candidates’ questions. Vigilant / walks round room during test. Takes corrective action where appropriate. Concludes test session effectively. Assessor Observation Customised Vienna Test System (VTS) report used. Assessor 4.6.7 Date Audited What corrective action do you take if a candidate disrupts a testing session? Why do you use the particular action that you have taken? Observation Inspection Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. Page 48 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.6.1 Compliance area 4.6.2 Compliance standard 4.6.1.3 Recording test scores Scores are transferred to the OMS correctly. Pass / fail is correctly calculated. Pass / fail is correctly entered in the relevant boxes provided on the OMS in line with the TDAPM. Print of the VTS report is stored within the candidate’s individual file. 4.6.3 Rating 4.6.4 Source(s) of evidence 4.6.5 Method of collecting evidence 4.6.6 Comments/Questions/Notes (if non-compliant) Assessor Observation Inspection 4.6.7 Date Audited How are errors picked up? What do you do to minimise the number of errors? Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. RSSB Page 49 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.7 Administration of the Situation Judgement Exercise (SJE) 4.7.1 Compliance area 4.7.2 Compliance standard 4.7.1.1 Test administration 4.7.1.2 Scoring test 4.7.4 Source(s) of evidence 4.7.5 Method of collecting evidence 4.7.6 Comments/Questions/Notes (if non-compliant) Adherence to Test Administration Instructions. Effective organisation of the log in process. Clear diction. Appropriate pacing of delivery. Interpersonal style. Deals with candidates’ questions. Vigilant / walks round room during test. Takes corrective action where appropriate. Concludes test session effectively. No time limit is imposed. Assessor Observation SJE Results and Interview Notes (SJE RIN) document produced for each candidate. Assessor Page 50 of 67 RIS-3751-TOM Issue Three : March 2015 4.7.3 Rating Observation Inspection RSSB What corrective action do you take if a candidate disrupts a testing session? Why do you use the particular action that you have taken? 4.7.7 Date Audited Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.7.1 Compliance area 4.7.2 Compliance standard 4.7.1.3 Recording test scores 4.7.3 Rating Scores are transferred to the OMS correctly. 4.7.4 Source(s) of evidence 4.7.5 Method of collecting evidence 4.7.6 Comments/Questions/Notes (if non-compliant) Assessor Observation Inspection 4.7.7 Date Audited How are errors picked up? What do you do to minimise the number of errors? 4.8 Conducting the Multimodal Interview (MMI) 4.8.1 Compliance area 4.8.2 Compliance standard 4.8.1.1 Administration of the Pre-Interview Form (PIF) PIF administered at the assessment centre. Adequate time allowed to complete PIF. 4.8.3 Rating 4.8.4 Source(s) of evidence 4.8.5 Method of collecting evidence Assessor Observation 4.8.6 Comments/Questions/Notes (if non-compliant) 4.8.7 Date Audited Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. RSSB Page 51 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.8.1 Compliance area 4.8.2 Compliance standard 4.8.1.2 Rating evidence and decision making Evidence marked on the SJE RIN correctly according to the current MMI test manual. Ratings correctly calculated according to the proportion of positive and negative evidence as detailed in the current MMI test manual. Overall score for the behavioural criteria correctly calculated using combination of MMI ratings and SJE scores according to the current MMI test manual. Topic area scores and assessment outcome correctly transferred to the OMS. PIF and SJE RIN stored within the candidate’s individual file. 4.8.3 Rating 4.8.4 Source(s) of evidence 4.8.5 Method of collecting evidence Assessor Observation 4.8.6 Comments/Questions/Notes (if non-compliant) 4.8.7 Date Audited Inspection Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. Page 52 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.9 Feedback to candidates 4.9.1 Compliance area 4.9.2 Compliance standard 4.9.1.1 Offering feedback 4.9.1.2 Candidate feedback Feedback is: 4.9.1.3 Candidate questions All candidates are offered feedback on their assessment results. 4.9.4 Source(s) of evidence 4.9.5 Method of collecting evidence Assessor Observation Assessor Observation Assessor Observation 4.9.6 Comments/Questions/Notes (if non-compliant) 4.9.7 Date Audited Meaningful. Accurate. Fair. In line with the guidance provided in the MMI test manual, and includes assessment outcome. During the feedback: 4.9.3 Rating Candidate is given the opportunity to ask questions. Points are clarified by the assessor. Candidate is invited to comment on the assessment process. Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. RSSB Page 53 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.9.1 Compliance area 4.9.2 Compliance standard 4.9.1.4 Candidates fully informed Assessor explains: 4.9.3 Rating 4.9.4 Source(s) of evidence 4.9.5 Method of collecting evidence Assessor Observation 4.9.6 Comments/Questions/Notes (if non-compliant) 4.9.7 Date Audited How the results will be used. Who will have access to the results. Test validity. Storage of results on National Database. 4.10 Preparing the Candidate Profile Form (CPF) 4.10.1 Compliance area 4.10.2 Compliance standard 4.10.1.1 Making the overall assessment decision Overall assessment decision is made correctly in line the current Train Driver Assessment Procedures Manual (TDAPM). 4.10.3 Rating 4.10.4 Source(s) of evidence 4.10.5 Method of collecting evidence Assessor Document 4.10.6 Comments/Questions/Notes (if non-compliant) 4.10.7 Date Audited Document Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. Page 54 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 4.10.1 Compliance area 4.10.2 Compliance standard 4.10.1.2 Preparation of Candidate Profile Form (CPF) Assessment outcome is entered correctly on the CPF. Name, national insurance number, assessment date and company assessed for are accurately entered. Re-test box ticked ‘Yes’ or ‘No’. Result for each selection criterion correctly entered. Overall Assessment Result is clearly identified on CPF. CPF is signed by the supervising assessor. CPF is signed by the interviewer (if different from the supervising assessor). CPF is checked and verified by a qualified train driver assessor (where assessment provider has more than one qualified train driver assessor). 4.10.3 Rating 4.10.4 Source(s) of evidence 4.10.5 Method of collecting evidence Assessor Document 4.10.6 Comments/Questions/Notes (if non-compliant) 4.10.7 Date Audited Document Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. RSSB Page 55 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Section 5. Historical data 5.1 Compliance area 5.2 Compliance standard 5.1.1 Candidate files audited 5.1.2 Content of candidate files External Audit – a minimum of 24 candidate files to be examined. Internal Audit – a minimum of six candidate files to be examined. For candidates who have completed the full assessment each file should contain all the original assessment centre documentation, that is: 5.3 Rating 5.4 Source(s) of evidence 5.5 Method of collecting evidence Document Inspection Document Inspection 5.6 Comments/Questions/Notes (if non-compliant) 5.7 Date Audited CDQ. OMS. CPF (where appropriate). Group Bourdon, TRP1, TRP 2 and TEA-Occ Answer Sheets. VTS report. PIF. Completed SJE RIN. CPF should be clearly endorsed for re-test. Re-test documents transferred into current candidate file. Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with Safety-critical implications. Page 56 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 5.1 Compliance area 5.2 Compliance standard 5.1.3 Candidate files accuracy of scoring and recording of Group Bourdon test 5.1.4 Candidate files accuracy of scoring and recording of TRP Part One test 5.3 Rating 5.4 Source(s) of evidence 5.5 Method of collecting evidence Scoring in line with the current Train Driver Assessment Procedures Manual (TDAPM). Errors account for no more than one in six test papers. Scores shown on the OMS are accurately recorded. Assessment outcome on the CPF accurately reflects that of the OMS. Document Inspection Scoring in line with the current Train Driver Assessment Procedures Manual (TDAPM). Errors account for no more than one in six test papers. Scores shown on the OMS are accurately recorded. Assessment outcome on the CPF accurately reflects that of the OMS. Document Inspection 5.6 Comments/Questions/Notes (if non-compliant) 5.7 Date Audited Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. RSSB Page 57 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 5.1 Compliance area 5.2 Compliance standard 5.1.5 Candidate files accuracy of scoring and recording of TRP Part Two test 5.1.6 Candidate files accuracy of scoring and recording of TEAOcc test 5.3 Rating 5.4 Source(s) of evidence 5.5 Method of collecting evidence Scoring in line with the current Train Driver Assessment Procedures Manual (TDAPM). Errors account for no more than one in six test papers. Scores shown on the OMS are accurately recorded. Assessment outcome on the CPF accurately reflects that of the OMS. Document Inspection Scoring in line with the current Train Driver Assessment Procedures Manual (TDAPM). Errors account for no more than one in six test papers. Scores shown on the OMS are accurately recorded. Assessment outcome on the CPF accurately reflects that of the OMS. Document Inspection 5.6 Comments/Questions/Notes (if non-compliant) 5.7 Date Audited Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. Page 58 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 5.1 Compliance area 5.2 Compliance standard 5.1.7 Candidate files accuracy of scoring and recording of VTS tests 5.1.8 Candidate files – recording and scoring of SJE and MMI 5.1.9 Candidate files – overall assessment rating 5.3 Rating 5.4 Source(s) of evidence 5.5 Method of collecting evidence Scoring is done using customised report function of VTS. Scores shown on the OMS are accurately recorded. Assessment outcomes on the CPF accurately reflect that of the OMS. Document Inspection Scoring in line with the current MMI and SJE Test Manual. Errors account for no more than one in six test papers. Scores shown on the OMS are accurately recorded. Assessment outcomes on the CPF accurately reflect that of the OMS. Document Inspection No evidence of a candidate ‘failing’ an assessment but receiving a ‘pass’ at the assessment centre. Document Inspection 5.6 Comments/Questions/Notes (if non-compliant) 5.7 Date Audited Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. RSSB Page 59 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection 5.1 Compliance area 5.2 Compliance standard 5.1.10 Candidate files – National Database entry Candidate information entered accurately onto National Database. All biographical data recorded on the CDQ entered onto the National Database. Data entered within seven working days of the assessment. Errors account for no more than two of the candidates entered onto the National Database, within an external audit. Errors account for no more than one of the candidates entered onto the National Database, within an internal audit. 5.3 Rating 5.4 Source(s) of evidence 5.5 Method of collecting evidence Document Inspection 5.6 Comments/Questions/Notes (if non-compliant) 5.7 Date Audited Rating Key: A - Meets requirements. B - Minor non-compliance with specification. C - Serious non-compliance with specification. D - Serious non-compliance with specification with safety-critical implications. Page 60 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Section 6. Assessor competence recording sheet The auditor should use this sheet to record the following. 6.1 MMI Certification 6.1.1 Name of assessor(s) carrying out MMI 6.1.2 Expiry date of MMI Certificate 6.2 Test User: Occupational Ability Qualification 6.2.1 Name of assessor(s) 6.2.2 Date of qualification 6.3 ORR register of psychologists 6.3.1 Name of assessor(s) RSSB 6.3.2 Is the assessor on the ORR register for psychologists? (circle as appropriate) Yes No Yes No 6.3.3 Date of registration Page 61 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Section 7. Historical data check sheet The auditor should use this sheet to record the details of the candidate records checked as part of the audit. 7.1 General information 7.1.1 Date of history check 7.1.2 Name of auditor(s) conducting check 7.2 Historical data checked 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Page 62 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB National Database entry Candidate profile OMS SJE and MMI PIF 7.2.6 Comments VTS TEA-Occ 7.2.5 Item checked (tick) TRP2 7.2.4 Database input date TRP1 7.2.3 Assessment date Group Bourdon 7.2.2 Candidate name CDQ 7.2.1 No Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection National Database entry Candidate profile OMS SJE and MMI PIF 7.2.6 Comments VTS TEA-Occ 7.2.5 Item checked (tick) TRP2 7.2.4 Database input date TRP1 7.2.3 Assessment date Group Bourdon 7.2.2 Candidate name CDQ 7.2.1 No 16 17 18 19 20 21 22 23 24 RSSB Page 63 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Appendix E Recommended qualifications, competences and experience for internal and external auditors E.1.1 Internal and external auditors require competence of the psychometric assessment process and of auditing. Railway undertakings are to satisfy themselves that auditors are competent for the tasks they undertake. This may be achieved through obtaining auditing formal qualifications or by any other means deemed appropriate. E.1.2 Recommended specific qualifications, competences and experience for internal and external auditors of the train driver psychometric assessment process are: ‘Test User: Occupational, Ability’ British Psychological Society (BPS) qualification. Registration on the British Psychological Society’s Register of Qualified Test Users. Registration on the Office of Rail Regulation (ORR) list of registered psychologists. Certificate to conduct the Multi-Modal Interview. Full delivery of the psychometric assessment process specified in this document and knowledge of the risks this process is intended to control. Two years or more experience conducting train driver psychometric assessment, of which at least six months is current at the time of the audit. Knowledge of the audit protocol specified in this document. Knowledge and application of good practice in audit planning, interviewing and reviewing, documentation review and observation, written and verbal communications. Page 64 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Definitions 2HAND Two Hand Coordination Test A computerised psychological assessment tool designed to measure hand coordination. Assessment Centre An organisation or part of an organisation that provides train driver psychometric assessment services to railway undertakings. Assessment centres can be within or external to a railway undertaking. Assessment method A standardised way of measuring one or more of the train driver selection criteria. Assessor Any person competent to administer the train driver psychometric assessment process. ATAVT Adaptive Tachistoscopic Traffic Test An adaptive computerised psychological assessment tool designed to measure perception. Behavioural aptitudes selection criteria The selection criteria that relate to the behavioural tendencies of the candidate as relevant to safe train driving. Candidate A person being considered by a railway undertaking for a role as a train driver. Candidate Details Questionnaire (CDQ) Document to record a set of pre-defined candidate details relevant to the train driver psychometric assessment process, as outlined in this document. Candidate Profile Form (CPF) Document to record, for each candidate, the assessment results of the train driver psychometric assessment process, as outlined in this document. Driver Any person competent and authorised to drive trains. Fairness A psychometric assessment method is considered to be fair if it does not unjustifiably discriminate against any respondent groups based on any protected characteristic as defined by the Equality Act 2010. Group Bourdon A paper-based psychometric test designed to measure attention. Multi-Modal Interview (MMI) Interview process used in conjunction with the situational judgement exercise to assess the behavioural aptitudes selection criteria. Occupational testing The use of psychometric assessment methods as part of a selection process for a job. Overall Marking Sheet (OMS) Document to record a candidates scores for each assessment method undertaken as part of the train driver psychometric assessment process, as outlined in this document. Pre Interview Form (PIF) A paper based form, completed by the candidate, to gather relevant information to inform the focus of questions asked in the Multi-Modal Interview (MMI). RSSB Page 65 of 67 RIS-3751-TOM Issue Three : March 2015 Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection Psychometric assessment process The standardised assessment of safety-related cognitive, psychomotor and behavioural aptitudes for candidate train drivers. The process consists of standard selection criteria and assessment methods as set out in Appendix A. Reliability The extent to which a psychometric assessment method produces consistent results under similar conditions. Safety-critical work This term is used as defined in The Railways and Other Guided Transport Systems (Safety) Regulations 2006 [ROGS] (as amended). Selection criteria Aptitudes or skills that candidates must possess to a sufficient level in order to be considered as potential train drivers. Selection process The entire recruitment process including: advertising the role and attracting candidates, the application process, sifting applications, the standardised psychometric assessment process, any additional psychometric assessments, any additional interviews or assessments and the final recruitment decision. Situational Judgement Exercise (SJE) A computerised measure of behavioural aptitudes for train driving. It is designed to be used with the Multi-Modal Interview. Test of Everyday Attention – Occupational (TEA-Occ) A paper-based psychometric assessment designed to measure attention. Train This term includes an on-track machine being driven outside a possession. Trainability for Rules and Procedures Test (TRP) A paper-based psychometric assessment designed to measure memory and reasoning. Train Driver Assessment Procedures Manual Document containing detailed instructions about how to administer the psychometric assessment process. Validity The extent to which a psychometric assessment method measures what it is intended to measure. WAFV Vigilance Test A computerised psychological assessment tool designed to measure vigilance. Written Communication Test A paper-based psychometric assessment designed to measure written communication skills. Page 66 of 67 RIS-3751-TOM Issue Three : March 2015 RSSB Uncontrolled When Printed Document comes into force 06/06/2015 Supersedes RIS-3751-TOM Iss 2 as of 06/06/2015 Rail Industry Standard for Train Driver Selection References The Catalogue of Railway Group Standards give the current issue number and status of documents published by RSSB. This information is also available from www.rgsonline.co.uk. RGSC 01 RGSC 02 Railway Group Standards Code The Standards Manual Documents referenced in the text Railway Group Standards GO/RT3119 GO/RT3451 Accident and Incident Investigation Train Drivers – Suitability and Medical Fitness Requirements Other References BS ISO 10667-1:2011 Assessment service delivery — Procedures and methods to assess people in work and organizational settings. Part 1: Requirements for the client BS ISO 10667-2:2011 Assessment service delivery — Procedures and methods to assess people in work and organizational settings. Part 2: Requirements for service providers ORR (2014) Guide to the Train Driving Licences and Certificates Regulations – includes the referenced Annex B5 RSSB (2006) Psychometric testing - a review of the train driver selection process: Recommendations for standardising and improving the selection process (T340 Report) RSSB (2010) Driver Selection: Development Phase - Updated selection criteria and validation study (T628 Report) RSSB (2012) Driver selection: Implementation phase (T948) TDLCR (2010) Train Driving Licences and Certificates Regulations Other relevant documents RSSB documents GO/RT3452 Train Movement – Medical Fitness Requirements Other references Equality Act (2010) The Railways and Other Guided Transport Systems (Safety) Regulations 2006 [ROGS] (as amended) RSSB Page 67 of 67 RIS-3751-TOM Issue Three : March 2015