Research Brief Non-technical skills for rail: development, piloting, and evaluation of a training course T869 - June 2012 - updated August 2012 Introduction Non-technical skills (NTS) are generic skills that underpin and enhance technical tasks and improve safety by helping people to anticipate, identify and mitigate against errors. NTS are more general than technical skills and can be applied to a range of tasks and procedures. Examples of non-technical skills are conscientiousness, communication, rule compliance and workload management. Reviews of incidents and accidents in the industry have consistently shown that NTS are key contributors to these events. While behavioural preferences form part of the train driver selection criteria (as part of RSSB projects T628 and T948, new assessment techniques have been designed to better measure behavioural preferences in candidates applying for the train driver role), at the start of this project there was very little formal coverage of NTS in other areas of the driver competence management system. Until very recently, training programmes for operational staff within the GB rail industry have been based largely on rules and traction training, and ongoing competence development has been concerned only with technical skills. Crew Resource Management, a form of NTS training, was introduced in the aviation industry in the 1970s, and it has since spread to various other safety critical industries such as nuclear, healthcare, and shipping. In more recent years, this form of NTS training has been introduced to rail industries in Australia, USA and Canada. Evaluations of this training suggest that it can reduce safety occurrences attributable to human error. RSSB project T869 was commissioned to produce and pilot a NTS training course and other reinforcement activities for drivers which would also be relevant to other safety critical roles in the industry. This work builds upon previous RSSB work in this area. RSSB R&D Programme Block 2 Angel Square 1 Torrens Street London EC1V 1NY enquirydesk@rssb.co.uk www.rssb.co.uk/research/pages/ default.aspx 1 Non-technical skills for rail: development, piloting, and evaluation of a training course T869 - June 2012 - updated August 2012 Aims The aim of this project was to design, produce and pilot a suite of materials for training and reinforcing NTS. Long-term, the vision is for this suite of materials to be adopted and adapted as appropriate by other companies, and for other operational roles, integrating into existing technical training programmes as necessary. In delivering T869, RSSB sought to meet the following objectives: Phase 1 Identify suitable companies to lead in the development and piloting of the training. Identify the NTS relevant to the train driver role. Conduct a brief training needs analysis to identify gaps in current NTS training provision. Phase 2 Develop a stand-alone NTS training course for front line staff and their managers. Develop activities to reinforce the development of NTS, including the provision of feedback on skills development. Phase 3 Pilot the training course with two train operating companies. Evaluate the training. Phase 4 Create a suite of materials and guidance (including guidance on how to integrate with technical training and other competence management activities). Method and methodology Throughout the T869 project, work has been carried out in line with good research, development and evaluation practices. As appropriate, the methods have been guided by lessons learnt by rail companies in other countries and by other safety-critical industries in integrating NTS in competence management systems. The methods used to identify and confirm which NTS are important in the role of the train driver, and develop behavioural markers, are detailed in the first deliverable. The methods used to develop training and reinforcement activities, and to evaluate the training, are provided in the second deliverable (the final project report). 2 RSSB Findings The courses were positively received by participants. Ratings and comments on the course suggested that the training was perceived to be useful, and provided tools to enhance competence and safety within the industry. Evaluation of the pilot training courses for drivers and their managers (based on a sample of 29 drivers and 12 managers) demonstrated significant improvements in non-technical skills (based on self-reports and manager ratings of drivers on the job). Significant improvements were demonstrated across managerial skills (as perceived by managers themselves), specifically the understanding of NTS and relevance to the driver role, and knowledge of what should be documented and why. Improvements were also seen in attitudes to safety, and in safety culture ratings, although these did not reach significance. Over the course of the project, an increasing number of companies have begun to formally consider NTS and explore how NTS can be integrated into training and competence measurement. Deliverables Deliverable 1: Non-technical skills for rail - a list of skills and behavioural markers, for train drivers, with guidance notes (updated 2012) This report includes: A general introduction to NTS. An overview of the methods used to identify NTS and map them to the driver role. A validated list of NTS applicable to the train driver role. Behavioural markers (examples of good and poor behaviour), definitions and typical driver tasks for each NTS. The NTS and behavioural markers in the first deliverable formed the focus of subsequent training and development activities in the rest of the T869 project. Deliverable 2: Non-technical skills for rail - developing an integrated approach to training and investment This report has been written for those responsible for setting company policy with regard to competence management, and those responsible for planning and implementing changes to the competence management system (CMS). It is designed to assist RSSB 3 Non-technical skills for rail: development, piloting, and evaluation of a training course T869 - June 2012 - updated August 2012 in decisions about how NTS can be effectively integrated within the company. The report includes: A case for investment in NTS integration. Recommendations on how to go about integrating NTS into all areas of the company (including who should receive the training, who should train, how NTS can be integrated into competence management systems, potential challenges, and recommendations for ongoing evaluation). Further information on the T869 project methodology (including evaluation results from the pilot courses). Frequently asked questions. Deliverable 3: NTS course materials and corresponding tools for managers These materials are available to RSSB member companies only via the SPARK web portal. Benefits/ Implementation The T869 project provides industry with: A clear description of the generic skills required of a train driver to help him or her anticipate, identify and mitigate against threats and errors as part of the train driver role. The behavioural markers provided with each skill have been specifically developed to be observable, specific and free from jargon. Where the markers are difficult to observe, the training tools provide managers with questions to ask to determine whether a marker has been evidenced. The typical driver tasks listed by each skill will help to illustrate how NTS underpin technical tasks. This will enable drivers to better understand what is expected of them, and people involved in the competence management of drivers (eg driver managers, instructors and trainers) to measure NTS in an objective way and provide meaningful feedback. A core set of NTS training materials The core content of the course is applicable to other safety critical roles, and RSSB members can customise the training to suit their own companies and operational roles. The materials provide a consistent basis for NTS training and reinforcement across the industry. A method for identifying NTS, mapping on to operational roles, developing and evaluating training and integrating NTS into competence management systems 4 RSSB The principles of this work, and the methods used, can also be applied to other operational roles. Research has shown that the NTS required of many operational roles are very similar. Evaluation information and lessons learned The training course has been evaluated to assess whether it results in changed behaviour. Over time, it will be possible for companies to monitor whether this corresponds with a reduction in incidents and accidents. Advice and guidance on how to best integrate NTS training and reinforcement activities has been provided. Positive safety benefits Over time, it is expected that if NTS are trained and reinforced and appropriately integrated into competence management systems, that this will result in safer behaviour and subsequently, a reduction in incidents and accidents. In order to maximise the benefits from this research, individual companies would need to: Raise awareness and understanding of the relevance of NTS at all levels of the company. Build manager, instructor and assessor competence in NTS; how to objectively measure NTS and provide meaningful feedback. Customise the training materials to enhance the relevance to their company and staff. Commit to ongoing development and reinforcement of staff NTS. Use NTS and behavioural markers for development purposes only, and not use them to 'assess' drivers in the sense of passing or failing them. RSSB 5 Non-technical skills for rail: development, piloting, and evaluation of a training course T869 - June 2012 - updated August 2012 Next steps A launch event for this research was held on 21 June 2012 at which delegates were requested to indicate their level of 'NTS maturity' on a five-point scale. All 13 companies who provided a response indicated that there was interest in NTS, but implementation ranged from none (8%, due to limited resources), to smaller scale implementation with a small group of staff (46%) through to full implementation across the front line staff population (46%, including Arriva Trains Wales and members of the First Group). No respondents considered NTS yet to be integrated into the competence management system, fully understood by managers and all front-line staff and constantly developed. Responses indicated that companies are making plans to further integrate NTS for other roles and to fully incorporate NTS into competency investigation and recruitment. The Rail Industry Skills Forum, which sponsored this work, has recommended this important research to railway companies in Great Britain. Following the launch event, ORR said 'Non-technical skills underpin every aspect of professional behaviour, and the most progressive industries worldwide are increasingly recognising the benefits of improving their staff's competence in this area. Continuous improvement in NTS will be vital in ensuring the rail industry makes best use of its most valuable asset - its people - in operating a safe and efficient mode of transport. The industry, the travelling public and staff themselves can all benefit from the kind of approach advocated in this work, and the Office of Rail Regulation strongly supports the efforts being made by RSSB and the rail industry to improve the competence of rail staff in non-technical skills.' Contact For more information please contact: Michael Woods Head of Operations and Management Research R&D Programme, RSSB enquirydesk@rssb.co.uk 6 RSSB