Cross-National Perspectives on the Practical Application of Structured Violence Risk Assessment Tools Abstract Mental health professionals are routinely called upon to assess the violence risk presented by their adult patients, frequently aided by structured risk assessment tools. Though surveys of instrument use and perceived utility have been conducted, these efforts have been largely circumscribed to individual countries and have not compared the viewpoints of psychologists, psychiatrists, and nurses. The current proposal aims to address these gaps by developing a Web-based survey that will be translated into 7 languages and distributed via e-mail to members of professional organizations in 16 countries. The survey will compare the use of violence risk assessment tools across nations and compare the perceived role of such instruments in risk assessment and management across professional groups. The project represents the first multilingual survey of international violence risk assessment practices and perspectives, and will provide the first data on the prevalence of specific tool use in a number of countries. Singh Grant Proposal 1 Statement of the Problem Prior surveys of risk assessment tool use and perceived utility have been circumscribed to a small set of countries, namely the United States and the United Kingdom, and have not compared findings across different groups of mental health professionals. Consequently, many questions remain regarding the application of these instruments in practice. Specifically, what structured violence risk assessment tools are most frequently used, how they are being used, and what are their perceived roles in the assessment and management of future risk for violence? The proposed project seeks to answer these questions by conducting a multilingual Web-based survey to compare violence risk assessment practices and attitudes of psychologists, psychiatrists, and nurses in 16 countries regarding the application of structured violence risk assessment tools. Relation of the Problem to the State of the Field Numerous violence risk assessment instruments, composed of static, dynamic, risk and/or protective factors combined using either actuarial formulae or professional judgment, have been introduced in recent years (Heilbrun, 2009; Otto & Douglas, 2010). Such structured instruments have been implemented in psychiatric and correctional settings worldwide, where they are used to inform medico-legal decisions that have significant implications for individual liberty and public protection (e.g., involuntary hospitalization, length of hospital detention for treatment, discharge from psychiatric hospitals and release from prisons). As there are a number of structured violence risk assessment tools currently available and recent meta-analytic evidence suggests that they are relatively interchangeable in terms of predictive validity (Yang, Wong, & Coid, 2011), clinicians are faced with the challenge of selecting the instrument that they perceive to be the best fit for their population and that will best guide treatment planning. Knowledge of which tools are currently being used in practice and which tools colleagues working in similar settings believe have the greatest utility in risk assessment and management may inform this decision. Survey methodology represents one approach that can be used to provide clinicians with such information. Further, surveys can compare clinical practice to best practice recommendations and clarify reasons for any discrepancies. For example, surveying clinicians could elucidate why, despite a large evidence base concerning the superior predictive validity of structured approaches (Dvoskin, 2002; Nicholls, Ogloff, & Ledwidge, 2007; Quinsey, Harris, Rice, & Cormier, 2006), unstructured assessments of violence risk remain common in practice (Viljoen, McLachlan, & Vincent, 2010). According to a search of the PsycINFO, EMBASE, and MEDLINE databases, five surveys have been published investigating violence risk assessment practices. First, Tolman and Mullendore (2003) conducted a postal survey to identify instruments used by psychologists in violence risk evaluations of adults for United States courts. Respondents were 93 clinicians licensed in Michigan and 71 diplomates of the American Board of Forensic Psychology (ABFP). The researchers found significant differences in specific tool use, with the Violence Risk Appraisal Guide (VRAG; Harris, Rice, & Quinsey, 1993) being the most commonly used risk instrument amongst the clinicians and the Psychopathy Checklist-Revised (PCL-R; Hare, 2003) amongst the diplomates. No measures of perceived utility were included. Second, Higgins and colleagues (2005) surveyed violence risk assessment practices in mental health trusts across England. General adult consultants from 45 randomly-selected trusts responded to a postal survey and reported that although some form of structured instrument was used in 67% of trusts, there was wide variation in the specific instruments that had been implemented (details not reported). Though perceived utility was not examined, the authors suggested that future research should investigate the perspectives of administering clinicians on structured tools’ roles in assessment and management. Singh Grant Proposal 2 Third, Archer and colleagues (2006) used a Web-based survey to examine the use of psychological tests (including those used to assess violence risk in adults) among 152 clinical psychologists recruited through the American Psychology-Law Society (AP-LS) LISTSERV and the online ABFP diplomate directory. The researchers concluded that violence risk assessment instruments such as the VRAG, Historical, Clinical, Risk Management-20 (HCR-20; Webster, Douglas, Eaves, & Hart, 1997), and the Level of Service Inventory-Revised (LSI-R; Andrews & Bonta, 1995) were used less frequently than were psychopathic personality scales such as the PCL-R and the Psychopathy Checklist: Screening version (PCL:SV; Hart, Cox, & Hare, 1995). Consistent with previous surveys, no measures of perceived utility were included. Fourth, Khiroya, Weaver, and Maden (2009) surveyed violence risk assessment tool use in adult medium secure units throughout the United Kingdom. The 29 clinical service directors who responded to a brief postal questionnaire reported that a variety of violence risk assessment tools had been implemented (most commonly the PCL-R/PCL:SV and the HCR-20) and that most units routinely used more than one tool. Though they did examine perceived utility, the authors measured this using a single item that did not specify utility in assessment and/or management. Finally, Viljoen and colleagues (2010) conducted a Web-based survey of psychologists to examine use of violence risk assessment tools in forensic evaluations of adults and juveniles. The survey was distributed to members of several North American professional organizations as well as the International Association of Forensic Mental Health Services (IAFMHS). Based on data from 199 respondents, the authors found that clinicians routinely used structured instruments (most commonly a PCL measure or the HCR-20 for adults and intelligence scales for juveniles), though differences in tool use between countries were not examined. In terms of perceived utility, respondents were asked whether they preferred actuarial tools, SPJ tools, or both; however, they were not asked the reason(s) for their preference. These prior studies have advanced our understanding of the use of violence risk assessment tools in practice, but also share important limitations. First, no surveys have been published comparing what instruments are used in routine practice in different countries. Second, previous surveys have not compared patterns of tool use and perceived utility across professional groups. Third, previous surveys have not attempted to disentangle risk assessment and management practices. To address these limitations, the proposed project will survey the frequency of use and perceived role of violence risk assessment tools in assessing and managing risk among psychologists, psychiatrists, and nurses in 16 countries. Project Method Participants Participants in the proposed study will include psychologists, psychiatrists, and nurses in 16 countries: the United States, Argentina, Australia, Belgium, Canada, Chile, Denmark, Germany, Hong Kong, Mexico, The Netherlands, New Zealand, Spain, Sweden, Switzerland, and the UK. For each country, an expert in risk assessment has been recruited to assist in the translation and distribution of the survey. Participants will be included if they are between 18 to 65 years old and have conducted at least one assessment of violence risk for an adult. Participants will be excluded if they are not between 18 to 65 years old and/or have not conducted at least one assessment of violence risk for an adult. Materials Survey. A survey including closed-ended questions based on a review of the violence risk assessment literature and previous surveys concerning forensic assessment (e.g,. Borum & Grisso, 1995; Jackson & Hess, 2007; Ryba, Cooper, & Zapf, 2003) has been developed. The Singh Grant Proposal 3 survey has been designed to be between 20-25 minutes in length (cf. Viljoen et al., 2010). Items are organized into seven blocks (with a general introduction and debriefing), including: (1) respondent characteristics, (2) lifetime risk assessment procedures, (3) current risk assessment procedures, (4) perceptions of actuarial and structured professional judgment instruments, (5) legal defensiveness, (6) openness to evidence-based practice, and (7) the measurement of predictive validity. In terms of benefits, respondents who complete the survey will have the opportunity to provide their e-mail addresses to be entered into a drawing for eight cash prizes, each valued at $50 USD. Respondents who wish to enter this drawing will do so by selecting a hyperlink that will redirect them to a Qualtrics page that will allow them to enter this information and have it stored independently of their survey responses, thus preserving confidentiality. There are no foreseen risks or other benefits for survey respondents. Participation letters. Three letters to invite participation have also been developed. These letters explain the nature and purpose of the survey, apprise respondents that by electing to participate they are giving their informed consent, and delineate the drawing incentive (see Procedure section for details on these letters). Procedures The proposed study will be conducted in four phases: (1) survey development, (2) translation, (3) distribution, and (4) data analysis and manuscript preparation. Phase 1: Survey development. In Phase 1, the Web-based survey has been developed to query the use of specific violence risk assessment tools in practice and the perceived utility of those instruments in predicting future violence and making risk management decisions. Webbased surveys are an appropriate alternative to paper-based approaches when sampling mental health professionals because of their generally high levels of Internet access (Solomon, 2001). The survey has been constructed and will be administered using Qualtrics, an electronic survey software. Qualtrics has been used in recent surveys of forensic mental health workers (e.g., Kimonis Fanniff, Borum, & Elliott, 2011; Williams, 2011) and has a number of benefits, including data collection through a secure server, use of logic statements to guide respondents through the survey, libraries of customizable question templates, and a continuous file saving function to minimize data loss due to browser crashes. Qualtrics is licensed and supported at the University of South Florida. A review of the risk assessment literature and previous surveys of tool use was used to compile a list of items for the survey. The items as well as the participation letters detailing the purpose and nature of the survey were screened by my study team members in the Department of Mental Health and Law: Drs. Sarah Desmarais, Randy Otto, Paul Stiles, and Richard Van Dorn. In addition to this role, Drs. Desmarais and Van Dorn will assist in addressing regulatory issues and in conducting analyses. Drs. Stiles and Otto will take on supervisory roles and assist with data interpretation and ultimate publication. The primary investigator (Jay Singh) will be responsible for coordinating these roles, communicating with the IRB, and data management. Phase 2: Translation. In Phase 2, the survey and participation letters will be professionally translated into seven languages, including Danish, Dutch, French, German, Spanish (Latin American), Spanish (Spain), and Swedish. Translated materials will be sent to the international collaborators for proofreading to ensure accuracy of the translations in terms of both content and meaning. The collaborators also will make country-specific modifications to survey content, as necessary. For example, changes to response options for items concerning professional degrees and types of clinical evaluations conducted will be made where appropriate. Phase 3: Distribution. In Phase 3, the participation letters and survey will be distributed either electronically using e-mail addresses contained in either LISTSERVs or online Singh Grant Proposal 4 membership directories of national and international professional organizations (with the permission of the relevant list or website administrators) or through links on the websites of professional organizations. Alternatively, professional organizations may wish to notify their membership via a newsletter advertisement. Invitations letter and surveys will be distributed by each participating country’s collaborator in the native language to the membership of three classes of national organization: (1) a national organization of forensic psychologists, (2) a national organization of forensic psychiatrists, and (3) a national organization of forensic psychiatric nurses. Organizations from which permissions will be requested include: the Argentinean Association of Behavioral Science, the American Academy of Psychiatry and the Law, the American Board of Forensic Psychology, the American Psychology-Law Society, the Association of German Professional Psychologists, the American Psychiatric Nurses Association, Forensic Behavioral Services Incorporated, the Australian Psychological Society, the Belgian Federation of Psychologists, the British Psychological Society, the Canadian Psychological Association, the Dutch Association of Psychiatrists, Dutch Institute of Psychologists (Forensic Section), Verpleegkundigen and Verzorgenden Nederland, the Forensic Psychiatric Nurses Council, the German Psychological Association, the Mexican Psychological Association, the Netherlands Institute of Psychologists, the New Zealand Psychological Society, the Royal College of Forensic Psychiatrists, the Spanish Federation of Psychological Associations, the Spanish Psychological Association, the Swedish Psychological Association, and the UK Nursing and Midwifery Council. In addition, the English-language version of the invitation letter with links to all translations of the survey will be distributed to the membership of the following international organizations related to forensic mental health: International Association of Forensic Mental Health Services, International Association for the Treatment of Sexual Offenders, International Association of Forensic Nurses. The primary investigator (JPS) will have responsibility for recruitment in the US and to international organizations. A list of all organizations contacted will be maintained and provided to collaborators to prevent crosspostings. To the extent possible, survey distribution will follow the widely-accepted Dillman method (Dillman, Smyth, & Christian, 2009). In concordance with this approach, initial participation letters will be sent out via e-mail on a Friday and will contain direct and active links to the survey. Two reminder e-mails will be sent in seven day increments after the initial distribution to remind potential research participants about the study. A fourth and final e-mail will include the participation letter and relevant hyperlink, indicating that it is the final opportunity to participate. Based on previous Web-based surveys of violence risk assessment practices that have used similar methodology (Archer et al., 2006; Viljoen et al., 2010), a response rate between 45% and 65% is expected, comparable to electronic surveys in other fields (Baruch, 1999). Data will be monitored as it is entered into Qualtrics by respondents. Phase 4: Data analysis and manuscript preparation. In Phase 4, respondent data will be exported from Qualtrics to STATA 10.1 (StataCorp, 2007) for analysis. Data will be kept on a password protected desktop computer in a locked office in the Department of Mental Health Law and Policy. Frequency distributions will be examined and measures of central tendency and dispersion will be calculated for all variables. Differences in ratings of perceived utility between individual countries will be explored descriptively, and, for omnibus testing, countries will be collapsed into three groups (North America, UK, and other).1 For survey items with dichotomous responses, differences between psychologists, psychiatrists, and nurses across countries will be assessed using Kruskal-Wallis ANOVAs with post-hoc Pearson χ2 tests. For survey items with 1 These groups were chosen to allow me to examine whether the findings of previous surveys that have focused on tool use in North America and the UK are generalizable to other countries. Singh Grant Proposal 5 continuous responses, differences between professional groups and across countries will be analyzed using parametric ANOVAs with post-hoc Bonferonni tests. It is anticipated given the number of organizations being surveyed in the 16 countries that the sample size will be approximately 10,000 participants. Power analyses using G*Power 3 (Faul et al., 2007) suggest that 159 respondents (n = 53 per subgroup) need to be recruited to detect moderate effect size (F = 0.25) differences across professions and geographic location. To have adequate power to examine profession by geographic location interactions, 42 psychologists, psychiatrists, and nurses need to be recruited in North America, the UK, and in the other participating countries. Thus, we anticipate that there will be more than enough participants to conduct the intended analyses. In all analyses, covariates (e.g., sex, age, race/ethnicity) will be assessed using standard procedures and controlled for as necessary. When analysis has been completed, a manuscript will be prepared for submission to a peerreviewed journal. Results also will be used to support a federal grant application (NIMH K99/R00) focused on improving the efficiency of violence risk assessment through the application of innovative technologies. Five years after the final report on the data has been submitted for publication, the data will be destroyed by deleting the survey on Qualtrics (removing all associated data) and all associated STATA files will be placed into the Microsoft Windows electronic “Recycle Bin” and the “Empty the Recycle Bin” function will be used. Anticipated Contribution Despite the proliferation of violence risk assessment tools in criminal justice and mental health systems, research on how these tools are actually being used in practice has remained rare (Elbogen, Huss, Tomkins, & Scalora, 2005). Work comparing risk assessment practices in different countries and professional groups has been particularly scarce. Therefore, the proposed study, with its focus on the use and perceived utility of violence risk assessment tools in practice by psychologists, psychiatrists, and nurses in 16 countries, represents an original contribution that may inform research, clinical practice, and public policy (and, therefore, has the potential to benefit society). Specifically, the survey will speak to whether recent research findings such as the interchangeability of violence risk assessment tools in terms of predictive validity (Yang et al., 2010), the instability of risk estimates produced by actuarial instruments when applied to individuals (Hart, Cooke, & Michie, 2007), and the importance of clinical training in the administration of structured assessment instruments (McNiel et al., 2008) are reflected in the practice and attitudes of psychologists, psychiatrists, and nurses. The survey also may influence practitioners’ selection of which risk assessment tools to implement, as it will identify which instruments are perceived to be the most useful in assessing the likelihood of future violence and which in identifying potential treatment targets to reduce violence risk. Further, survey results will clarify which risk assessment tools are being used to meet current guidelines by professional organizations of psychologists (e.g., American Psychological Association, 2006), psychiatrists (e.g., American Psychiatric Association, 2004; National Institute for Clinical Excellence, 2009), and nurses (e.g., Nursing and Midwifery Council, 2004) which recommend that violence risk be assessed using evidence-based methods. Finally, the survey may inform public policy by identifying countries where the use of empirically-validated violence risk assessment tools is not common despite the large evidence base demonstrating the superiority of structured methods over unstructured clinical judgment. Singh Grant Proposal 6 References American Psychiatric Association. (2004). Practice guidelines for the treatment of patients with schizophrenia. Arlington, VA: American Psychiatric Association. American Psychological Association Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61, 271-285. Andrews, D. A., & Bonta, J. (1995). LSI-R: The Level of Service Inventory-Revised. Toronto, ON: Multi-Health Systems. Archer, R. P., Buffington-Vollum, J. K., Stredny, R. V., & Handel, R. W. (2006). A survey of psychological test use patterns among forensic psychologists. Journal of Personality Assessment, 87, 84-94. Baruch, Y. (1999). Response rates in academic studies: A comparative analysis. Human Relations, 52, 421-434. Borum, R., & Grisso, T. (1995). Psychological test use in criminal forensic evaluations. Professional Psychology: Research & Practice, 26, 465-473. Dillman, D., Smyth, J., & Christian, L. (2009). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). Hoboken, NJ: John Wiley & Sons, Inc. Dvoskin, J. (2002). Knowledge is not power – Knowledge is obligation. Journal of the American Academy of Psychiatry & the Law, 30, 533-540. Elbogen, E., Huss, M. T., Tomkins, A. J., & Scalora, M. J. (2005). Clinical decision-making about psychopathy and violence risk assessment in public sector mental health settings. Psychological Services, 2, 133-141. Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191. Hare, R. D. (2003). The Hare Psychopathy Checklist-Revised (2nd ed.). Toronto, ON: MultiHealth Systems. Harris, G. T., Rice, M. E., & Quinsey, V. L. (1993). Violent recidivism of mentally disordered offenders: The development of a statistical prediction instrument. Criminal Justice & Behavior, 20, 315-335. Hart, S. D., Cox, D. N., & Hare, R. D. (1995). Manual for the Hare Psychopathy ChecklistRevised: Screening Version (PCL:SV). Toronto, ON: Multi-Health Systems. Hart, S. D., Michie, C., & Cooke, D. J. (2007). Precision of actuarial risk assessment instruments: Evaluating the ‘margins of error’ of group v. individual predictions of violence. British Journal of Psychiatry, 190, s60-s65. Heilbrun, K. (2009). Evaluation for risk in violence in adults. New York: Oxford University Press. Higgins, N., Watts, D., Bindman, J., Slade, M., & Thornicroft, G. (2005). Assessing violence risk in general adult psychiatry. Psychiatric Bulletin, 29, 131-133. Jackson, R., & Hess, D. T. (2007). Evaluation for civil commitment of sex offenders: A survey of experts. Sexual Abuse: A Journal of Research & Treatment, 19, 425-448. Khiroya, R., Weaver, T., & Maden, T. (2009). Use and perceived utility of structured violence risk assessments in English medium secure forensic units. Psychiatrist, 33, 129-132. Kimonis, E. R., Fanniff, A., Borum, R., & Elliott, K. (2011). Clinician’s perceptions of indicators of amenability to sex offender-specific treatment in juveniles. Sexual Abuse: A Journal of Research & Treatment, 23, 193-211. McNiel, D. E., Chamerblain, J. R., Weaver, C. M., Hall, S. E., Fordwood, S. R., & Binder, R. L. (2008). Impact of clinical training on violence risk assessment. American Journal of Psychiatry, 165, 195-200. Singh Grant Proposal 7 National Institute for Health and Clinical Excellence. (2009). Core interventions in the treatment and management of schizophrenia in primary and secondary care. London: National Institute for Health and Clinical Excellence. Nicholls, T. L., Ogloff, J. R. P., & Ledwidge, B. (2007). Is the profound distrust of unbridled clinical opinion in the violence risk assessment field unfounded? Poster presented at the 4th Annual Forensic Psychiatry Conference, Victoria, BC. Nursing and Midwifery Council. (2004). Standards of proficiency for specialist community public health nurses. London: Nursing and Midwifery Council. Otto, R. K., & Douglas, K. S. (Eds.) (2010). Handbook of violence risk assessment. New York, NY: Taylor & Francis Group. Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (2006). Violent offenders: Appraising and managing risk (2nd ed.). Washington, DC: American Psychological Association. Ryba, N., Cooper, V., & Zapf, P. (2003). Juvenile competence to stand trial evaluations: A survey of current practices and test usage among psychologists. Professional Psychology: Research & Practice, 34, 499-507. Solomon, D. J. (2001). Conducting web-based surveys. Practical Assessment, Research, & Evaluation, 7. Retrieved November 30, 2011, from http://PAREonline.net/getvn.asp?v=7&n=19 StataCorp. (2007). Stata statistical software: Release 10.1. College Station, TX: StataCorp LP. Tolman, A., & Mullendore, K. (2003). Risk evaluations for the courts: Is service quality a function of specialization? Professional Psychology: Research & Practice, 34, 225-232. Viljoen, J. L., McLachlan, K., & Vincent, G. M. (2010). Assessing violence risk and psychopathy in juvenile and adult offenders: A survey of clinical practices. Assessment, 17, 377-395. Webster, C. D., Douglas, K. S., Eaves, D., & Hart, S. D. (1997). HCR-20: Assessing risk for violence (version 2). Burnaby, BC: Simon Fraser University, Mental Health, Law, and Policy Institute. Williams, A. M. (2011). Changes in community behavioral healthcare organizations subsequent to the implementation of Medicaid managed care. Unpublished doctoral dissertation, Indiana University of Pennsylvania, Indiana, PA. Yang, M., Wong, S. C. P., & Coid, J. (2010). The efficacy of violent prediction: A meta-analytic comparison of nine risk assessment tools. Psychological Bulletin, 136, 740-767.