Global Appraisal of Individual Needs (GAIN): A Standardized Biopsychosocial Assessment Tool Michael Dennis, Ph.D. Chestnut Health Systems, Normal, IL Workshop at 16th Annual Drug Court Training Conference of the National Association of Drug Court Professionals, Boston, MA, June 3, 2010.. This presentation reports on treatment & research funded by the Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health Services Administration (SAMHSA) under contracts 270-2003-00006 and 270-07-0191, as well as several individual CSAT, NIAAA, NIDA and private foundation grants. The opinions are those of the author and do not reflect official positions of the consortium or government. Available on line at www.chestnut.org/LI/Posters or by contacting Joan Unsicker at 448 Wylie Drive, Normal, IL 61761, phone: (309) 451-7801, Fax: (309) 451-7763, e-mail: junsicker@Chestnut.Org Goals of this Presentation are to 1. Provide an overview of the role of the GAIN as a piece of infrastructure in support the move toward both evidence based practice and practice based evidence 2. Describe each of the measures, the reports that they use to help the assessment guide clinical decision making and illustrate how they provide a successively more detailed picture of client needs 3. Highlight our current work to using actuarial estimates of outcomes to improve placement decisions 4. Summarize the status of efforts to make the data available for secondary analysis and translate the software, measures and reports from English into Spanish, French, Portuguese and other languages Part 1. Provide an overview of the role of the GAIN as a piece of infrastructure in support the move toward both evidence based practice and practice based evidence The Global Appraisal of Individual Needs (GAIN) is .. A family of instruments ranging from screening, to quick assessment to a full Biopsychosocial and monitoring tools Designed to integrate clinical and research assessment Designed to support clinical decision making at the individual client level Designed to support evaluation and planning at program level Designed to support secondary analyses and comparisons across individuals and programs GAIN Collaborators in the U.S (4/10) NH VT WA MT ME MN ND MA OR ID WY NV CA UT WI SD MI NE CO KS AZ OK NM TX AK NY PA IA NJ OH DE WV MO VA MD KY DC NC TN State or No of AR SC GAIN Sites Regional System GA GAIN-Short None (Yet) MS AL Screener 1 to 14 GAIN-Quick LA 15 to 30 IL IN FL HI RI CT More in BZ, CA, CN, JP, MX 31 to 164 GAIN-Full VI PR 3/10 5 Some numbers as of April 2010 1,368 Licensed GAIN administrative units from 49 states (all by ND) and 7 countries 2,853 users in 405 Agencies using GAIN ABS 43,968 intake assessments (largest in field) 22,045 (88% w 1+ follow-up) from 219 CSAT grantees (largest follow-up data set in field) 22 states, 12 Federal, 6 provincial, and 3 foundations mandate or strongly encourage its use 4 dozen researchers have published 186 GAINrelated research publications to date Medicaid, Health Canada, several states, and private insurance systems accept it as evidence based 6 So what does it mean to move the field towards Evidence Based Practice (EBP)? Introducing explicit intervention protocols that are – Targeted at specific problems/subgroups and outcomes – Having explicit quality assurance procedures to cause adherence at the individual level and implementation at the program level Introducing reliable and valid assessment that can be used – At the individual level to immediately guide clinical judgments about diagnosis/severity, placement, treatment planning, and the response to treatment – At the program level to drive program evaluation, needs assessment, performance monitoring and long term program planning Having the ability to evaluate client and program outcomes – For the same person or program over time, – Relative to other people or interventions Key Issues that we try to address with the GAIN Instruments and Coordinating Center 1. High turnover workforce with variable education background related to diagnosis, placement, treatment planning and referral to other services 2. Heterogeneous needs and severity characterized by multiple problems, chronic relapse, and multiple episodes of care over several years 3. Lack of access to or use of data at the program level to guide immediate clinical decisions, billing and program planning 4. Missing, bad or misrepresented data that needs to be minimized and incorporated into interpretations 5. Lack of Infrastructure that is needed to support implementation and fidelity 1. High Turnover Workforce with Variable Education Questions spelled out and simple question format Lay wording mapped onto expert standards for given area Built in definitions, transition statements, prompts, and checks for inconsistent and missing information. Standardized approach to asking questions across domains Range checks and skip logic built into electronic applications Formal training and certification protocols on administration, clinical interpretation, data management, coordination, local, regional, and national “trainers” Above focuses on consistency across populations, level of care, staff and time On-going quality assurance and data monitoring for the reoccurrence or problems at the staff (site or item) level Availability of training resources, responses to frequently asked questions, and technical assistance Outcome: Improved Reliability and Efficiency 2. Heterogeneous Needs and Severity Multiple domains Focus on most common problems Participant self description of characteristics, problems, needs, personal strengths and resources Behavior problem recency, breadth , and frequency Utilization lifetime, recency and frequency Dimensional measures to measure change with interpretative cut points to facilitate decisions Items and cut points mapped onto DSM for diagnosis, ASAM for placement, and to multiple standards and evidence- based practices for treatment planning Computer generated scoring and reports to guide decisions Treatment planning recommendations and links to evidence-based practice Basic and advanced clinical interpretation training and certification Outcome: Comprehensive Assessment 3. Lack of Access to or use of Data at the Program Level Data immediately available to support clinical decision making for a case Data can be transferred to other clinical information system to support billing, progress reports, treatment planning and on-going monitoring Data can be exported and cleaned to support further analyses Data can be pooled with other sites to facilitate comparison and evaluation PC and web based software applications and support Formal training and certification on using data at the individual level and data management at the program level Data routinely pooled to support comparisons across programs and secondary analysis Over three dozen scientists already working with data to link to evidence-based practice Outcome: Improved Program Planning and Outcomes 4. Missing, Bad or Misrepresented Data Assurances, time anchoring, definitions, transition, and question order to reduce confusion and increase valid responses Cognitive impairment check Validity checks on missing, bad, inconsistency and unlikely responses Validity checks for atypical and overly random symptom presentations Validity ratings by staff Training on optimizing clinical rapport Training on time anchoring Training answering questions, resolving vague or inconsistent responses, following assessment protocol and accurate documentation. Utilization and documentation of other sources of information Post hoc checks for on-going site, staff or item problems Outcome: Improved Validity 5. Lack of Infrastructure Development Direct Services Training and quality assurance on administration, clinical interpretation, data management, follow-up and project coordination Clinical Product Development Software Development Collaboration with IT vendors (e.g., WITS) Data management Evaluation and data available for secondary analysis Over 36 internal & external scientists and students Workgroups focused on specific subgroup, problem, or treatment approach Labor supply (e.g., consultant pool, college courses) Software support Technical assistance and back up to local trainer/expert Outcome: Implementation with Fidelity Across measures, the GAIN has a Common Factor Structure of Psychopathology Source: Dennis, Chan, and Funk (2006) CFI=.92, RMSEA=.06 allowing for age Alcohol and Other Drug Abuse, Dependence and Problem Use are Age Related 100 90 80 70 Percentage 60 Over 90% of use and problems start between the ages of 12-20 People with drug dependence die an average of 22.5 years sooner than those without a diagnosis It takes decades before most recover or die Severity Category Other drug or heavy alcohol use in the past year 50 40 30 Alcohol or Drug Use (AOD) Abuse or Dependence in the past year 20 10 0 65+ 50-64 35-49 30-34 21-29 18-20 16-17 14-15 12-13 Age Source: 2002 NSDUH and Dennis & Scott, 2007, Neumark et al., 2000 Co-occurring Mental Health Problems are Common, Prevalence of co-occurring problems but the Type of Problems also Changes with Age by age groups Internalizing Disorders go up with age 100 Prevalence (%) 80 Any Internalizing 60 Externalizing Both internalizing and externalizing 40 20 0 <15 15-17 18-25 25-39 Age groups 40+ Externalizing Disorders go down with age (but do NOT go away) Source: Chan, YF; Dennis, M L.; Funk, RR. (2008). Prevalence and comorbidity of major internalizing and externalizing problems among adolescents and adults presenting to substance abuse treatment. Journal of Substance Abuse Treatment, 34(1) 14-24 . Progressive Continuum of Measurement (Common Measures) Quick Comprehensive Special More Extensive / Longer/ Expensive Screener Screening to Identify Who Needs to be “Assessed” (5-10 min) – – – – – – Focus on brevity, simplicity for administration & scoring Needs to be adequate for triage and referral GAIN Short Screener for SUD, MH & Crime ASSIST, AUDIT, CAGE, CRAFT, DAST, MAST for SUD SCL, HSCL, BSI, CANS for Mental Health LSI, MAYSI, YLS for Crime Quick Assessment for Targeted Referral (20-30 min) – Assessment of who needs a feedback, brief intervention or referral for more specialized assessment or treatment – Needs to be adequate for brief intervention – GAIN Quick – ADI, ASI, SASSI, T-ASI, MINI Comprehensive Biopsychosocial (1-2 hours) – Used to identify common problems and how they are interrelated – Needs to be adequate for diagnosis, treatment planning and placement of common problems – GAIN Initial (Clinical Core and Full) – CASI, A-CASI, MATE Specialized Assessment (additional time per area) – – Additional assessment by a specialist (e.g., psychiatrist, MD, nurse, spec ed) may be needed to rule out a diagnosis or develop a treatment plan or individual education plan CIDI, DISC, KSADS, PDI, SCAN Longer assessments identify more areas of unmet need 100% 90% 7% 9% 3% 8% 8% 22% 13% 80% 70% 1 Prob. 22% 2 Probs. 40% 3 Probs. 40% 30% 0 Reported 69% 60% 50% 1% 0% 98% 20% 4 Probs. 10% 0% GAIN SS GAIN Q GAIN I Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) Most substance users have multiple problems 18 Part 2. Describe each of the measures, the reports that they use to help the assessment guide clinical decision making and illustrate how they provide a successively more detailed picture of client needs 19 Next slides will Describe the difference in the breadth of information you get with different levels of assessment Summarize validation studies to date Illustrate the difference using data from a single sample (Reclaiming futures project) Demonstrate that multi-morbidity is the norm and varies by type of client and program GAIN-Short Screener (GSS) Administration Time: A 3- to 5-minute screener Purpose: Used in general populations to – identify or rule-out clients who will be identified as having any behavioral health disorders on the 60-120 min versions of the GAIN – triage area of problem – serve as a simple measure of change – Easy for administration and interpretation by staff with minimal training or direct supervision Mode: Designed for self- or staff-administration, with paper and pen, computer, or on the web Scales: Four screeners for Internalizing Disorders, Externalizing Disorders, Substance Disorders, Crime/Violence, and a Total GAIN-Short Screener (GSS) (continued) Response Set: Recency of 20 problems rated past month (3), 212 months ago (2), more than a year ago (1), never (0) Interpretation: Combined by cumulative time period as: – Past month count (3s) to measure of change – Past year count (2s or 3s) to predict diagnosis – Lifetime count (1s, 2s or 3s) as a measure of peak severity – Can be classified within time period low (0), moderate (1-2) or high (3) – Can also be used to classify remission as – Early (lifetime but not past month) – Sustained (lifetime but not past year) Reports: Narrative, tabular, and graphical reports built into web based GAIN ABS and/or ASP application for local hosting GAIN-Short Screener (GSS) GAIN SS Psychometric Properties 100% Low Mod. High Prevalence (% 1+ disorder) 90% Sensitivity (% w disorder above) 80% Specificity (% w/o disorder below) (n=6194 adolescents) 70% 60% 50% 40% 20% At 3 or more symptoms we get 99% prevalence, 10% 91% sensitivity, & 89% specificity 30% Using a higher cut point increases prevalence and specificity, but decreases sensitivity 0% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Total Disorder Screener (TDScr) Total score has alpha of .85 and is correlated .94 Source: Dennis et al 2006 with full GAIN version GSS Performance by Subscale and Disorders Screener/Disorder Internal Disorder Screener (0-5) Any Internal Disorder Major Depression Generalized Anxiety Suicide Ideation Mod/High Traumatic Stress External Disorder Screener (0-5) Any External Disorder AD, HD or Both Conduct Disorder Substance Use Disorder Screener (0-5) Any Substance Disorder Dependence Abuse Crime Violence Screener (0-5) Any Crime/Violence High Physical Conflict Mod/High General Crime Total Disorder Screener (0-5) Any Disorder Any Internal Disorder Any External Disorder Any Substance Disorder Any Crime/Violence Prevalence 1+ 3+ Sensitivity 1+ 3+ Specificity 1+ 3+ 81% 56% 32% 24% 60% 99% 87% 56% 43% 82% 94% 98% 100% 100% 94% 55% 72% 83% 84% 60% 71% 54% 44% 41% 55% 99% 94% 83% 79% 90% 88% 65% 78% 97% 82% 91% 98% 99% 98% 67% 78% 70% 75% 51% 62% 96% 85% 90% 96% 65% 30% 100% 87% 13% 96% 100% 89% 68% 91% 25% 73% 30% 14% 100% 82% 28% 88% 31% 85% 99% 46% 100% 94% 100% 94% 49% 70% 51% 76% 38% 71% 99% 77% 100% 97% 58% 68% 89% 68% 99% 63% 75% 92% 73% 99% 100% 100% 99% 100% 91% 98% 99% 92% 96% 47% 8% 10% 20% 10% 89% 28% 37% 51% 32% Moderate (1+) gives best result for sensitivity High (3+) gives best result for specificity Recommend Triage as 0=Not likely 1-2 Possible 3+=Likely GAIN SS Total Score is Correlated With Level Of Care Placement: Adolescents Total Disorder Screener for Adolescents % within Level of Care and Age Group 11% Lo 10% w 9% Mod High -> OP/IOP Median=6.0 Residential Median=10.5 OP/IOP (n=2499) Residential (n=1965) 8% 7% 6% 5% 4% 3% 2% 1% 0% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 TDScr Score GAIN SS Total Score is Correlated With Level Of Care Placement: Adults GAIN SS Can Also be Used for Monitoring 20 12+ Mon.s ago (#1s) 2-12 Mon.s ago (#2s) Past Month (#3s) Lifetime (#1,2,or 3) 16 12 10 11 9 9 10 Track Gap Between Prior and current Lifetime Problems to identify “under reporting” 8 8 3 4 2 2 0 Intake 3 6 9 12 15 18 21 24 Mon Mon Mon Mon Mon Mon Mon Mon Track progress in reducing current (past month) symptoms) Total Disorder Screener (TDScr) Monitor for Relapse GAIN Short Screener Profile: Reclaiming Futures (Range based on 0/1-2/3+ Symptoms) 100% 90% Low 80% 70% 60% 50% Mod. 40% 81% 30% 20% 48% 33% 37% 38% High 10% 0% Internalizing Externalizing Disorder Disorder Screener Screener Substance Disorder Screener Crime/ Violence Screener Total Disorder Screener Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) GAIN Short Screener Number of Problems Mod/Hi in Reclaiming Futures 100% 90% 7% 9% No SR prob 80% 70% 22% 1 Prob. 22% 2 Probs. 93% endorsed one or more problems (40% 4 or more) 60% 50% 40% 30% 40% 3 Probs. 20% 4 Probs. 10% 0% No. of Problems Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) Full GAIN measure Construct Validity of GSS Internalizing Disorder Screener 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% % Days with MH problem Mod/High on Emotional Problem Scale (EPS) Mod/High on Internal Mental Distress Scale (IMDS) Internalizing Disorder Screener (IDScr) 0 1 2 3 4 5 Source: Education Service District 113 (n=979) and King County (n=1002) Construct Validity of GSS Externalizing Disorder Screener Full GAIN measure 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% % Days with behavioral problems Mod/High on High on Behavior Emotional Problem Complexity Scale Scale (EPS) (BCS) Externalizing Disorder Screener (EDScr) 0 1 2 3 4 5 Source: Education Service District 113 (n=979) and King County (n=1002) Construct Validity of GSS Substance Disorder Screener 100% 90% Full GAIN measure 80% 70% 60% 50% 40% 30% 20% 10% 0% % Days of AOD use Past Year Abuse or Dependence Past Year Dependence Substance Disorder Screener (SDScr) 0 1 2 3 4 5 Source: Education Service District 113 (n=979) and King County (n=1002) Construct Validity of GSS Crime/Violence Screener Full GAIN measure 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% % Days of illegal activities Mod/High on Illegal Activity Scale (IAS) High on Crime/Violence Scale (CVS) Crime and Violence Screener (CVScr) 0 1 2 3 4 5 Source: Education Service District 113 (n=979) and King County (n=1002) Problems could be easily identified Comorbidity is common 75% 75% 12% 12% Substance Abuse Student Assistance Treatment Programs (n=8,213) (n=8,777) Either Juvenile Justice (n=2,024) High on Mental Health Mental Health Treatment (10,937) High on Substance 12% 11% 46% 35% 61% 60% 73% 62% 40% 37% 86% 83% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 77% 67% 57% 47% Adolescent Rates of High (2+) Scores on Mental Health (MH) or Substance Abuse (SA) Screener by Setting in Washington State Children's Administration (n=239) High on Both Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/ 4% 3% 17% 17% 18% 17% Lower than expected rates of SA in Mental Health & Children’s Admin 69% 69% 44% 51% 31% 64% 43% 53% 31% 65% 51% 46% 78% 73% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 81% 68% 69% 56% Adult Rates of High (2+) Scores on Mental Health (MH) or Substance Abuse (SA) Screener by Setting in Washington State Substance Abuse Treatment (n=75,208) Either Eastern State Hospital (n=422) Corrections: Community (n=2,723) High on Mental Health Corrections: Prison (n=7,881) Mental Health Treatment (55,847) High on Substance Childrens Administration (n=1,238) High on Both Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/ Adolescent Client Validation of Hi Co-occurring from GAIN Short Screener vs Clinical Records by Setting in Washington State Substance Abuse Treatment (n=8,213) Juvenile Justice (n=2,024) GAIN Short Screener Mental Health Treatment (10,937) 9% 11% 15% 12% 34% 35% 56% Two page measure closely approximated all found in the clinical record after the next two years 47% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Children's Administration (n=239) Clinical Indicators Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/ Higher rate in clinical record in Mental Health and Children’s Administration (But that was past on “any use” vs. “abuse/dependence” and 2 years vs. past year 3% 17% 22% 39% 59% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 56% Adult Client Validation of Hi Co-occurring from GAIN Short Screener vs Clinical Records by Setting in Washington State Substance Abuse Treatment (n=75,208) Mental Health Treatment (55,847) GAIN Short Screener Childrens Administration (n=1,238) Clinical Indicators Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/ Other Validations Confirmatory Factor Analysis Dennis, Chan & Funk (2006) found that the 20 item GSS and its four subscales were highly correlated (.84 to .94) with the full scale, had 90% sensitivity and over 90% area under the curve relative to the full GAIN; Confirmatory factors analysis also found it to be consistent with the overall model of psychopathology after allowing for age (CFI=.92; RMSEA=.06). Substance Disorders: McDonnell and colleagues (2009) found that the 5-item GAIN SS Substance Disorder Screener had 92% sensitivity and 85% correct classification relative to the Diagnostic Inventory Scale for Children (DISC) Predictive Scales (DPS; Lucas et al 2001) and 88% sensitivity and 88% correct classification relative to the CRAFFT (Knight et al 2001) Internalizing Disorders: McDonnell and colleagues (2009) found that the 5-item GAIN SS Internalizing Disorder Screener had 100% sensitivity and 75% correct classification relative to the Youth Self Report (YSR; Achenbach et al, 2001) and that the 5-item GAIN SS Externalizing Disorder Screener had 89% sensitivity and 65% correct classification to the YSR. Riley and colleagues (2009) found that the 5-item GAIN SS’s Internalizing Disorder Screener had 92% sensitivity and 80% area under the curve relative to the Structured Clinical Interview for DSM (SCID) and was more efficient relative to 11 item Addiction Severity Index (ASI) psychiatric composite score (McLellan et al., 1992), 10 item K10 (Kessler et al., 2002) and the 87 item Psychiatric Diagnostic Screening Questionnaire (PDSQ; Zimmerman and Mattia, 2001) GAIN Quick (GQ): Version 2 Strengths: Length (20-30min) in desired range, range of topics, efficiently categorizes, narrative reports to support screening, brief intervention, and referral to treatment Problems: – – – – Lacks scales or recency to support analyses or outcomes related to “change over time” Item response choices do not provide information about lifetime problems important when someone has been incarcerated for more than a few weeks Current Personal Feedback Report focuses only on substance use and does not address the other content areas of the GAIN-Q Only about 60% of the items can be directly imported into the GAIN-I GAIN Quick (GQ): Changes from Version 2 to 3: Kept focus on screening, brief intervention and referral to treatment Broke out sections for Crime/Violence, HIV risk, Work and School problems Subsumed GSS and added similar screeners in other GAIN Q areas with recency response to address change and lifetime issues Change measures for each symptom count and days items Created reasons for change items in each area to support brief intervention, reducing number of items in substance use Make all questions importable into full GAIN Expand narrative report to have more treatment planning statements and to allow motivational interviewing within each area GAIN-Quick (GQ) Version 3 Administration Time: about 25 minutes for core (varies depending on severity) and on average 25-45 minutes using full with motivational interview questions (depending on number of problem areas probed). Training Requirements: 1 day (train the trainer) training plus certification within 1-2 months for administration certification, and 2 days of motivational intervention training plus 1-3 months for clinical certification. Mode: Generally staff-administered on computer (can be done on paper or self-administered with proctor). Purpose: Designed for use in targeted populations for more detailed screening, to support brief intervention or referral for further assessment or behavioral intervention. GAIN-Quick (GQ) Scales: – GAIN SS scales + similar scales for school, work, physical health, psychosocial stress, and HIV risks – Additional “days” items and scale for measuring behavioral change – Recency and past 90 day measures of service utilization in each area to aid in placement, track implementation and estimate quarterly costs to society – Reasons for change to support motivational interviewing in each area – Life Satisfaction Scale and interview quality documentation Response Set: Recency (“the last time” scale), Breadth (past year symptom counts for behavior and lifetime for utilization) and Prevalence (past 90 days) Source: GAIN-Q Pilot (n=138) m e& io l in g in g en ce iso Pr s m s s 68% ob lem bl e Pr o s s rd er rd er Di so D es s bl em St r m s 63% V ith s m s bl e Pr o bl e rP ro io Us e na liz an ce er Su bs t Ex t liz rn a Be ha v m sw lth ea Pr o 46% In te Ri sk H bl e al k bl em 52% Cr i ic or W 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Pr o Ph ys lP ro Sc ho o GAIN-Quick (GQ) Problem Profile (Range based on 0/1-2/3+ Symptoms) Low Mod 58% 47% High 32% 26% 6% Number of Problems 100% 90% 1% 1% 3% 94% No SR prob 80% 70% 1 Prob. 60% 2 Probs. 50% 40% 30% 3 Probs. 20% 10% 4 Probs. 0% No. of Problems Source: GAIN-Q Pilot (n=138) 99% endorsed one or more problems (94% 4 or more problems) Distribution of Summary Indices 100% High-good 90% 80% Mod 70% 60% Low 50% Low 40% 30% 20% 10% Mod 49% 31% 15% 13% Problem Prevelance Index Quarterly Cost to Society High-bad 0% Source: GAIN Q Pilot . (n=138) Quality of Life Life Satisfaction Index Problem Prevalence Index Percent of days of problems in each of 8 areas summed, divided by range (23 items), and multiplied by 100 to get a score from 0 to 100. Problem Prevalence Index can be interpreted continuously where up is bad and subjectively unpleasant. It can also be triaged to low (0-5), moderate (6-24) or high (100) based on roughly 50%, 40% and 10% of the clinical population. Mean score here was 12.7 (10.2 Std. Dev). Quarterly Costs to Society The frequency of using tangible services in the 90 days before intake (e.g., health care utilization, days in detention, probation, parole, days of missed school) in each of the 8 areas valued in 2009 dollars, and summed. Quarterly costs to society can be used continuously with up as more expensive to society. It can also be triaged as low ($0 to $1999), moderate ($2000 to $9999) and high ($10000 or more) based on average costs of outpatient and residential treatment respectively. The 138 clients here cost society an average of $6,118 (SD=$12,382) per person in the quarter before intake and $24,471 (SD=$49,551) in the year before intake. Source: GAIN-Q Pilot (n=138) Quality of Life Year (QOLY) Triage of past year problem count in each of the 9 screeners weighted as No or Low=2, Moderate=1 and High=0; summed, divided by range 18, and multiple by 100 to get a quality of life score from 0 to 100. Quality of Life Year measure can be interpreted continuously where up is good and subjectively pleasant. It can also be triaged to low (0-36), moderate (3769) or high (70 to 100) based on roughly 50%, 40% and 10% of the clinical population. Mean score here was 39.7 (18.8 Std. Dev). Life Satisfaction Index Likert rating from strongly disagree (1) to strongly agree (5) in 5 areas (sexual relationship, living situation, family relationships, school/work situation, how life is going so far, general level of happiness) is summed and ranges from 6 to 30 Life Satisfaction Index can be interpreted continuously where up is good It can also be triaged to low (6-15), moderate (1627) or high (28-30) on roughly 50%, 40% and 10% of the clinical population Mean score here was 19.3 (5.7 Std. Dev) and was related to other summary indices GAIN Treatment Planning/Placement Grid Problem Recency/Severity None Past Current (past 90 days)* Low-Mod Past Current Treatment History** None 1. No Problem 0. Not Logical Check understanding of problem or lying and recode. 2. Past problem Consider monitoring and relapse prevention. . 5. No current problems; Currently in treatment Review for step down or discharge. 3. Low/Moderate problems; Not in treatment Consider initial or low invasive treatment. 6. Low/Moderate problems; Currently in treatment Review need to continue or step up. | High Severity 4. Severe problems; Not in treatment Consider a more intensive treatment or intervention strategies. 7. Severe problems; Currently in treatment Review need for more intensive or assertive levels. * Current for Intoxication & Withdrawal = Past 7 days ** Engagement in what ever the relevant service system is (school or work for vocational) Treatment Planning Needs by GQ Problem Area 0% 20% 40% 60% 80% 100% School* Work* Health Problems Stress Risk Behaviors Internal Disorders External Disorders Substance Disorders Crime & Violence* Inconsistent High Prob No problem No Prob in Tx Past Prob L/M Prob in Tx Low/Mod Prob H Prob in Tx * For school and work, in TX is defined as engaged; for Crime & violence, it is involved with CJS. Source: GAIN-Q Pilot (n=138) Use of Motivational Interviewing by GQ Problem Area GAIN Q Section 0% 20% 40% 60% School Work 80% 59% 8% Physical Health 67% Stress Risk Behaviors 77% 45% Mental Health 88% Substance Use Crime & Violence Source: GAIN-Q Pilot (n=138) 100% 85% 62% Duration Varies by Number of Problem Areas Where MI Questions Asked Minutes: Mean=45, sd=15 100% Minutes 50 80% 40 60% 30 40% 20 10 20% 0 0% 0 1 2 3 4 5 6 7 8 Number of areas where MI done Minutes Source: GAIN-Q Pilot (n=138) % of Sample % of Sample 60 Median=4-5 Problems Areas GAIN Initial (GI) Administration Time: Core version 60-90 minutes; Full version 110140 minutes (depending on severity) Training Requirements: 3.5 days (train the trainer) plus recommend formal certification program (administration certification within 3 months of training; local trainer certification within 6 months of training); Advanced clinical interpretation recommended for clinical supervisors and lead clinicians Mode: Generally Staff Administered on Computer (can be done on paper or self administered with proctor) Purpose: Designed to provide a standardized biopsychosocial for people presenting to a substance abuse treatment using DSM-IV for diagnosis, ASAM for placement, and needing to meet common (CARF, COA, JCAHO, insurance, CDS/TEDS, Medicaid, CSAT, NIDA) requirements for assessment, diagnosis, placement, treatment planning, accreditation, performance/outcome monitoring, economic analysis, program planning and to support referral/communications with other systems GAIN Initial (GI) (continued) Scales: The GI has 9 sections (access to care, substance use, physical health, risk and protective behaviors, mental health, recovery environment, legal, vocational, and staff ratings) that include 103 long (alpha over .9) and short (alpha over .7) scales, summative indices, and over 3000 created variables to support clinical decision making and evaluation. It is also modularized to support customization Response Set: Breadth (past year symptom counts for behavior and lifetime for utilization), Recency (48 hours, 3-7 days, 1-4 weeks, 2-3 months, 4-12 months, 1+ years, never) and Prevalence (past 90 days), patient and staff ratings Interpretation: – Items can be used individually or to create specific diagnostic or treatment planning statements – Items can be summed into scales or indices for each behavior problem or type of service utilization – All scales, indices and selected individual items have interpretative cut-points to facilitate clinical interpretation and decision making GAIN Initial (GI) (continued) Reports: – Validity Report (VR): identifying missing/bad data and potentially problematic areas of assessment – Individual Clinical Profile (ICP): lab report with graphical and tabular summary with links back to the items – GAIN Recommendation and Referral Summary (GRRS): Draft of biopsychosocial narrative for clinician to use for initial assessment summary, diagnosis, placement and treatment planning – Personal Feedback Report (PFR): used to support Motivational Interviewing (MI) / Motivational Enhancement Therapy (MET) – Program Profile: program level report that allows comparison of client characteristics, services received and outcomes between programs, cohorts or types of clients. GAIN Initial Profile: Substance Problems Past Year (Range based range of clinical/logical/statistical rules) 100% 90% Low 80% 70% 60% Mod. 50% 40% 30% High 20% 19% Sub. Prob. Past Year 7% Sub. Use/ Induced Prob. 0% 15% Dependence 10% Abuse 31% Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) GAIN Initial Profile: Substance Problems by Time (Range based range of clinical/logical/statistical rules) 100% 90% Low 80% 70% 60% Mod. 50% 40% 30% 39% 10% High 31% 13% Sub. Prob. Past Month Sub. Prob. Past Year Sub. Prob. Lifetime 0% 2% Withdrawal Sx Past Week 20% Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) GAIN Initial Profile: Motivation and Readiness (Range based range of clinical/logical/statistical rules) 100% 90% Low 80% 70% 60% Mod. 50% 40% 76% 30% High 20% 32% 10% Problem Orientation Treatment Motivation Treatment Pressure 7% Treatment Resistance 0% SelfEfficacy 0% 19% Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) GAIN Initial Profile: Crime/Violence (Range based range of clinical/logical/statistical rules) 100% 90% 80% 70% 60% Mod. 51% 8% Drug Crime Crime Violence 5% Interpersonal Crime 33% Property Crime 25% ConflictTactic 50% 40% 30% 20% 10% 0% Low Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) High GAIN Initial Profile: Environmental Risk (Range based range of clinical/logical/statistical rules) Low Mod. 64% 54% 39% Environmental Risk Social Environment Risk 28% Vocational Environment Risk Living Environment Risk 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) High GAIN Initial Profile: Internalizing Disorders (Range based range of clinical/logical/statistical rules) 100% 90% 80% 70% 60% Low 50% Mod. 40% 30% 20% Sucide Risk Depression 1% 9% 9% Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) Internalizing 3% Truama 15% Somatic 0% High 24% AnxietyFear 10% GAIN Initial Profile: Externalizing Disorders (Range based range of clinical/logical/statistical rules) Low 12% 14% 20% Conduct Disorder Externalizing 20% Hyperactivityimplusive Mod. Inattentiveness 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) High GAIN Initial Profile: Personality Disorders (Range based range of clinical/logical/statistical rules) 100% 90% Low 80% 70% 60% Mod. 50% 40% 30% 53% 20% 10% Total Personality Worrying (Cluster C) Implusive (Cluster B) Cautious (Cluster A) 0% Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) High GAIN Initial Profile: General Factors / Stress (Range based range of clinical/logical/statistical rules) 100% 90% 80% 70% 60% 50% 40% 30% Low Mod. 44% 20% 10% 0% High 10% 12% Person Axis IV Other Axis IV Victimization School Prob. Employment Prob. 26% Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) GAIN Initial Profile: Other Problem Scales (Range based range of clinical/logical/statistical rules) 100% 90% Low 80% 70% 60% Mod. 50% 40% 30% High 20% 2% 4% Life Satisfaction Health 0% 17% Social Support 12% Gambling 10% Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) GAIN Initial Profile: Measures of Behavior Change (Range based range of clinical/logical/statistical rules) 100% 90% 80% Low 70% 60% 50% Mod. 40% 30% Work Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) Financial 3% 4% School Substance Use 14% High 23% Illegal Activity 3% 0% 10% Recovery Environment 41% Emotions 41% Health 20% 10% GAIN Initial Number of Problems Mod/Hi 100% 90% 1% 0% 98% No problems 80% 70% 1 Prob. 60% 2 Probs. 50% 99% endorsed one or more problems (98% 4 or more) 40% 30% 3 Probs. 20% 10% 4 Probs. 0% No. of Problems Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) Any Illegal Activity in the Next Six Months by Intake Severity on Crime/Violence and Substance Problem Scales Knowing both is a better predictor (high –high group is 5.5 times more likely than low low) Any Ilegal Activity (months1-6) Intake Crime/ Violence Severity Predicts Recidivism 60% 58% 40% 20% 46% 53% 33% 44% 27% 36% 26% 20% High 0% Intake Substance Problem Severity Predicts Recidivism While there is risk, most (4280%) actually do not commit additional crime Mod High Mod Low Crime/Violence Scale (Intake) Low Substance Problem Scale (Intake) Source: CSAT 2008 V5 dataset Adolescents aged 12-17 with 3 and/or 6 month follow-up (N=9006) GAIN Treatment Planning/Placement Grid Problem Recency/Severity None Past Current (past 90 days)* Low-Mod None 1. No Problem Past Current Treatment History 0. Not Logical Check understanding of problem or lying and recode. 2. Past problem Consider monitoring and relapse prevention. . 5. No current problems; Currently in treatment Review for step down or discharge. 3. Low/Moderate problems; Not in treatment Consider initial or low invasive treatment. 6. Low/Moderate problems; Currently in treatment Review need to continue or step up. * Current for Intoxication & Withdrawal = Past 7 days | High Severity 4. Severe problems; Not in treatment Consider a more intensive treatment or intervention strategies. 7. Severe problems; Currently in treatment Review need for more intensive or assertive levels. Reclaiming Futures ASAM Placement Cells 0% 20% 40% 60% 80% 100% B1.Intox/Withd. B2 Biomedical B3.Psych/Beh B4.Readiness B5.Rel. Pot. B6.Environ. Inconsistent High Prob No problem No Prob in Tx Past Prob L/M Prob in Tx Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) Low/Mod Prob H Prob in Tx Continuing care 90% Case management 89% Recovery Environment Risk 87% Coping w/ Psychosocial Stressors 80% Accessing Treatment 76% Child Maltreatment 74% Disatisfaction with Environment 73% Behavior Control School Problems 70% 66% Anger Management 62% Vocational Assistance 61% Detox or Withdrawal 59% Recovery Coach 58% HIV risk reduction (sex) 57% Tobacco Cessation 56% Source: Reclaiming Futures (n=192) 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Other Common Treatment Planning Needs: Reclaiming Futures Part 3. Highlight our current work to using actuarial estimates of outcomes to improve placement decisions 75 CSAT Adolescent Treatment GAIN Data from 203 level of care x site combinations Levels of Care Long-term Residential Moderate-Term Residential Short-Term Residential Source: Dennis, Funk & Hanes-Stevens, 2008 Outpatient Continuing Care Intensive Outpatient Outpatient Early Intervention General Group Home Corrections Other Ratings of Problem Severity (x-axis) by Treatment Utilization (y-axis) by Population Size (circle size) Utilization Average Current Treatment . 1.00 F. HiHi (CC) 12% 0.80 0.60 0.40 B Low- Mod 0.20 0.00 C Mod-Mod 20% A Low-Low D Hi-Low 8% 12% -0.20 -0.20 G. Hi-Mod (Env Sx/ PH Tx) 9% E HiMod 14% 14% H. Hi-Hi (Intx Sx; PH/MH Tx) 12% 0.00 0.20 0.40 0.60 Average Current Problem Severity 0.80 1.00 Variance Explained in 10 NOMS Outcomes Percent of Variance Explained 0% 5% 10% 15% 20% 25% 24% No AOD related Prob. 11% No Health Problems 25% No Mental Health Prob. 15% No Illegal Activity 33% No JJ System Involve. 26% Living in Community 18% No Family Prob. 14% Vocationally Engaged 8% Count of above \1 35% 26% No AOD Use Social Support 30% \2 Past 90 days *All statistically Significant Past month Source: CSAT 2007 AT Outcome Data Set (n=11,013) 24% Predicted Count of Positive Outcomes by Level Predicted Count of Positive Outcomes by Level of Care: of Care: Cluster A Low Low (n=1,025) Cluster A Low - Low (n=1,025) 10 10 9 9 8 8 7 7 6 6 5 5 4 Person “A” does better in Outpatient 3 Person “B” does better in Higher Levels of Care 2 4 3 2 Outpatient Higher LOC Best Level of Care*: Level of Care*: Cluster A LowBest - Low (n=1,025) Cluster A Low - Low (n=1,025) 120% % Best Predicted Outcomes 99.6% 100% 80% 60% 40% 20% 0.4% 0% Outpatient * Based on Maximum Predicted Count of Positive Outcomes Source: CSAT 2007 AT Outcome Data Set (n=11,013) Higher LOC Best Level of Care*: Cluster B Low - Mod (n=1,654) 90% % Best Predicted Outcomes 80% 75.1% 70% 60% 50% 40% 30% 20% 14.1% 10.5% 10% 0.4% 0% Outpatient IOP * Based on Maximum Predicted Count of Positive Outcomes Source: CSAT 2007 AT Outcome Data Set (n=11,013) OPCC Residential Best Level of Care*: Best Level of Care*: Cluster C Mod-Mod (n=1209) Cluster C Mod-Mod (n=1209) 90% % Best Predicted Outcomes 80% 70% 60% 50% 40% 38.6% 30.2% 30% 23.6% 20% 7.6% 10% 0% Outpatient IOP * Based on Maximum Predicted Count of Positive Outcomes Source: CSAT 2007 AT Outcome Data Set (n=11,013) OPCC Residential Best Level of Care*: Cluster D Hi-Low (n=687) 90% 80% 70% 60% 50% 40% 38.3% 33.8% 27.9% 30% 20% 10% 0% Outpatient * Based on Maximum Predicted Count of Positive Outcomes Source: CSAT 2007 AT Outcome Data Set (n=11,013) IOP/OPCC Residential Best Level of Care*: Best Level of Care*: Cluster F Hi-Hi (n=968) Cluster(CC) F Hi-Hi (CC) (n=968) 90% 81.5% % Best Predicted Outcomes 80% 70% 60% 50% 40% 30% 20% 10% 9.9% 8.6% 0.0% 0% Outpatient IOP * Based on Maximum Predicted Count of Positive Outcomes Source: CSAT 2007 AT Outcome Data Set (n=11,013) OPCC Residential Best Level of Care*: Cluster Cluster H Hi-Hi (Intx/PH/MH) (n=1,017) 90% 78.2% % Best Predicted Outcomes 80% 70% 60% 50% 40% 30% 17.2% 20% 10% 0.0% 4.6% 0% Outpatient IOP * Based on Maximum Predicted Count of Positive Outcomes Source: CSAT 2007 AT Outcome Data Set (n=11,013) OPCC Residential Best Level of Care*: Cluster E Hi-Mod (n=1,190) 88.3% % Best Predicted Outcomes 90% 80% 70% 60% 50% 40% 30% 20% 10.6% 10% 0.0% 1.1% IOP OPCC 0% Outpatient * Based on Maximum Predicted Count of Positive Outcomes Source: CSAT 2007 AT Outcome Data Set (n=11,013) Residential Best Level of Care*: Best Level of Care*: Cluster G Hi-Mod (Env/PH) (n=749) Cluster G Hi-Mod (Env/PH) (n=749) 100% 94.1% 90% 80% 70% 60% 50% 40% 30% 20% 10% 5.9% 0.0% 0% Outpatient IOP/OPCC * Based on Maximum Predicted Count of Positive Outcomes Source: CSAT 2007 AT Outcome Data Set (n=11,013) Residential Best (x) by Actual (y) Level of Care Placement 3500 3132 3000 2339 2500 1968 2000 Higher Best 1500 Lower 1000 797 500 0 Outpatient (n=3132) Intensive Outpatient (n=797) OP - Continuing Care (n=1968) Residential (n=2339) Exploring Need, Unmet Need, & Targeting of Mental Health Services in AAFT At Intake . No/Low After 3 mon Any Treatment Mod/High Need Need 6 218 Total 224 218/224=97% to targeted No Treatment 205 553 758 553/771=72% unmet need Total 211 771 982 771/982=79% in need Size of the Problem Extent to which services are not reaching those in most need Extent to which services are currently being targeted Mental Health Problem (at intake) vs. Any MH Treatment by 3 months 97% 100% 90% 80% 79% 72% 70% 60% 50% 40% 30% 20% 10% 0% % of Clients With Mod/High Need (n=771/982)* % w Need but No Service % of Services Going to After 3 months Those in Need (n=553/771) (n=218/224) *3+ on ASAM dimension B3 criteria Source: 2008 CSAT AAFT Summary Analytic Dataset Why Do We Care About Unmet Need? If we subset to those in need, getting mental health services predicts reduced mental health problems Both psychosocial and medication interventions are associated with reduced problems If we subset to those NOT in need, getting mental health services does NOT predict change in mental health problems Conversely, we also care about services being poorly targeted to those in need. Residential Treatment need (at intake) vs. 7+ Residential days at 3 months 100% 90% 80% 70% 60% 50% 40% 30% 90% Opportunity to redirect existing funds through better targeting 52% 36% 20% 10% 0% % of Clients With Mod/High Need (n=349/980)* % w Need but No % of Services Going to Service After 3 months Those in Need (n=34/66) (n=315/349) Source: 2008 CSAT AAFT Summary Analytic Dataset Part 4. Summarize the status of efforts to make the data available for secondary analysis and translate the software, measures and reports from English into Spanish, French, Portuguese and other languages 93 Secondary Analysis We currently pool data from Center for Substance Abuse Treatment (CSAT) grantees annual and make it available for secondary analysis: – – – – Requires abstract length proposal/ feasibility Requires agreement to respect privacy and not attempt to re-identify We will get permission from any active grantees No cost to the end user Over 36 scientist and evaluators have already accessed the data and about 1-2 more come get approval each month We can also negotiate access to additional data from individual grantees and/or regional projects 94 Status of Translations Short Screener Other Instruments Software Reports English Spanish Done Done Done Done Done Done In progress In progress French In progress In progress In progress In progress Portuguese Done Starting Not yet Not Yet Japanese Hmong, Japanese, Russian, Pilipino, Punjabi, Vietnamese Done Not yet Not yet Not Yet Done Not yet Not yet Not Yet Language 95 Acknowledgments and Contact Information This presentation was supported by analytic runs provided by Chestnut Health Systems for the Substance Abuse and Mental Health Services Administration's (SAMHSA's) Center for Substance Abuse Treatment (CSAT) under Contracts 207-98-7047, 277-00-6500, 270-2003-00006 and 2702007-00004C using data provided by the following 152 grantees: TI11317 TI11321 TI11323 TI11324 TI11422 TI11423 TI11424 TI11432 TI11433 TI11871 TI11874 TI11888 TI11892 TI11894 TI13190TI13305 TI13308 TI13313 TI13322 TI13323 TI13344 TI13345 TI13354 TI13356 TI13601 TI14090 TI14188 TI14189 TI14196 TI14252 TI14261 TI14267 TI14271 TI14272 TI14283 TI14311 TI14315 TI14376 TI15413 TI15415 TI15421 TI15433 TI15438 TI15446 TI15447 TI15458 TI15461 TI15466 TI15467 TI15469 TI15475 TI15478 TI15479 TI15481 TI15483 TI15485 TI15486 TI15489 TI15511 TI15514 TI15524 TI15524 TI15527 TI15545 TI15562 TI15577 TI15584 TI15586 TI15670 TI15671 TI15672 TI15674 TI15677 TI15678 TI15682 TI15686 TI16386 TI16400 TI16414 TI16904 TI16928 TI16939 TI16961 TI16984 TI16992 TI17046 TI17070 TI17071 TI17334 TI17433 TI17434 TI17446 TI17475 TI17476 TI17484 TI17486 TI17490 TI17517 TI17523 TI17535 TI17547 TI17589 TI17604 TI17605 TI17638 TI17646 TI17648 TI17673 TI17702 TI17719 TI17724 TI17728 TI17742 TI17744 TI17751 TI17755 TI17761 TI17763 TI17765 TI17769 TI17775 TI17779 TI17786 TI17788 TI17812 TI17817 TI17825 TI17830 TI17831 TI17864 TI18406 TI18587 TI18671 TI18723 TI19313 TI19323 TI655374. Any opinions about this data are those of the authors and do not reflect official positions of the government or individual grantees. It is available at www.chestnut.org/li/posters. Comments or questions can be addressed to Michael Dennis, Chestnut Health Systems, 448 Wylie Drive, Normal, IL 61761. Phone 1-309-451-7801; E-mail: mdennis@chestnut.org. More information on the GAIN is available at www.chestnut.org/li/gain or by e-mailing gaininfo@chestnut.org . 96