California Institute for Mental Health Page 1 IMPLEMENTING EVIDENCE-BASED PRACTICES Transition to Independence Process (TIP) Model Description of the practice Target group and mission: Persons age 14-25 who have emotional or behavioral difficulties. This term is used to “encompass a variety of diagnoses and classifications that are applied differently in different child- and adult-serving systems and states (e.g. Severe Emotional Disturbance [SED], childhood chronic depression, emotionally handicapped [EH], and severe mental illness.)” The target group criteria for the TIP may thus be more inclusive than those applied in most county mental health systems, which focus only on those defined (by various legal entities) as meeting SED or Serious mental illness (SMI) criteria. The mission is to assist young persons in making a successful transition into adulthood, so that they achieve personal goals in the transition domains of employment, education, living situation, personal adjustment, and community life functioning. Practice components: TIP comprises both a set of principles, or “guidelines,” and an organizational structure. GUIDELINES: 1 Engage young people through relationship development, person-centered planning, and a focus on their futures. Tailor services and supports to be accessible, coordinated, developmentally-appropriate, and build on strengths to enable the young people to pursue their goals across all transition domains. Acknowledge and develop personal choice and social responsibility with young people. Ensure a safety-net of support by involving a young person’s parents, family members, and other informal and formal key players. Enhance young persons’ competencies to assist them in achieving greater self- sufficiency and confidence. Maintain an outcome focus in the TIP system at the young person, program, and community levels. Involve young people, parents, and other community partners in the TIP system at the practice, program, and community levels. California Institute for Mental Health Page 2 ORGANIZATIONAL STRUCTURE AND FUNCTIONS: The primary means for accomplishing the guidelines is a team of “transition facilitators.” Facilitators perform a wide range of functions, the most critical of which appears to be: Conduct strength-based assessments and facilitate person-centered planning, including crisis prevention Assist the youth and young adults in setting and achieving their goals across the transition domains Coach and teach these young people as they progress through service stages Work collaboratively with mental health staff, welfare staff, teachers, rehabilitation specialists, and others in order to ensure appropriate services Promote the involvement of parents and additional informal and formal key players – and assist the young people in the development of a healthy supportive social support network. Make appropriate referrals for clinical, medical or other services Work with young people in community settings that are comfortable and non-stigmatizing, at times that are convenient for them There is to be a Community Steering Committee which functions either as a governing board or advisory committee. It is made up of representatives of a wide range of child- and adultserving agencies relevant to the transition process; and also including advocates, parents, and young people. There is to be a Community Steering Committee which functions either as a governing board or advisory committee. It is made up of representatives of a wide range of agencies relevant to the transition process, including advocates and work-force development organizations. There must be extensive resource development through the transition facilitators and their supervisor, as many of the resources and links necessary for success will not exist (e.g., community college, housing, job development). The services are organized temporally in this way: 1. Initial Assessment and Planning to be completed within 7 to 45 days of assignment to the TIP team 2. Active Coaching Status lasts on average 8 months with a typical range from 3 to 15 months 3. Maintenance Coaching Status lasts on average 18 months with range from 4 to 48 months 4. Follow-along Status to maintain community services/supports Caseload: A transition facilitator can usually serve a maximum of 15 young people, with one third being in each of the status categories (i.e., active coaching, maintenance coaching, follow-along) as described California Institute for Mental Health Page 3 previously. Most sites have transition facilitators serving 8 to 15 young people at a time. Also, there is a supervisor for the transition facilitators who is responsible for recruiting and training transition facilitators, weekly supervision sessions with the transition facilitators, development of resources, monitoring of progress and outcomes, and overall management of the project. Almost all of the TIP sites assign the young people to transition facilitators in the traditional case management style where each facilitator has responsibility for specific clients. A few sites have tried, or are using, more of an ACT team caseload assignment in which responsibility for each client is shared by the whole team. Extent of evidence A. Studies of programs that have been identified with the TIP model. There are three empirical studies of programs that were intended to embody most or all of the TIP guidelines and associated practices. In addition, there is a five-site “Partnerships for Youth Transition” study underway: preliminary results from a Washington Partnerships for Youth Transition site are currently available as are preliminary results aggregated for all sites. 1. The “Steps to Success” Program.2 The Steps to Success (STS) program was one of the early TIP program adopters. Sixty-eight students exited the program during the period from 1997-2002. Of the 68 leaving the program, 43 had at least one year of exposure to the program. This STS group was compared, by the TIP developers, on several outcome measures with a matched comparison group of students in the Miami-Dade schools having an emotional-behavioral disturbance (EBD) classification in 1997 and with a matched comparison group of students with no disability classifications (called “Typical”). All participants were from this same geographical area of Miami, Florida. Matching was done on gender, ethnicity/race, and age. The findings showed STS exiters to have a non-statistically significant lower rate of employment than the EBD control group. (The proportion of Typical young adults employed was higher than in either of these groups.) The post-secondary education involvement by the STS exiters approached the norm reference set by the Typical young adults, and was statistically significantly higher than that of the EBD matched comparison group. The STS exiters had significantly fewer incarcerations than did the EBD comparison group. Although the Steps to Success group did better than the EBD group on two of the three measures, there are two serious methodological problems that weaken the conclusion that the Steps-toSuccess intervention was responsible. First, there are significant issues of selection bias. A) 11 persons were excluded from the experimental group because of disciplinary problems (but young people with disciplinary issues were eligible to be drawn from the EBD population for inclusion in the matched EBD comparison group). B) There was selfselection into the STS program (by students themselves and their IEP committees) rather than random assignment. C) Although participants were matched on age, sex, and race/ethnicity, this is not comparable to random assignment as a way of eliminating potential bias. D) It was not possible to control for “pre” status (such as high school graduation or earlier incarcerations). A second major problem is that the Steps-toSuccess program does not appear to reflect the model in critical ways: for example, no funding was ever obtained for ”official” transition facilitators. Instead services appear to be an ad hoc mix of planning, vocational preparation, treatment, and education carried out by existing personnel assuming the role of “transition facilitators.” California Institute for Mental Health Page 4 2. Jump on Board for Success (JOBS).3 The JOBS program is an implementation of the TIP model in several sites in Vermont. The initial site is in Washington County, VT. The JOBS program there dates back to the early 90s. Participants are SED and are eligible for admission through age 20 and can receive services through age 22. The program is a collaboration of education, corrections, vocational rehabilitation and mental health agencies. There is a transition specialist with functions as described above. Preliminary outcomes [not reported] resulted in expansion of the model to eight other sites, with the Department of Vocational Rehabilitation having a lead role supported by Corrections, Mental Health, and Child Welfare. A pre-post evaluation was performed for 80 participants. The 80 were graduates of three sites over 7 years, 94-2001. They averaged almost 18 when entering (range 16-20) and received an average of 7.6 hours of service a month. Like Vermont overall, 97% were white; two thirds were males. Study group members had to meet DVR requirements for successful closure, that is, to have been employed 90 days. The 80 participants showed positive pre-post increases in employment and high school graduation or obtaining a GED; there were decreases in homelessness, residential treatment, intensive mental health treatment, welfare support, and corrections involvement. Only a narrow age range (young adult) is represented in the study group, limiting external validity (as does the atypical ethnic composition). Like the previous study there are also very serious methodological problems. A) There was no control group. B) There was an equal number of participants who did not achieve successful DVR closure, but data are presented only for those who did. C) There was no control on what other services (such as case management or mental health treatments) participants might have received. 3. Partnerships for Youth Transition programs. The OPTIONS program in Clark County, Washington4 is the only Partnerships for Youth Transition site to have reported detailed preliminary findings. Options has transition specialists, a job-developer, and a supervisor for a population of 47 at any one time. Participants are in (or at risk of) out of home placement and have a DSM IV diagnosis. Two-thirds had been in special education and 75% had been arrested prior to admission. Clients are 14-21 at time of intake but can stay until age 25 if necessary. A modified TIP fidelity process is used. Outcome data through 9 months of service are available for 51 participants regarding living situation, employment, education, criminal justice contacts. Participants received an average of 99 hours of service each. At 9 months, about half of the youth (46%) showed consistently positive outcomes in all four or three domains and 31% of the youth showed negative outcomes in three or four domains. Education showed the most consistent gains with a decrease in arrest showing the largest improvement. These results, though modestly positive, also have serious methodological problems: A) There is no control group; and B) data are limited to persons who completed nine months of services (we don’t know about drop outs). C) Although participants had very serious histories, there may be a selection effect (persons with more promise chose to be in the program), so that results might not be replicated with a broad population. The other Partnerships for Youth Transition programs have begun reporting preliminary outcomes. Although detailed results are not yet available the, developers have described preliminary overall findings for the 194 persons who were enrolled in a PYT program for at least one year. In general, there were positive trends for involvement in employment, or school, and a reduction in school dropouts. There was a trend toward less interference with functioning by mental health and by substance abuse problems. Arrests were low California Institute for Mental Health Page 5 and did not change significantly. Although the TIP developers are consultants to and evaluators of the Partnerships in Transition programs, the degree of fidelity to the TIP model varies considerably. The evaluator of one program told us they do not use TIP at all; however, the data are not yet disaggregated by service model. The data system is one that is required by SAMHSA, and it is limited in what it can report uniformly; attrition, for example, is not reported uniformly. None of the programs have a control group. Thus the PYT program evaluations share the major methodological problems of the other studies: lack of clarity in the model, lack of attrition information, and lack of a control group or even solid “pre” information. B. Studies of programs which embody a number of the TIP guidelines 1. Project ARIES in Springfield, Oregon.5 Oregon has a long-established transition program based on interagency collaboration, but this was the first implementation to focus on youth with emotional problems. Project Aries was administered through the education system, but had a separate house as a center and very experienced teachers with masters’ degrees as transition specialists. Caseloads were in the range of 10-15, and a number of the TIP guidelines were observed, such as person-centered planning and individualized services. Initially there was a focus on competitive employment, but it changed to a focus on completing educational programs. There were 85 participants, with 61 formally existing the program. Of the latter, 61% obtained some sort of educational degree or certificate and about half planned to go on to get more schooling or training. Some sort of paid work was performed during services by 55% of those exiting formally, and 36% were working at exit. Information about the program and participants is quite limited. Clear methodological problems include: A) Lack of a control group; B) Lack of baseline information on employment and schooling; C) and lack of information about the 25% who did not formally exit the program. 2. Project RENEW (Rehabilitation, Empowerment, Natural Supports, Education, and Work) in New Hampshire.6 This was a small project starting in 1995 and based in a junior college. Staff were career and education specialists. There was strong interagency collaboration and a mentoring component as well as personal futures planning and flexible education services. Other components include skill building and flexible funds. We learn the outcomes of 18 participants (age 16-20) who stayed in services two years, but do not learn whether there were others who dropped out. At baseline: seven (39%) had already completed high school or obtained a general equivalency diploma (GED), and 2 participants (11%) were engaged in paid employment. Thirteen participants (72%) had had police or court involvement in the past 3 months. After two years only 17% had recent criminal justice contact, 17 of 18 had completed secondary school or were still in school, and 11 of 18 (61%) were employed. (In the two years 83% had been employed at some point.) Methodological problems include: A) Lack of a comparison group, B) little information on recruitment, C) no information on drop outs. 3. Job Designs7 was an Oregon employment-focused transition program situated on the campus of a residential mental health treatment program. The program lasted for ten years, starting in 1989, and limited evaluation data are available for the 1992-1995 period. The area is largely rural and the clients mostly white. Clients had a variety of behavioral problems and about a third were eligible for SED services. Outcome data were collected for 79 persons during this period. Some competitive work was recorded for 56 persons (71%) during the study period. (Earlier data show 39% of those working to be “successful” and 61% to be “unsuccessful.”) During the study period 53 persons left the California Institute for Mental Health Page 6 program, but the reasons for leaving are not clearly defined. Of the 53 persons leaving, 21% had dropped out of school, 30% had finished high school and 49% had attended school during their tenure in the Job Designs program. An exit interview was conducted with 36 clients; of these 24 (67%) said they would recommend Job Designs to a friend. The typical client received services from three or four agencies. The actual Job Designs program included a supervisor and three transition facilitators with caseloads of 1/15 or lower. The success rate is very low (39% of the 71% who worked were successful, or 28%). The rate of consumer satisfaction is very low compared to most mental health programs, where 85% or more typically would recommend a program to a friend. There was no control group, no pre vs. post measures, and no follow-up once participants left the program. C. D. Systematic Reviews of TIP Model Research 1. There are no reviews of TIP model studies, per se. A recent general review of the issues of transition to adulthood for persons with SED is provided by Davis.8 She cites several research efforts but concludes, “…None of the studies have employed an appropriate control or comparison group, so the findings should be viewed with caution.” 2. A comprehensive review of policy and research relevant to youth aging out of foster care was done by Collins.9 The empirical results reviewed are studies of the federal Independent Living Program. 3. A CIMH publication written by Lynne Marsenich reviews the literature on clinical needs and treatment of transition age young women.10 Many psychiatric disorders (eating disorders, PTSD) are much more likely to be experienced by young women, and services need to be specialized for them; for example, young mothers need special supports. The report includes important suggestions for how to provide transition services but does not review the transition services empirical literature. Adaptations for culture and other sub-populations TIP guidelines stress cultural competence. However, although youth of multiple races and cultures have been part of some of the research regarding TIP, more of the studies have been in New England or other places with limited diversity. No adaptations for other cultures (regarding, say, the role of the family) have been reported. As noted above, gender has significant effects on transition needs, but the TIP studies reported here have not focused on gender differences nor does the TIP model. The model (and studies of the model) also do not clearly distinguish subgroups by clinical status (e.g., conduct disorders vs. SMI vs. anxiety disorders such as PTSD). Capsule Summary of Evidence: Effective, Efficacious, Promising, or Emerging, Not effective or Harmful. Because of the numerous methodological problems cited in exiting studies, the empirical support for the TIP model to date is weak. The model has, however, a strong theoretical base and fairly well-developed tools for implementation and is continuing to build a research base. It is rated as an promising evidencebased practice since it has some empirical support. Clearly there is a critical need for studies to examine transition practices and systems for improving the progress and outcomes of youth and young adults with emotional and behavioral difficulties. In order to be a strong evidence-based practice, several randomized controlled trials will be necessary. Unlike ACT, California Institute for Mental Health Page 7 for example, which has one specialized function (reducing hospital days) for a clearly defined population (high hospital utilizers), TIP takes aim at a very wide range of systems, problems, and populations. In order to test the model adequately, trials are needed with a) different age groups (14 year olds have little in common with 24 year olds) b) different types and severity of disorder (behavior problems in school are quite different from schizophrenia spectrum disorders), c) different lead agencies and funding sources (services are likely to be different if located in schools as opposed to probation or mental health agencies, in part because of the requirements of categorical funding sources), and d) different institutional sources of clients (corrections, special education, residential treatment, foster care). Since TIP is based on broadly applicable principles it may well be effective in all these situations, but the very diversity of transition challenges necessitates an equal diversity in empirical trials. Information about the implementation process 1. Fidelity Scale The approach to fidelity we have attempted to use in these summaries is stated in a manual on EBP sponsored by the American College of Mental Health Administration11: Fidelity is adherence to the key elements of an evidence-based practice, as described in the controlled experimental design, and that are shown to be critical to achieving the positive results found in a controlled trial. The three steps in developing a fidelity scale then are a) determining a program model or practice is effective (preferably through randomized controlled trials) and b) determining what components of the program are associated with the effective outcomes, and c) measuring the effective elements with a scale that, when scored high, indicates (based on further empirical trials) that the effective outcomes in the randomized research will be replicated in the field. Since TIP does not yet have any controlled studies, “fidelity” here is limited to adherence to an a not yet validated set of program guidelines. Such guidelines are useful in ensuring model implementation but—lacking correlation with rigorous outcome research—high fidelity will not necessarily ensure good outcomes. In a personal communication the developers write: “We have developed, pilot-tested, and are refining a protocol for TIP Program Fidelity. We will be applying the new version to three or four of the [PYT] sites.” In addition, materials on the developers’ website offer information on an previous version of a TIP system improvement tool. “The TIP Case Study Protocol for Continuing System Improvement can be used to assist stakeholders in establishing a profile of the system's areas of strengths and weaknesses. This protocol uses multiple sources of information (e.g., the young person, a family member, a teacher, the transition facilitator, a service provider and record review) to document various components of the transition process (e.g., strengths, needs, transition planning, coordination, supports and services provided, gaps in support/service provision, effectiveness, and satisfaction) in a particular environment (e.g., school, school district, community). The information provided through the TIP Case Study analysis can be used by the administrators, staff, and other interested stakeholders to recognize the strengths of the TIP system and to set the occasion for their making modifications to further enhance the service system.” California Institute for Mental Health 2. Page 8 Implementation assistance available a. Manual or other detailed description by developers. There is an “TIP Operations Manual”12 and training modules are available. b. Developers available to assist. Competency-based training and training-of-trainer workshops can be arranged to strengthen an organization’s transition program capacity with Hewitt B. "Rusty" Clark, Ph.D., Nicole DeschÍnes, M.Ed., Jordan Knab, Ed.S., Arun Karpur, M.PH, or Mason Haber, Ph.D. of the National Center on Youth Transition for Behavioral Health: System Development & Research Team, Florida Mental Health Institute, University of South Florida, Tel. 813-974-6409; Fax. 974-6257 Email: clark@fmhi.usf.edu c. Other resources Advice of Consensus Panels and Other Experts: Davis13 reviewed the state of transition research and policy, including results of two ad hoc consensus panels which made recommendations prior to 2003. These groups were still struggling with issues of leadership and funding and did not address service provision. Clark cites a wide range of articles as representing “professional consensus” in his “TIP System Guidelines and Practice Elements with Associated Empirical Research Findings or Professional Consensus,” available from the TIP website. These are not, however, consensus panel findings but consist of conceptual papers and literature reviews of specific elements related to TIP guidelines. The Division on Career Development and Transition of the Council for Exceptional Children has published a series of “position statements” on transition.14 As noted above, there is a textbook available on transition services for young persons with disabilities.15 An edited book about transition-age youth issues and strategies for serving them has been published by the developers;16 available for $29.95 at: http://www.brookespublishing.com/ An edited book provides detailed instructions and forms/instruments for many transition services and functions from the standpoint of schools. Bullis, M., & Fredericks, H. D. (Eds.). (2002). Vocational and transition services for adolescents with emotional and behavioral disorders: Strategies and best practice. Champaign, IL: Research Press. There is a computerized outcome tracking system (Transition to Adulthood Program Information System [TAPIS] Progress Tracker) that is currently being programmed into a software system for dissemination. The Transition to Adulthood Assessment Protocol (TAAP) is an integrated management information system developed for youth in the PYT program. The NCYT Team maintains two websites that contain many other resources: <http://tip.fmhi.usf.edu> and <http://ncyt.fmhi.usf.edu>. The latter is the National Center on Youth Transition for Behavioral Health website. California Institute for Mental Health Page 9 Endnotes 1 Description is from the developers of the model. The primary developer is Hewitt B. (Rusty) Clark. He has worked with and published with a number of other researchers. The current University of South Florida team is described at: http://tip.fmhi.usf.edu/tip.cfm?page_ID=14 2 Information is drawn from: Karpur, A., Clark, H. B., Caproni, P., & Sterner, H. (2005). Transition to adult roles for students with emotional/behavioral disturbances: A follow-up study of student exiters from Steps-to-Success. Career Development for Exceptional Individuals, 28(1), 36-46; and Carroccio, D. F., Whitfield, D., Clark, H. B., & Karpur. (2003). Transition to Independence Process (TIP): System Development & Research Merged Data Analysis (MDA) Project: Parts I through V. Retrieved November 14, 2006. tip.fmhi.usf.edu/files/report_5_MDA_STS_Follow-up_web_02_05_04.pdf 3 Clark, H. B., Pschorr, O., Wells, P., Curtis, M., & Tighe, T. (2004). Transition into community roles for young people with emotional/behavioral difficulties: Collaborative systems and program outcomes. In D. Cheney (Ed.), Transition issues and strategies for youth and young adults with emotional and/or behavioral difficulties to facilitate movement in to community life : Council for Exceptional Children. 4 Lead evaluator is Nancy Koroloff: 503.725.4040, koroloffn@pdx.edu. Presentations are available at: http://www.rri.pdx.edu/CCTransitions/CCTranreports.htm 5 Bullis, M., Moran, T., Benz, M., Todis, B., & Johnson, M. (2002). Description and evaluation of the Aries Project: Achieving rehabilitation, individualized education, and employment success for adolescents with emotional disturbance. Career Development for Exceptional Individuals, 25(1), 41-58. 6 Cheney, D., Hagner, D., Malloy, J., Cormier, G., & Bernstein, S. (1998). Transitional services for youth and young adults with emotional disturbance: Description and initial results of Project RENEW. Career Development for Exceptional Individuals, 21(1); and Hagner, D., Cheney, D., & Malloy, J. (1999). Career-Related Outcomes of a Model Transition Demonstration for Young Adults With Emotional Disturbance. Rehabilitation Counseling Bulletin, 42(3). 7 This information is from Bullis, M., Tehan, C. J., & Clark, H. B. (2000). Teaching and developing improved community life competencies. In H. B. Clark & M. Davis (Eds.), Transition to Adulthood. Baltimore: Paul H. Brookes Publishing Co. A separate article presents data on the first three years of the project, with fewer participants. The article contains much more information about the program design. Bullis, M. (1994). Description and evaluation of the Job Designs program for adolescents with emotional or behavioral disorders. Behavioral Disorders, 19, 254-268. 8 Davis, M. (2003). Addressing the needs of youth in transition to adulthood. Administration and Policy in Mental Health, 30(6), 495-509. 9 Collins, M. E. (2001). Transition to adulthood for vulnerable youths: A review of research and implications for policy. Social Service Review, 75(2), 271-291. 10 Marsenich, L. (2005). A Roadmap to Mental Health Services for Transition Age Young Women: A Research Review Sacramento: California Women’s Mental Health Policy Council in conjunction with the California Institute for Mental Health. 11 Hyde, P. S., Falls, K., Morris, J., John A., & Schoenwald, S. K. (2003). Turning Knowledge into Practice: A Manual for Behavioral Health Administrators and Practitioners About Understanding and Implementing Evidence-Based Practices. Boston: Technical Assistance Collaborative, Inc., for The American College of Mental Health Administration. 12 Clark, H. B. (2004). TIP System Development and Operations Manual Tampa, Department of Child and Family Studies, Louis de la Parte Florida Mental Health Institute, University of South Florida. Downloadable from the TIP website: http://tip.fmhi.usf.edu 13 Davis, M. (2003). Addressing the needs of youth in transition to adulthood. Administration and Policy in Mental Health, 30(6), 495-509. 14 See for example: Halpern, A. S. (1994). The transition of youth with disabilities to adult life: A position statement of the Division on Career Development and Transition of the Council for Exceptional Children. Career Development for Exceptional Individuals, 17(2), 115-124; available:http://eric.ed.gov/ERICWebPortal/Home.portal?_nfpb=true&_pageLabel=RecordDetails&ERICExtSearch_SearchValue_0=ED299757&ERICExtSearch_SearchType_0 =eric_accno&objectId=0900000b80044b36 15 Sitlington, P. L., Clark, G., & Kolstoe, O. (2000). Transition Education and Services for Adolescents with Disabilities (Third ed.). Boston: Allyn and Bacon. 16 Clark, H. B., & Davis, M. (Eds.). (2000). Transition to Adulthood: A resource for assisting young people with emotional or behavioral difficulties. Baltimore: Paul H. Brookes.