Measuring to Scale: Applying Performance Based Contracting (PBC) to
a Statewide Residential Service System
Abstract
Supported by a DHHS Children’s Bureau grant, a collaborative workgroup
involving Illinois DCFS, area universities and residential program representatives
developed performance indicators, risk adjustment criteria, benchmarks and
related fiscal incentives that impact over 70 agencies statewide beginning FY
2009. This workshop provided a forum for discussion regarding the development
and utilization of clinically informed client centered performance measures.
Alan Morris, PsyD, University of Illinois at Chicago
Dr. Morris is the Associate Director of the CARTS Program and faculty member, Institute for
Juvenile Research, Department of Psychiatry, UIC
Brice Bloom-Ellis, LCSW, Illinois Department of Children and Family Services
Mr. Bloom-Ellis is the Statewide Residential Quality Assurance Manager, IDCFS
Correspondence may be directed to Alan Morris, PsyD, at the University of Illinois at Chicago,
Institute for Juvenile Research, 1747 W. Roosevelt Rd, M/C 747, Chicago, Illinois 60608 or
amorris@psych.uic.edu, and to Brice Bloom-Ellis at the Illinois Dept. of Children and Family
Services, 10 Collinsville Ave., East St. Louis, IL 62201 or brice.bloom-ellis@illinois.gov.
Background
During the 1990’s, the Illinois child welfare system was widely characterized as overwhelmed.
Especially noted were unsafe and unstable placements for youth, as well as insufficient and
misaligned mental health services that often were of very poor quality. Further, there existed an
overreliance on intensive highly restrictive services coupled with a paucity of effective
community based alternatives.
In the mid 1990’s the state entered into a consent decree following a federal class action lawsuit
(B.H.v McDonald) that acknowledged the above indicated deficiencies and systematically began
remediation efforts. With respect to residential treatment, these efforts included establishment of
a centralized reviewing mechanism for entry into residential treatment, return of youth placed in
out of state programs across the country, and a plan to no longer utilize out of state placements
for highly challenging youth.
These remedies resulted in a significant reduction in the overall number of youth residing in
residential treatment programs from 1995 to present as well as a dramatic reduction of youth
referred for out of state treatment (see Figures 1 and 2). In Illinois, a significant majority of youth
admitted into residential treatment are referred by the child welfare agency.
1
Youth in Residential Treatment
Illinois Trends
Youth in OutOut-ofof-State Placements
Illinois Trends
800
700
600
500
400
300
200
100
0
1989
5,000
4,000
3,000
2,000
1,000
0
1989 1992 1995 1998 2001 2004 2007
1992
Figure 1
1995
1998
2001
2004
2007
Figure 2
These changes, which occurred over a relatively brief period of time, resulted in a smaller, but
much more challenging, population of youth placed in residential treatment settings. During this
period, many Illinois providers struggled to enhance their programs to meet the needs of this
concentration of challenging youth. Figure 3 illustrates the change in population over time in
terms of number of adverse events youth experienced prior to admission.
Average Number of Adverse
Events at Entry to Residential Care
Average Number of
Adverse Events
3.0
2.9 (452%)
2.0
1.0
1.8 (364%)
0.6
0.8 (404%)
0.5
0.2
0.0
1997
1999
2001
2003
2005
2007
Year of Entry to Residential Treatment
Runaway
Psych hospitalization
Juvenile detention
Figure 3
Over the past 10 years, several additional initiatives were implemented that developed an
infrastructure that ultimately provided a framework for the initiative discussed in this paper.
These included comprehensive program plans for each residential agency that described in detail
their treatment philosophy, clinical program(s), resources and client population; a program
classification system that organized all programs by severity level and specialty population; a
centralized matching and referral system; an IDCFS residential monitoring program; system wide
electronic data collection and reporting capability; and, a university based specialized
consultation and support program that addressed both clinical as well as programmatic and
organizational issues. Finally, an active mechanism for maintaining ongoing communication and
mutual problem solving was developed that incorporated representatives from IDCFS, university
partners, and provider representatives from across the state.
2
Despite the system improvements, including enhanced clinical functioning of programs, increased
opportunities to access consultation and greater resource allocation for residential programs,
youth outcomes were disappointing on a system wide basis. During the three years from FY04
through FY06 60% of residential discharges were categorized as “negative”, with 40%
considered “positive” (criteria will be defined in the following sections). Further, only 61% of the
“positively” discharged youth (or 25% of the total youth discharged) remained in the same less
restrictive placement six months post discharge. In order to further address issues associated with
outcomes in residential treatment, IDCFS developed a PBC initiative.
Performance Based Contracting
In 2006, the Director of IDCFS co-authored a grant from the federal DHHS Children’s Bureau to
implement PBC across residential treatment programs throughout Illinois. This grant was written
in collaboration with the Child Care Association of Illinois and the Child Welfare Institute of the
University of Illinois, School of Social Work. Illinois is one of three sites in the United States
participating in this grant which is overseen by the Quality Improvement Center for the
Privatization of Child Welfare (QIC) at the University of Kentucky. QIC has been especially
interested in the Illinois site as its scope is statewide, considerably larger than any other location.
The Illinois site involves most residential providers across the state, including approximately 70
individual residential programs within 45 agencies.
The PBC Process: The Data Test Workgroup
Development of the specific goals, indicators, benchmarks and other measures associated with
PBC was assigned to the Data Test Workgroup, part of the Child Welfare Advisory Committee,
an influential public-private body created in 1999 to identify and propose solutions for system
wide problems. Prior to assuming this assignment, the workgroup had been involved in various
QI related activities associated with residential treatment for several years. The workgroup was
comprised of representatives from IDCFS, private residential provider agencies across Illinois,
and three Chicago area universities – University of Illinois at Chicago, Northwestern, and
University of Chicago-Chapin Hall.
This group, composed of researchers, residential program administrators and clinicians, and state
child welfare staff met regularly over the course of a year addressing issues associated with PBC.
Meetings were often quite animated and highly iterative in nature. Frequently, university
members presented data that were reviewed by residential program representatives for face
validity. Often university and IDCFS members prepared data for review in response to requests of
provider members. Partly due to the fact that the workgroup members had a long standing prior
working relationship, an impressive degree of trust and collegiality characterized the group,
despite the fact that the members often had quite diverse perspectives.
Between meetings the residential members provided updates and opportunities for reaction from
the larger provider community through periodic statewide provider meetings and through a
provider list serve maintained by a Data Test Workgroup committee member. Additionally,
IDCFS has sponsored several statewide “PBC Summits” to report progress and discuss relevant
issues with residential providers. Overall, this integrative process has been so well received and
fruitful that it has been proposed as a model for future child welfare and private sector initiatives
in Illinois.
3
Step 1: Developing the Goals
In discussions regarding the mandate for the Data Test Workgroup, IDCFS indicated that over the
past decade residential treatment capacity was “right sized” and this initiative should focus on
issues associated with quality and outcomes. Accordingly, over the course of several months, the
committee arrived at three overarching goals:
Goal 1:
Improve safety/stability of youth during their residential stay
Goal 2:
Reduce severity of clinical symptoms and increase functional skills
effectively and efficiently
Goal 3:
Improve outcomes at and following discharge from treatment
Regarding Goal 3, provider members noted that they had very little control over discharge
decision making as well as outcomes of youth post discharge and they expressed some hesitation
regarding assuming accountability in this area. However, recognizing that it is precisely those
outcomes that are most critical for the long term viability of residential treatment, they ultimately
decided to include that goal. It should be noted that over the ensuing months, IDCFS
implemented a comprehensive Transition and Discharge Protocol that grants residential program
staff far more input regarding decision making around discharge as well as additional resources to
support youth post discharge.
Step 2: Identifying Measurable Performance Indicators
In order to implement PBC across a large service system, measurable indicators were required
that addressed each goal. The workgroup identified several criteria for the indicators:



Indicators should meaningfully address each goal
Indicators would need to rely on currently available data
Indicators would need to utilize reasonably reliable data
The committee reviewed the extent data collected by IDCFS as well as those relevant data
collected by area universities in collaboration with IDCFS. It became clear that much of this data,
while useful for other purposes, did not meet the criteria identified.
For example, although the IDCFS system utilized the CANS as a decision support tool, and
residential agencies administered this measure at regular intervals, providers questioned the
reliability of administration of the instrument, as well as the utility of the instrument to adequately
be used as an outcomes measure for the subset of youth requiring residential treatment. Further,
potential conflict of interest issues were noted for providers who would have a financial stake in a
measure that they would administer and score.
In another example, Illinois providers also centrally report various adverse events electronically
to IDCFS (unusual incident reports). However, fidelity reviews of this data revealed uneven
reporting across agencies and the committee recognized that if this data were utilized providers
could be negatively impacted for good reporting fidelity. It was determined that payment data for
providers were much more reliable, although more limited in scope.
4
The agreed upon indicator(s) for each goal are as follows:

Indictor for Goal 1 -- Improving safety/stability of youth during their residential stay
Treatment Opportunities Days Rate (TODR).
This goal measured the overall proportion of days youth placed at each agency were
available for active treatment and not in detention, on runaway or in the psychiatric
hospital (see Figure 4). Several providers expressed concerns regarding the fact that
psychiatric hospitalization was an appropriate intervention for youth in acute crisis and
therefore should not be considered together with runaway and detention as a “negative”
event. The workgroup considered the fact that agencies with very similar populations had
highly disparate psychiatric hospitalization rates, and youth who moved across agencies
experienced highly disparate hospitalization rates even in the absence of changes in other
treatments such a psychotropic medication (see Appendix A for a detailed explanation of
this issue). As a result of these discussions, the workgroup agreed to incorporate
psychiatric hospitalization agency wide rates along with the other measures as proxies for
stability within the treatment setting.
Performance Indicators
Treatment Opportunity Days Rate
Percentage of time in treatment during residential
stay, i.e.
– at the facility
– not on runaway, in detention, or psychiatric hospital
RNY DET
0
5
10
15
20
HHF
25
30
RNY/DET/HHF
…
180
365
Bed days x 10 youth
100
50
215
T possible = 3650
Missed TODs = 365
Actual TODs = 3285
Calculation is # of days at facility / total # days in residential stay
Treatment Opportunity Days Rate: 3285 / 3650 = 90%
Figure 4 (RNY -- runaway; DET -- detention; HHF -- psychiatric
hospitalization)

Indicators for Goals 2 and 3 – Effectively and efficiently reducing symptoms/enhancing
functionality; improving outcomes at and following discharge
Sustained Favorable Discharge Rate (SFDR)
Initially these goals were addressed through three separate indicators: immediate
discharge disposition, sustained positive discharge, and length of stay. Immediate
discharge disposition of youth was determined by categorizing discharges into those that
were “positive” and “negative”. In general positive discharges were defined as those in
which the youth “stepped down” to a less restrictive setting including a group home or
5
residential program by classification, transitional living program, independent living
program, foster care, or home of relative. Negative discharges included lateral or step up
to a more restrictive residential or group home setting, and discharge from placement due
to detention/DOC, runaway or during a psychiatric hospital stay.
Sustained positive discharges refer to those discharges categorized as positive that also
remain in the subsequent setting for 180 days without discharge due to disruption.
Length of stay refers to the duration of the residential episode for youth residing in the
residential program.
These three original indicators pertaining to goals 2 and 3 were collapsed into the
Sustained Favorable Discharge Rate, a percentage of the total residential episodes
resulting in sustained favorable discharges. Rather than specify any length of stay
expectations, the workgroup preferred that each agency balance the desire to achieve
favorable sustained discharges with the requirement to function efficiently through
individual planning based on the best interests of each youth (see Figure 5).
Goal 1:
Improve Safety/Stability
During Treatment
Goal 2:
Effectively and Efficiently
Reduce Symptoms/
Increase Functionality
Goal 3:
Improve Outcomes At
And Following
Discharge
(Original) Indicators:
Immediate Discharge Disposition
Sustained Positive Discharge
Length of Stay
Indicator:
* Treatment Opportunity Days Rate
Indicator:
* Sustained Favorable Discharge Rate
Figure 5
Step 3: Leveling the Playing Field through Risk Adjustment
Each residential provider serves a unique case mix of children whose characteristics may impact
the outcomes the providers are expected to achieve. Because of their potential impact on
outcomes, these characteristics are termed risk factors. A strategy that attempts to take these
factors into account was developed so that fair comparisons could be drawn between the
performances of different providers. The Illinois residential risk adjustment model incorporates
risk factors primarily related to child characteristics for which data is available from existing
administrative databases. Through risk adjustment, individual provider performance benchmarks
were established which were then written into FY09 contracts.
6
The Data Test Workgroup undertook an iterative process of identifying risk factors by beginning
with a set of factors contained in a “Case Severity Index” recommended in Chapin Hall’s 2004
study of the Illinois residential treatment system (Chapin Hall, 2004). The Case Severity Index
included such factors as a child’s history of runaway, psychiatric hospitalization, and placement
in detention or residential treatment, prior to the residential placement of interest. Ideally a risk
adjustment model would also incorporate standard measures of a youth’s clinical status, delivered
via third-party administration, just prior to entry into residential treatment. The development of
risk adjustment strategies in the mental health field has been hindered by a number of limitations
associated with the availability and use of clinical data. (Hermann et al, 2007; also see McMillen,
et al, 2008.) The Data Test Workgroup similarly struggled with the lack of availability of factors
directly related to clinical status, and primarily relied on proxies such as the case history factors
in the Case Severity Index.
Prospective risk factors were first reviewed by members of the Workgroup for their face validity,
that is, their likely relationship to treatment outcomes based on the clinical experience of
Workgroup members, particularly those from the provider community. Then these risk factor-tooutcomes relationships were tested through multivariate regression analyses of a large population
of youth served in Illinois’ residential treatment system over a three year period. Essentially the
characteristics of those youth (e.g. their history of runaway, psychiatric hospitalization, etc.) were
tested simultaneously to determine the relative impact of those characteristics on the likelihood
that the youth in this sample experienced positive or negative outcomes. Risk factors that were
determined to have a statistically significant relationship to the outcomes of Treatment
Opportunity Days and Sustained Favorable Discharge rates can be generally grouped into three
categories, as follows:
1
2
3
demographic characteristics of youth
a age
b gender
c geographic region of origin
historical “child systems” involvement of youth
a runaway
b psychiatric hospitalization
c detention placement
d prior residential treatment
e prescribed anti-psychotic medications for symptoms of aggression
other placement characteristics
a length of residential placement (1 year or less)
b placement by severity level and specialty population
c geographic location of residential placement
The regression analyses indicated both the direction and relative weight of each of these factors.
For example, those youth with a history of prior psychiatric hospitalizations were found to have a
TODR nearly 2.5% lower than youth without that history, all other factors considered. Given this
finding, it is important to account for prior psychiatric hospitalizations when determining
providers’ TODR performance benchmarks.
Step 4: Setting Performance Benchmarks for Each Program
After determining the relative weight of risk factors in relationship to positive and negative
outcomes on the performance measures, those factors were applied to each youth served at each
7
residential agency during the fiscal years of 2006 and 2007. (Two years were used in an attempt
to minimize the impact of “outliers,” or youth generally not representative of provider
populations in any one year. The most recent fiscal year of 2008 was not used due to the longterm nature of the SFDR measure, and the desire to compare risk adjusted performance with
actual performance. By comparing actual to risk adjusted data of individual providers from FY06
and 07, the Data Test Workgroup was able to assess how well the risk adjustment model was
working based on its knowledge of those providers.) Risk factors applied to individual children
were aggregated up to the agency level (to a contract level as well for agencies with more than
one residential contract) with the results for the two fiscal years given a weighted average to take
into account the number of youth served. The resulting value was then used as the risk adjusted
performance benchmark for FY09.
In the example below, (Figure 6) Agency A achieved an actual TODR of 87.98% in FY06. Based
on the application of the weighted risk factors to the youth served at the agency during that year,
Agency A would be expected to have achieved a TODR of 94.43%, or 6.45% better than their
actual performance. This means that, from what was learned by conducting the multivariate
analyses of risk factors on all youth in residential treatment and their TODR outcomes, the youth
served by Agency A in FY06 would have been expected to have fared better on this measure.
Similarly, Agency A also underperformed in FY07, falling short of their expected TODR
performance of 94.88%. The weighted average of Agency A’s expected performance for the two
years resulted in the FY09 benchmark of 94.64%.
Setting Performance Benchmarks
Treatment Opportunity Days Rate
FY06
program
classification
# spells
actual TOD
rate (%)
risk adjusted TOD
rate (%)
actual minus
RA rate
agency
contract
Agency A
99999999
severe
24
87.98
94.43
-6.45
agency
contract
program
classification
# spells
actual TOD
rate (%)
risk adjusted TOD
rate (%)
actual minus
RA rate
Agency A
99999999
severe
25
91.73
FY07
94.88
-3.15
FY09 Benchmark
agency
contract
program
classification
avg. #
spells
avg. TOD
rate (%)
avg. risk adjusted
TOD rate (%)
avg. TOD
minus avg.
RA rate
Agency A
99999999
severe
25
89.71
94.64
-4.93
Figure 6
Step 5: Incorporating Financial Incentives
In addition to incorporating performance benchmarks, the Department’s FY09 PBC contracts also
establish financial incentives related to those benchmarks. The financial incentives apply to each
of the two performance measures. Agencies whose TODR performance falls below their
benchmark for the year will be assessed a financial penalty, while those that exceed their annual
SFDR benchmark will receive a financial bonus.
8
In the earlier explanation of TODR, an example was provided for an agency that served 10 youth
and achieved a TODR of 90%. That meant that the youth at that agency were present for 3285
days out of 3650 possible (3650 * .90 = 3285). If that agency’s TODR benchmark was set at
95%, it would be expected that the 10 youth would be present at the agency for a total of 3468
days out of the year (3650 * .95 = 3468), so the agency would have missed its benchmark by 183
days (3468 – 3285). The penalty for not meeting the TODR benchmark is 25% of the agency’s
per diem rate. If the agency in this example had a per diem rate of $300, it would be assessed a
penalty of $75 for each of those 183 days.
An agency serving 10 youth over the course of a year, with a benchmark SFDR rate of 20%
would be expected to achieve sustained favorable discharges for 2 youth. If that agency exceeded
its benchmark by achieving an SFDR of 40%, or 4 youth with sustained favorable discharges, the
agency would be entitled to a bonus. The bonus payment is derived by determining the
difference between the weighted averages of residential and “step down” living arrangement per
diem rates, multiplying the amount of that difference by the average number of days (up to 270)
that the 4 youth sustained their post-discharge placements, and then multiplying that amount by
the number of youth exceeding the benchmark, which in this example is 2. So, for example, if the
difference between the residential and “step down” living arrangement per diems is $150, and the
4 youth averaged 270 sustained favorable discharge days: $150 * 270 = $40,500. This amount
would be multiplied by 2, the number of youth that exceeded the benchmark, and the bonus in
this example would be $81,000.
Other PBC Fundamentals
There are other key components of this initiative with fiscal implications for the Department and
the residential provider community. The Department and residential providers worked
collaboratively over the past few years to develop a residential treatment cost structure that
standardized rates within residential classification levels. Doing so reduced existing disparities
between similar programs, and provided a more equitable range of resources among providers in
preparation for the implementation of a performance-based fiscal environment.
The Department also offered providers a more stable source of financing by purchasing a
guaranteed number of residential beds. (Prior to the current contract year, the Department paid
for residential beds on a per diem, utilization basis.) The Department reviewed its historical
utilization and projected its anticipated needs and then negotiated bed purchases with each
provider for FY09. In return for assuming the risks of underutilization and underestimating need,
the Department contractually required providers to assume the risk of accepting referred youth on
a “no decline” basis. There were two issues that had to be addressed in order for this to occur.
One was that providers had to commit in writing to their clinical capabilities and the parameters
within which youth would be deemed acceptable referrals. The other was that the Department
had to improve its referral process by centralizing decision-making so that a cadre of experienced
individuals with detailed knowledge of provider characteristics and capabilities would make the
referrals.
Complementary Initiatives
The IDCFS and UIC developed several supports for residential programs targeting both PBC
indicators, the TODR and SFDR. These are discussed in Appendix B.
9
References
Chapin Hall (2004). Residential Care in Illinois: Trends and Alternatives. Chicago:
Chapin Hall Center for Children.
Hermann, R.C., Rollins, C.K., & Chan, J.A. (2007). Risk-Adjusting Outcomes of Mental
Health and Substance-Related Care: A Review of the Literature. Harvard Review of
Psychiatry, 15 (52-69).
McMillen, J.C., Lee, B.R., & Jonson-Reid, M. (2008). Outcomes for Youth Residential
Treatment Programs Using Administrative Data from the Child Welfare System: A Risk
Adjustment Application. Administration and Policy in Mental Health, 35 (189-197).
10
Appendix A
Rationale for Including Hospital Days in the TODR Indicator
Overall Rationale
While it is understood that psychiatric hospitalization is an essential option along our
service continuum for very “high end” youth, an understanding of the characteristics of
that population and the role of residential treatment provide insight into why overall
hospital utilization is one legitimate measure of program effectiveness.
Our Data Test Work Group agreed early on that measures of youth’s stability within
programs are appropriate indicators for performance contracting. Youth referred to RTCs
in Illinois are extraordinarily context dependent based upon a wide range of clinical
considerations including effects of trauma on neurodevelopment. These youth are
referred to RTCs as it has been determined that they require a highly specialized setting
and milieu for stabilization which is essential to establish so that other interventions
(specific treatments, skill development, resiliency building efforts etc) can be successful.
It is widely understood that, for the vast majority of DCFS youth in residential treatment,
manifestation of clinical symptomotology and behaviors that result in an assessment for
hospital admission result from a combination of individual characteristics of youth as
well as the capacity of the program to maintain the youth’s psychological equilibrium.
While any single episode involving a youth’s decompensation and subsequent hospital
admission may be attributable overwhelmingly to individual characteristics of that youth
(a major psychotic break for example), hospital rates over time for agencies always
reflect program factors as well.
It has been stated that SASS assesses youth as requiring hospitalization; therefore
hospitalization was an appropriate outcome. We assume that is generally the case and that
hospitalization was necessary and appropriate at the time of the assessment. But, that is
not the issue here. The complexity of this issue can be illustrated in an example. Lets say
that in Program A, on a given weekend, all 10 youth on the unit demonstrated
behaviors/symptoms that were sufficiently serious to warrant hospitalization for their
own and others safety. No one disagrees with each of the SASS assessments and
consequent referral for admission on all 10 youth. However, at the same time, an analysis
of the course of events that weekend reveals that programmatic factors played a major
role in all 10 youths becoming destabilized to the point of requiring hospitalization.
We -- and agency staff -- have countless examples along these lines. I deliver talks and
presentations with real life examples all the time. Many agencies do “post mortems”
following these events in order to address programmatic issues that contributed to the
outcome. Again, these programmatic factors reflect situations where the milieu (and/or
other treatment elements such as psychiatric assessment and medication) was not able to
provide the context required to maintain the youth’s psychological equilibrium;
consequently, symptoms and behaviors escalate to the point of requiring hospitalization.
11
Supporting Data
Evidence for utilizing psychiatric hospital rates as a measure of program effectiveness
with respect to “in placement” stability is seen in the following:

Programs that serve similar populations consistently reveal dramatic differences
in hospitalization rates. And, this is seen over all geographical locations and
therefore is not merely a function of psychiatric hospital availability.

Programs that serve the same youth reveal dramatic differences in hospital
utilization (as well as many other indicators of functioning) following a change in
placement. This effect is often immediate and sustained over time – at least while
the youth remains in placement. The RTS consultants at UIC regularly see these
differences and understand this phenomenon as a manifestation of the “context
effect”.

Many programs internally view hospitalization rates through their own QI
processes as a measure of program effectiveness. I am aware of programs
reporting significant reductions in rates as an indication of quality improvement.
In this regard, hospitalization is somewhat analogous to restraints. While there is
general consensus that restraints are critical element in the behavior management
“toolbox” in order to maintain youth and staff safety -- and there are always
incidents of restraint use that are perfectly justifiable and necessary -- restraint
rates are commonly seen as a quality indicator and there are many efforts
underway to reduce the rates in agencies here and across the country.

We have completed an outcome evaluation of the UIC/CARTS Program that
relies on hospital utilization as a measure of the effectiveness of our program. We
compare utilization on the same youth before and following consultation. Our
consultation primarily focuses on milieu interventions that are designed to provide
greater levels of stabilization for the youth. These findings have been and will be
published/presented in peer reviewed journals and conferences which supports the
notion that psychiatric hospital utilization rates are a legitimate measure of
program effectiveness.
Currently, UIC, DCFS, several residential providers and two community hospitals
have initiated a pilot project that incorporates a scaled down version of CARTS
practices. This project will also track inpatient psychiatric utilization rates as one
component of outcome analysis.

Identification of hospital utilization rates is commonly utilized to reflect adequate
– or inadequate – levels of primary and secondary prevention for a wide variety of
disorders. For example, public health studies review local rates of hospitalization
of asthma patients due to acute exacerbations of symptoms as an indicator of the
availability and effectiveness of outpatient treatment within a given community to
adequately treat this problem. Here too, when hospitalization rates are high in one
12
community compared to others, no one is arguing that these patients didn’t
require hospitalization. The concern is, rather, that the effectiveness of efforts to
stabilize symptoms on an outpatient basis within that community is insufficient.
Again, while it is understood that in a given instance, perhaps, nothing could be
done to prevent a particular hospitalization, overall rates are an indication of the
adequacy of outpatient care.
Conclusion
Of course, given the challenges that Illinois RTCs face today, we do not expect that even
highly competent programs will maintain all youth such that they will not require
hospitalization. Hospitalization is indicated to achieve – or regain – a level of stability in
a youth that allow them to reintegrate successfully into the program upon discharge. That
is why the benchmarks allow for this. However, hospitalization also constitutes a
disruption in continuity of treatment, is typically highly traumatic for the youth and
others involved, and often is accompanied by a rupture in relationships that require
careful handling to successfully repair. Consequently, to the extent we can impact
psychiatric hospital utilization by strengthening our RTC’s capacity and effectiveness; we
are contributing to the overall well being of youth in our care.
13
Appendix B
DCFS/UIC Initiatives:
Residential Performance Based Contracting (PBC)
System and Agency Supports
In an effort to improve the overall outcomes for youth who are treated in residential
programs and assist agencies in their efforts to enhance service delivery and programming
DCFS, in collaboration with University of Illinois - Chicago, has developed several
initiatives that support and complement the PBC performance indicators. While each project
was conceptualized -- and several were initiated -- prior to the PBC effort, they directly
address highly relevant clinical and programmatic elements associated with each of the PBC
goals. Input from residential agencies who have participated in planning and pilots have
been, and will be, incorporated into the development of final recommendations and action
plans for each of the three projects. Each of these initiatives, together with the relevant PBC
indicator, will be described below.
PBC Indicator: Sustained Favorable Discharge Rate (SFDR)
Support Initiative: Residential Transition and Discharge Protocol
It has been widely observed that treatment planning and system support for effective
transitions across levels of care is insufficient and inconsistent with current knowledge
regarding the effects of trauma and contextual influences on behavior and symptoms.
Consequently, in many cases, youth who demonstrate gains over the course of their
residential stay do not sustain those gains in subsequent placement.
The Residential Transition and Discharge Protocol incorporate best practice into the
treatment planning process and coordination of efforts with respect to planned discharges.
Additionally, the protocol provides specific supports for critical transition related
activities and services. In December 2006, DCFS, in conjunction with UIC, began a one
year pilot program to study the implementation of a newly drafted residential transitional
policy. The pilot indicated that there appear to be sufficient evidence that positive results
can be achieved when protocol practices and principles are adopted. On whole, staff from
the sending residential program, receiving agency and DCFS involved in the pilot report
that good planning, clear identification of responsibilities and enhanced services to
support transitions results in improved outcomes for youth. Input from the pilot resulted
in significant changes in the protocol.
Throughout spring and early summer, 2008 all residential and receiving agencies as well
as both public and private sector caseworkers were trained on the protocol. The training
established a framework that continually focuses on transition and discharge planning
throughout the residential stay and into the next setting. There is a heavy focus on teamdecision making and connections to family, community and across levels of care.
The protocol was implemented system-wide starting July 1, 2008.
14
PBC Indicator: Treatment Opportunity Days Rate (TODR)
Support Initiative: Residential Runaway Initiative
The PBC Indicator, RTOD, addresses the extent to which youth in residential care are
available for ongoing residential treatment services and not absent from care due to
runaway, detention or psychiatric hospitalization. It is assumed that the overall rates for
treatment opportunity days are associated with the relative stability of the residential
program milieu. Specifically, the issue of runaway from residential agencies has
generated considerable attention due to issues of safety for youth on run in the
community. To better address this issue, DCFS, in conjunction with UIC, initiated the
Residential Runaway Project.
Phase 1 of the project was designed to elucidate clinical and systemic factors associated
with residential runaways; evaluate Illinois residential runaway rates in terms of
placement characteristics, nature of client populations, and geographical factors; survey
current practice in terms of runaway prevention, procedures utilized once a youth has run
and interventions utilized when youth return from run; and identify policy issues
requiring further clarification.
Phase 2 focused on the development of best practice guidelines for addressing runaway
behavior and recommendations for the development of a comprehensive runaway
protocol for all agencies.
Phase 3 resulted in the development of a tool, the Residential Runaway Risk Assessment
that addresses runaway prevention as well as youth vulnerability and dangerousness in
the community and includes an individual treatment planning component.
All residential agencies were expected to develop a comprehensive runaway protocol by
May, 2008 and a pilot project utilizing the Residential Runaway Risk Assessment and
treatment planning tool began July 1, 2008.
PBC Indicator: Treatment Opportunity Days Rate (TODR)
Support Initiative: Psychiatric Hospital Network Development
While acute psychiatric hospitalization is a critical element in the continuity of care for
the population of youth in residential care, its use is associated with the quality and
intensity of services available in less restrictive levels of care. A seven year outcomes
study evaluating the Comprehensive Assessment and Response Training System
(CARTS) at UIC has demonstrated that improved communication between hospital and
residential provider and effective transfer of technical milieu based treatment
interventions result in significantly reduced reliance on acute psychiatric hospitalization
for extremely challenging and severely emotionally disturbed youth with histories of very
high psychiatric hospital utilization.
Building upon the CARTS experience, DCFS and UIC are currently planning the
development of enhanced hospital-residential program networks that address
15
communication, continuity of treatment planning, and technology transfer. Best practice
guidelines and critical pathways were developed with these goals in mind.
A pilot involving two residential agencies and hospitals, including an evaluation
component, was implemented in March, 2008. Based on positive preliminary findings,
plans are currently underway to expand the network model to additional residential
programs and hospitals.
16