California’s Child Welfare
Performance Indicators Project:
“irresistible information” presented to (and thanks to)
the Stuart Foundation
Barbara Needell, MSW, PhD
Center for Social Services Research
University of California at Berkeley
The Performance Indicators Project is a collaboration of the
California Department of Social Services and the University of California at Berkeley,
and is supported by the
California Department of Social Services and the Stuart Foundation
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
agenda
1.
2.
3.
4.
5.
6.
7.
a (brief) history
tracking outcomes/data samples
data on allegations and foster care
example measure: C1.3
AOC reports
public data use (and misuse)
next steps…
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
california child welfare
performance indicators project
•
•
•
•
•
•
•
•
•
•
1994: first Performance Indicators Report with 1992 data
1995-1997: Yearly Performance Indicators Reports
1998: State data changed from FCIS to CWS/CMS
2000: CFSR final rule
2001: AB636 passed into law
2001: Data on web
2004: AB636 quarterly outcomes reports
2007: Move to dynamic site
2008: continued expansion of measures and views
ongoing: data analysis, data linkage, publications & presentations
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
tracking child welfare outcomes
rate of referrals/
substantiated referrals
home-based services
vs.
out of home care
reentry to care
permanency
through reunification,
adoption, or
guardianship
counterbalanced
indicators of system
performance
length
of stay
stability
of care
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
use of least
restrictive
form of care
positive attachments to
family, friends, and neighbors
Source: Usher, C.L., Wildfire, J.B., Gogan, H.C. & Brown, E.L. (2002).
Measuring Outcomes in Child Welfare. Chapel Hill: Jordan Institute for
Families,
data samples
entry
cohorts
data
point
in time
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
exit
cohorts
How long do children stay in foster care?
January 1, 2008
July 1, 2008
December 31, 2008
child 1
child 2
child 3
child 4
child 5
child 6
child 7
child 8
child 9
child 10
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
July 1, 2006-December 31, 2006
California:
Recurrence of Allegations By Disposition Type
50
substantiated
inconclusive
unsubstantiated
assessment only
45
6
40
7
35
% 25
6
5
3
7
6
13
15
11
4
10
10
3
3
0
6
7
7
6
5
14
8
6
8
12
8
7
12
13
12
9
7
6
16
14
12
7
5
11
5
5
6
5
9
9
8
9
5
11
12
10
15
10
13
6
7
10
5
7
30
20
6
4
8
10
5
7
9
10
6m 12m 18m 24m
6m 12m 18m 24m
6m 12m 18m 24m
6m 12m 18m 24m
substantiated
base=36,754
inconclusive
base=36,455
unsubstantiated
base=82,592
assessment only
base=49,063
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
1998-2008
California:
Foster Care Entries, Exits, and Caseload
70,000
104,324
109,395
100,450
60,000
92,361
84,858
50,000
79,883
Exits
76,079
70,222
40,000
Entries
30,000
20,000
10,000
0
1998
1999
2000
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
2001
2002
2003
2004
2005
2006
2007
2008
composite 1, measure 3 (C1.3)
C1.3: Of all children entering
foster care for the first time in
the 6-month period who
remained in foster care for 8
days or longer, what percent
were discharged from foster
care to reunification in less
than 12 months from the date
of latest removal from home?
extensions:
1.
2.
3.
4.
5.
variables: age, gender, placement type,
removal reason, episode count, agency
type, days in care
exit status (3m – 120m)
alternative exit types (adoption,
guardianship, etc.)
filters
multiple report types
And off to the site…
http://cssr.berkeley.edu/ucb_childwelfare/
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
July –December 2000 First Entries
California:
Percent Exited to Permanency 96 Months From Entry
(n=11,818)
100%
90%
25
80%
36
50
70%
54
55
56
56
57
57
60%
87%
50%
6
5
40%
30%
13
17
19
20
20
20
9
9
10
10
10
60
72
84
96
8
20%
10%
0%
6
12
In Care
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
24
Other
36
Emancipated
48
Guardianship
Adopted
Reunified
July –December 2000 First Entries
California:
Percent Exited to Permanency 96 Months From Entry
white (n=3,819)
100%
90%
22
black (n=2,430)
19
28
39
80%
31
42
54
70%
24
58
59
59
59
60
47
48
49
50
50
50
60
60%
6
5
90%
50%
7
6
40%
15
30%
83%
12
16
8
19
20
20
21
10
21
7
20%
8
8
8
9
19
19
20
21
11
11
11
12
9
10%
0%
3
6 12 24 36 48 60 72 84 96
In Care
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
Other
Emancipated
3
6 12 24 36 48 60 72 84 96
Guardianship
Adopted
Reunified
white, relative placements (n=1,410)
black, relative placements (n=987)
100%
100%
90%
90%
80%
33
37
80%
53
70%
23
56
41
2
57
57
57
57
57
42
43
44
19
20
20
21
22
22
23
24
72
84
96
54
54
44
44
70%
2
60%
60%
6
50%
=93%
50%
10
12
17
5
40%
12
11
40%
17
19
30%
19
19
19
16
21
30%
15
20%
16
17
17
20%
17
17
10%
10%
0%
0%
12
24
36
48
60
72
84
96
white, non-relative placements (n=2,409)
12
100%
100%
90%
90%
80%
80%
42
54
70%
60%
89%=
58
59
60
61
61
61
8
2
40%
46
30%
20
21
21
21
22
49
52
3
3
3
3
3
10%
53
54
78%=
6
2
9
2
16
18
19
20
20
3
3
3
4
60
72
84
96
3
30%
3
20%
60
1
40%
16
48
37
=86%
50%
50%
36
black, non-relative placements (n=1,443)
70%
60%
1
24
20%
10%
0%
0%
12
24
36
48
60
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
72
84
96
12
24
36
48
1998-2008
Percent Exited to Permanency Over Time:
First Entry Cohorts, 8 days or more in care
70
Reunified
Adopted
Guardianship
60
50
40
%
30
20
10
0
3m
6m
1yr
2yrs
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
3yrs
4yrs
5yrs
Time from Removal
6yrs
7yrs
8yrs
9yrs
10yrs
administrative office of the courts
• data restricted to court dependent children when possible
• data from child welfare and court sources (filings)
• custom data views as requested by AOC (e.g., combination
of family maintenance and foster care caseloads in one
view, custom in care placement type groupings)
• county/state and county/county table and graph
comparisons
• % change between time frames
• provides links to source reports on cws/cms website to
allow users to further explore data
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
administrative office of the courts
• static and dynamic versions
– (dynamic) presents all available time frames in four
views (intervals)
– (dynamic) % change between user-selected time
frames
– (dynamic) selected time frames are dynamically
marked on tables and charts
• Demo…
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
public data:
putting it all out there
pros:
– greater performance accountability
– community awareness and involvement, encourages public-private
partnerships
– ability to track improvement over time, identify areas where programmatic
adjustments are needed
- county/county and county/state collaboration
cons:
–
–
–
–
potential for misuse, misinterpretation, and misrepresentation
available to those with agendas or looking to create a sensational headline
misunderstood data can lead to the wrong policy decisions
“Torture numbers, and they’ll confess to anything”
Gregg Easterbrook
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
data misuse/abuse
“Foster Children in Fresno County are three
times more likely to remain in foster care
for more than a year than in Sacramento.”
SF Chronicle, “Accidents of Geography”, March 8, 2006
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
“Foster Children in Fresno County are three times more likely to remain in
foster care for more than a year than in Sacramento.”
1.
Different families and children served?
2.
Different related outcomes?
- First entry rates in Fresno are
consistently lower
- Re-entries in Fresno are
also lower…
3. Other considerations…
- Resources available, resource allocation choices
- Performance trends over time
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
maltreatment allegations
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
entries to foster care
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
policy
"Our collaboration with the University of California, Berkeley on the
Child Welfare Dynamic Report System allows the State of California
to make data accessible for analysis by the general public,
stakeholders, and policy-makers. The availability of this information
permits us to make informed public policy to improve outcomes for
children and youth in foster care."
John Wagner
Director
California Department of Social Services
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
legislation
“As a county administrator, I am fully in support of public access to county
level child welfare data for many reasons. One in particular that has been
most effective for me in my role as a legislative advocate is to be able to
cite data on the CSSR when arguing for or against a particular bill
impacting child welfare. The fact that we can drill down to a particular
population the bill addresses and help inform the debate on both policy
and fiscal impacts results in data driven legislation; data that all the
stakeholders can view and understand leads to consensus on the facts.
This is a major breakthrough in the legislative process that before had to
operate on inaccessible data or data embedded in paper reports at the
local level that were a challenge to gather and analyze.”
Kathy Watkins
Legislative Program Manager/Legislation and Research Unit
San Bernardino County Human Services System
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
training
“The publicly available data
provided by CSSR/CDSS is
invaluable for the training
system. All of the statewide and
regional training systems can
integrate actual data into their
curricula - this brings training
alive for the participants, and
reinforces practice that is
informed by outcomes.”
Barrett Johnson
Director
Child Welfare In-Service Training Project
California Social Work Education Center
(CalSWEC)
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
“As a trainer in child welfare, the
accessible public data base
provides current information to
support many of the topics we offer
in our Regional Training
Academy. We refer to it frequently
and are grateful for the gifted folks
who created this resource and
made it open to those of us who
are peripheral but important to
supporting good child welfare
practice.”
Liz Quinnett
Program Coordinator
Public Child Welfare Training Academy
(PCWTA)
program evaluation
“I use the data from the UCB/CDSS site almost every week in my job
as principal analyst in the Program Evaluation & Research unit. The
data is invaluable for analysis of trends over time, answering
specific questions posed by Children & Family Services managers,
and for routine program monitoring reports. I frequently respond to
requests for ad-hoc analyses by working directly with the staff
person and showing them how to create reports from the website following the principle of ‘teach them how to fish’. I also refer staff
from community based agencies who ask for data for grant
proposals to the website and often show them how to extract data.
Tom Clancy
Program Evaluation & Research
Alameda County Social Services Agency
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
administration
"We commend Dr. Needell and the work of her staff at UCB/CDSS in
creating and refining the child welfare services data posted on the
public website. From an administrative perspective, the information
has been instrumental in identifying trends, program adjustments
and training needs. Being available to the public, the data has
supported our efforts in program transparency as well as serving to
educate the community on client needs and agency services."
Ken Jensen
Deputy Director
Santa Barbara County Dept. of Social Services
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
…irresistible information
“. . .your website has been extremely useful to me in my small county.
There was a report that was provided to the press a couple of years ago
and our local newspaper picked up on the fact that Calaveras County
was doing worse than the State overall in many areas. I was able to
refer them to your website where they could see exactly where
Calaveras County ranked amongst all of the counties, not the state
(because let's face it, that's pretty much Los Angeles County). Further, I
was able to point out those areas where we are exceeding the Federal
threshold and doing better than most other counties. It was extremely
beneficial. I have since referred folks from our Citizen's Review Panel,
Grand Jury and Juvenile Justice Commission to your website when I'm
asked for statistical information. Not only is it a real timesaver, but
provides solid, non-biased statistics.”
Mikey Habbestad
Program Manager II
Calaveras Works and Human Services Agency
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
continued work/next steps
• collaboration with other states moving to public sites
• preparation of the next generation of researchers and
practitioners
– doctoral students/dissertation research
– pilot program with SJSU MSW students…growing
momentum for UCB course
• linkages with EDD, birth, death, emergency
department, and hospitalization records
• continued work with AOC/courts
• enhanced online graphic capability
• child welfare council/data linkage subcommittee
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
January 2004-July 2009
California CWS Outcomes System:
AB636 Measures, % IMPROVEMENT
(+) or (–) indicates direction of desired change
PR: Referral Rate (-)
PR: Substantiation Rate (-)
PR: Entry Rate (-)
PR: In Care Rate (-)
3.9%
19.6%
12.3%
23.7%
2B: Timely Response (1 day) (+)
2B: Timely Response (10 day) (+)
3.2%
8.9%
2C: Timely Social Worker Visits (+)
23.8%
4A: Siblings (All) (+)
4A: Siblings (Some or All) (+)
23.3%
8.4%
4B: Entries First Placement (Relative) (+)
4B: Entries First Placement (Group/Shelter) (-)
4B: PIT Placement (Relative) (+)
4B: PIT Placement (Group/Shelter) (-)
15.1%
28.3%
4.7%
14.8%
Decline in Performance
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
Improvement in Performance
January 2004-July 2009
California CWS Outcomes System:
Federal Measures, % IMPROVEMENT
(+) or (–) indicates direction of desired change
3.0%
S1.1: No Recurrence of Maltreatment (+)
S2.1: No Maltreatment in Foster Care (+)
-0.4%
**REUNIFICATION COMPOSITE (+)
C1.1: Reunification w/in 12m (Exit Cohort) (+)
C1.2: Median Time to Reunification (-)
C1.3: Reunification w/in 12m (Entry Cohort) (+)
C1.4: Reentry Following Reunification (-)
-1.1%
**ADOPTION COMPOSITE (+)
C2.1: Adoption w/in 24m (+)
C2.2: Median Time to Adoption (-)
C2.3: Adoption w/in 12m (17m In Care) (+)
C2.4: Legally Free w/in 6m (17m In Care) (+)
C2.5: Adoption w/in 12m (Legally Free) (+)
///110.4%
24.5%
15.1%
38.2%
12.7%
**LONG TERM CARE COMPOSITE (+)
C3.1: Exits to Permanency (24m In Care) (+)
C3.2: Exits to Permanency (Legally Free) (+)
C3.3: In Care 3+ Yrs (Emancipated/Age 18) (-)
**PLACEMENT STABILITY COMPOSITE (+)
C4.1: Placement Stability (8d-12m In Care) (+)
C4.2: Placement Stability (12-24m In Care) (+)
C4.3: Placement Stability (24m+ In Care) (+)
12.1%
7.5%
8.6%
15.4%
-1.4%
-14.8%
10.3%
12.7%
8.4%
4.0%
4.1%
8.0%
Decline in Performance Improvement in Performance
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
79.0%
Thank you to the Stuart Foundation
for all your support over the years!
We could not have done this work
without you.
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley
Barbara Needell
bneedell@berkeley.edu
CSSR.BERKELEY.EDU/UCB_CHILDWELFARE
Needell, B., Webster, D., Armijo, M., Lee, S., Dawson, W., Magruder, J., Exel, M., Glasser, T., Williams, D., Zimmerman, K.,
Simon, V., Putnam-Hornstein, E., Frerer, K., Cuccaro-Alamin, S., Winn, A., Lou, C., & Peng, C. (2009). Child Welfare Services
Reports for California. Retrieved July 1, 2009, from University of California at Berkeley Center for Social Services Research
website. URL: <http://cssr.berkeley.edu/ucb_childwelfare>
Presentation Developed by
Emily Putnam-Hornstein and Christine Wei-Mien Lou
CENTER FOR SOCIAL SERVICES RESEARCH
School of Social Welfare, UC Berkeley