Fostering a Permanent Home:
A Mixed Methods Evaluation of the ZERO TO THREE
Court Teams for Maltreated Infants and Toddlers Initiative
Kimberly L. McCombs-Thornton, PhD
University of North Carolina at Chapel Hill
December 10 , 2011
Outline
• Background and significance
• ZTT Court Teams program
• Quantitative study
• Qualitative study
• Limitations and Discussion
2
Background and Significance
• Infants experience highest rate of victimization; toddlers
second
– Infants (< age 1):
– Toddlers (age 1):
– Toddlers (age 2):
20.6 per 1000
11.9 per 1000
11.3 per 1000
• Developmental needs of young children are time sensitive
• The caregiver is important for the child’s healthy
development
• Infants and toddlers in child welfare are at risk for
– Attachment disorders
– Poor physical health
3
Background and Significance
• The Adoption and Safe Families Act (ASFA) of 1997 seeks to
expedite the time to reach a permanent, stable home for
children in child welfare
• A permanent home may include:
– family reunification
– legal guardianship
– or traditional adoption
• More children appear to be reaching permanency since ASFA’s
enactment, though the need is still great
– Adoptions rose 64.5% in the three years following ASFA’s enactment
– Represents only 28% of children eligible for adoption during these years
4
ZTT Court Teams Project
TARGET GROUP: CHILDREN ENTERING FOSTER CARE BEFORE AGE 3
LOCAL ACTORS
Judge
Community
Coordinator
ACTIVITIES
Monthly Case
Reviews
LONG-TERM OUTCOMES
Decrease in Time to
Permanency
Referral to Childfocused Services
Child-Parent
Psychotherapy
Reduction in Recurrence
of Maltreatment
Court Team
-DHS case worker
-CASA
-Attorneys
-Providers
-Etc.
Evidence-Based
Parent Education
Improved Child
Well-being
ZTT National Office
Activities
-Training and TA
-Resource materials
-M and E
5
Previous Evaluation
• JBA completed evaluation in 2009
– Process oriented
– 95% of closed cases reached permanency
– 55% of those reached permanency within one year
• No comparison
group
Problem of causal
inference
6
Study Aims
Aim 1: Determine the effect of the ZTT Court Teams
project on time to permanency.
Aim 2: Assess the influence of the ZTT Court Teams
program on how children exit the foster care system.
Aim 3: Examine successful and unsuccessful cases to
understand how program components and/or client
characteristics contribute to time to permanency.
7
Causal Inference
Q: How do we know outcomes are due to the program?
A: Need a comparison group
Basic
Pre/post
Test
Advanced
Secondary
Data as
Comparison
Matched
Comparison
Group
Randomized
Experiment
(w/ control group)
8
Quantitative Data
ZTT “Treatment” Group
Dataset:
Court Teams MIS
Data Source:
CPS documents, observations in
court, etc.
What collected:
Child background, service needs
and usage, visitation,
placements, case status
Study sample:
All children in first 4 ZTT sites
who entered by end of 12/2009
Number in study: 298 children
Follow up period: One year or more for 94% of
cases
9
Quantitative Data
ZTT “Treatment” Group
NSCAW Comparison Group
Dataset:
Court Teams MIS
National Survey of Child and
Adolescent Well-Being (NSCAW)
Data Source:
CPS documents, observations in
court, etc.
Interview with CPS case worker
What collected:
Child background, service needs Family risks, child living
and usage, visitation,
environments, services needed
placements, case status
and received, child behavior, etc.
Study sample:
All children in first 4 ZTT sites
who entered by end of 12/2009
Children who entered child welfare
supervised out of home placement
before the age of 3
Number in study: 298 children
511 children
Follow up period: One year or more for 94% of
cases
One year or more for all cases
10
Case Characteristics
Child Characteristics
Age of child at first out of home placement
Mean (months)
Infant
Child gender
Male
Race/Ethnicity
African American
Caucasian
Latino/a
Parent Characteristics
Substance Abuse
Severe Mental Health Needs
Poverty
Lack of Employment
ZTT
(n=298)
NSCAW
(n=511)
9.3 (1.0)
67% (.04)
11.0 (.99)
57% (.04)
50% (.02)
46% (.04)
37% (.10)
29% (.10)
14% (.10)
24% (.04)
41% (.05)
18% (.05)
72% (.06)
17% (.03)
95% (.02)
35% (.15)
59% (.05)
21% (.04)
81% (.03)
47% (.05)
11
Propensity Score
Propensity score = association between covariates
and group membership
12
Selection into ZTT Court Teams
How child selected within a site:
- Child under age of 3 at time of entry into child welfare
system
- Either all young children assigned to Court Teams judge or
randomly assigned
13
Predictors of Time to Permanency
Child
Characteristics
Reasons for
Removal
Parent
Characteristics
Community
Characteristics
14
Aim 1
Determine the effect of the ZTT Court Teams
project on time to permanency.
15
Aim 1
2 Outcome Measures:
– Time to “move in” permanency
– Time to “official” permanency
Methods:
– Time to event analysis with propensity score weights
– Test for proportional hazards
– Diagnostics
16
Aim 1
Time to “move in” permanency
Permanency
Move In
ZTT
NSCAW
(n=298)
(n=511)
Median (SE)
168 (10.5)
210 (25.6)
Mean (SE)
343 (32.1)
398 (23.4)
ZTT cases move into what becomes the permanent home
1.5 to 2 months faster on average
17
Aim 1
Time to “move in” permanency
Kaplan Meier
Kaplan-Meier survival estimates: Time to Move in Permanency
0.00
0.25
0.50
0.75
1.00
-
0
500
1000
1500
2000
Days until move into what becomes the permanent home
ZTT Court Teams
-
NSCAW
Time to Event Analysis
Parametric model (Gompertz): HR = 1.06 (CI .67, 1.65) p=.81
18
Aim 1
Time to “official” permanency
Permanency
Official
ZTT
NSCAW
(n=298)
(n=511)
Median (SE)
383 (10.8)
762 (26.8)
Mean (SE)
466 (20.5)
825 (18.5)
ZTT cases exit foster care one year earlier on average
19
Aim 1
Time to “official” permanency
Kaplan Meier
0.25
0.50
0.75
1.00
Kaplan-Meier survival estimates: Time to Official Permanency
0.00
-
0
500
1000
1500
2000
Days until officially discharged from foster care
ZTT Court Teams
NSCAW
20
Aim 1
Time to “official” permanency
- Time to Event Analysis
Parametric model (Weibull): HR = 2.67 (CI 1.65, 4.31) p=.000
ZTT cases exit foster care nearly three times as fast as the
comparison group
21
Aim 2
Assess the influence of the ZTT Court Teams
program on how children exit the
foster care system.
22
Aim 2
4 ways to exit foster care:
- Reunification
fastest
- Adoption
slowest
- Relative custodianship
- Non-relative legal guardian
23
Aim 2
Type of
Foster Care Exit
ZTT Court Teams
NSCAW sample
(n=298)
(n=511)
pct
se
pct
se
Reunification
37.6%
(.047)
29.3%
(.042)
Adoption
15.4%
(.059)
40.7%
(.045)
Relative custodian
24.8%
(.085)
8.4%
(.033)
Non-relative guardian
3.0%
(.017)
1.6%
(.007)
Still in foster care at end
of study period
19.1%
(.068)
20.1%
(.036)
24
Aim 2
Days until Exit Foster Care, by Type of Exit
Type of Exit from
Foster Care
Reunification
ZTT
NSCAW
(n=298)
(n=511)
Median
Mean
309
340
547
587
8 months faster on average
Adoption
Median
Mean
464
496
764
800
10 months faster on average
Relative custodian
Median
Mean
351
363
450
487
3-4 months faster on average
Non-relative guardian
Median
Mean
481
467
878
780
10-13 months faster
on average
Patterns hold when use propensity scores in a competing risks analysis
25
Aim 3
Examine successful and unsuccessful cases to
understand how program components
and/or client characteristics contribute to
time to permanency.
26
Aim 3
Examine successful and unsuccessful cases to
understand how program components
and/or client characteristics contribute to
time to permanency.
27
Aim 3
Methods
• One-on-one open-ended phone interviews with each
community coordinator; 2 interviews each
• Approach orientated to unique cases– discuss top and
bottom decile in each site (Patton, 1990)
• Community coordinator’s perception of how each
component of the program model contributed time to
permanency for each case
• Record and transcribed calls
• Coded using Atlas.ti
28
Aim 3
Parents comply with service plan?
Yes
Reunification
“[The mother] is absolutely amazing that, what is she, 20 years old, she had two children
with special needs that really require a lot of doctors visits. So she was able to show all
the professionals in the system that she was able to follow through with all of these
things. And at the same time, she was dealing with her own victimization issues and poor
relationship choices…..The children were [ultimately] reunified with the mother.”
29
Aim 3
Parents comply with service plan?
Yes
Reunification
No
Terminate/Surrender/Suspend
Parental Rights
Adoption
Legal Guardian
“The mother would come into court every time positive for cocaine, refused to get any
help….. She didn’t approach it [the service plan]. She just didn’t do anything….This child was
30
only placed in one foster home… [who] adopted the child.”
Aim 3
Parents comply with service plan?
Yes
Reunification
Somewhat
No
Terminate/Surrender/Suspend
Parental Rights
Adoption
Legal Guardian
“An inpatient drug treatment program was recommended for her. She went to one and she
completed it. And then when she came out of course they recommended outpatient care and
AA meetings and so forth. She started taking those and then slacked off. So that was a
discussion in every hearing. Was she or was she not in compliance with the aftercare
31
recommendation….”
Aim 3
Parent attitude/willingness
Parent behavior :
- substance abuse issues
- mental health issues
- relationship issues
Parent personal resources:
- education/job skills
- poverty
- transportation
Parents comply with
service plan?
Yes
Reunification
Somewhat
Judge:
- monitor case progress
- verbal support when parents comply
- verbal warning when parents not complying
- timekeeper
Department of Social Services:
- design service plan
- assistance seeking services
- visitation supervision
No
Service providers:
- attempt to meet needs, teach
skills, stabilize condition
Terminate/Surrender/Suspend
Parental Rights
Adoption
Legal Guardian
SOCIAL SUPPORT INFLUENCES
Family, friends, foster parents, and others to provide:
transportation, housing, visitation supervision, financial support, care for children, emotional support, etc.
32
Aim 3
Role of Judge in Time to Permanency:
• Motivate the parents
“[The judge] was very supportive and really wanted the children with their
mother. And you know basically would encourage her and would actually
praise her and tell her she was doing a good job and tell her to keep it
up…assuring her we were going in the right direction… [The judge] is very
good about praising when you’ve made progress on your service plan and
you’re doing what you’re supposed to do.”
“[The judge] became increasingly frustrated… and really confronted the
mom on how the professionals were working harder than she was and her
inability to take ownership for her part that she played in this. And
confronting mom on not understanding the impact it had on the children.”
33
Aim 3
Role of Judge in Time to Permanency:
• Motivate social support network
• Motivate the case workers
• Role as timekeeper
34
Aim 3
Role of Monthly Case Review in Time to Permanency:
•
Keeping all on task
“Everybody stayed on task because they knew we were gonna be staffing and we
were going to be in court. So there was no room for making, for example, making
a referral a week before we go to court because we were always going to court.”
“Usually CPS cases, the hearings are every 90 days. But with this, the parents
know that they have to be in court every month. It gets them motivated to get on
the ball so they don’t have to go to the judge in 30 days to explain to the judge
why they haven’t done what they are supposed to have done 30 days prior. “
• Monitor parental compliance
35
Aim 3
Fast v. Slow case comparison
• Program is consistent across the cases
- All have monthly case reviews
- Judge has equal contact with cases
36
Aim 3
Activity to Strengthen Results
Reliability Validity
Within a Site
Used same questions 5 to 6 times in one interview re: fastest cases

Used same questions 5 to 6 times in a second interview on a
different day re: slowest cases

Used same questions for fast and slow cases

Validation interviews – with one other professional in each site

Negative case analysis to search for examples contrary to findings

Across Sites
Asked same questions across the sites a total of 46 times

Compared findings to the James Bell Associates Court Teams
evaluation

Mixed methods – quantitative analysis

37
Conclusion
• The ZTT Court Teams program reduces time to official
permanency, but not time to move in permanency
• More kids are exiting foster care through reunification and
less through adoption
• Though – ZTT Court Teams cases are exiting faster
regardless of type of exit
• The program operates consistently for both “fast” and
“slow” cases
• The judge and the monthly case review components appear
to be most linked to time to permanency
38
Limitations and Next Steps
Quantitative study
• Propensity score subject to omitted variable bias
• Only considers child’s first episode in child welfare
• Assess effect of number of court hearings
• Are all types of exits equal?
• Next step – cost effectiveness study
39
Limitations and Next Steps
Qualitative study
• Only one person coded and analyzed data
• Sorted by time to move in permanency
• Collected before completed quantitative analysis
• Site differences – fidelity to the model?
40
Reflections
Judge Constance Cohen
Polk County, Iowa
Judge Ernestine Gray
Orleans Parish, Louisiana
Judge Michael McPhail
Forrest County, Mississippi
41