John D. Fluke, Ph.D.
Director
Child Protection Research Center
American Humane Association
The Second Canadian Roundtable on Child Welfare Outcomes
Center for Research on Children and Families, Centres of Excellence
for Children’s Well-Being: Child Welfare, Montreal, Canada
8 – 9 October, 2009





Brief History of National US Data
Collection
State and National Data
Outcome Policy and Trends: an Example
Building and Maintaining Infrastructure
Sustainability/Overcoming Inertia
The Value of Data/Outcomes Systems
The Second Canadian Roundtable on Child
Welfare Outcomes

Early Studies in the US

Other Studies of Incidence

Major US National Data Collection Activities
◦ Incidents Extracted from Media Reports (1950s – 1970s)
◦ National Reporting Study on Child Abuse and Neglect
 1976 – 1988
◦ Strauss and Gelles
◦ Finkelhor
◦ Prevent Child Abuse America
◦ National Child Abuse and Neglect Data System (NCANDS)
 1988 -
The Second Canadian Roundtable on Child
Welfare Outcomes

Early program conducted by the American Public Human Services
Association (then the American Public Welfare Association), Voluntary
Cooperative Information System (VCIS)
◦ Established 1982
◦ Aggregate data only

Chapin Hall Center for State Foster Care and Adoption Data (State
Data Center) and the Multistate Foster Care Data Archive
◦ Established circa 1989 (three states)
◦ Case level multi-year longitudinal data
◦ 20 States are now participating

Federal Adoption and Foster Care Analysis and Reporting System
(AFCARS)
◦ Regulations published in 1993 for mandatory reporting
◦ First data collection 1995
The Second Canadian Roundtable on Child
Welfare Outcomes

Other Major National Data Collection
and Analysis Programs
 National Incidence Study (NIS)
 Four Studies to Date (first released in 1980, most recent data
to be released 2009?)
 National Study of Child and Adolescent Well Being
(NSCAW)
 Two cohort cycles of data collection (1999 & 2008)
 Multiple rounds of data collection up to a seven year followup
 Chafee National Youth in Transition Database (NYTD)
 Under development (final regulations published)
The Second Canadian Roundtable on Child
Welfare Outcomes

Adoption and Safe Families Act of 1997 P.L. 96-272
◦ First to Identify Safety, Permanency and Well Being as Outcomes
of the US Child Welfare System
◦ Basis for Outcomes Regulations and the Child and Family
Services Review Process (CFSR)

State Automated Child Welfare Information System (SACWIS)
◦ Enhanced funding available for state child welfare agencies
enacted in 1993 under Title IV-E funding provisions
◦ Originally 75% match by US govt., as of 1997 at 50%
◦ Critical impetus for the feasibility of data collection (e.g.,
NCANDS, AFCARS)

Fostering Connections Act of 2008
◦ Improvements to kinship & subsidized guardianship regulation
◦ Allows tribes direct access to Title IVE funding
The Second Canadian Roundtable on Child
Welfare Outcomes

Several States and Localities Have Strong Data
Collection and Analysis Programs Coupled with
Research Centers
◦ Examples:
California
Illinois
North Carolina
Texas
Washington

All Use Data for Program Management and
Outcomes, but add contextualizing analysis
The Second Canadian Roundtable on Child
Welfare Outcomes
National Child Abuse and Neglect Data
System (NCANDS)




Meets US legal requirements to collect child
maltreatment data
Provides data that are critical to policymakers and
administrators of CPS programs
Data are used to support a range of US initiatives,
notably
◦ ASFA Outcome Measures
◦ Child and Family Services Review Process
◦ Office of Management and Budget Program
Assessment Rating Tool (PART)
Contribute to the understanding of child
maltreatment and the impact of intervention policy
The Second Canadian Roundtable on Child
Welfare Outcomes

Facilitators of Case Level Data Collection in
the US
◦ US Legislation
◦ Related Programs
 State Automated Child Welfare Information Systems
(SACWIS)
 Child and Family Service Review Process (CFSR)

System has evolved
◦ Almost all States are Now Providing Case Level Data
(48 in 2007)
◦ Data are timely and available annually
◦ Range of applications has expanded dramatically
The Second Canadian Roundtable on Child
Welfare Outcomes

Action (regulations, policy, programs)
◦ Data collection program is authorized by US Office of
Management and Budget (OMB)
◦ Supports US HHS Children’s Bureau Child and Family
Services Review Process (CSFR)
◦ Support Children’s Bureau OMB Program Assessment Rating
Tool (PART) measures

Evaluation
◦ Contract is competitively bid by the US HHS Children’s
Bureau

Feedback loop
◦ Project convenes a State Advisory Group each year
◦ Data program is reviewed for revision every three year by
the US Office of Management and Budget (OMB)
The Second Canadian Roundtable on Child
Welfare Outcomes

Longitudinal
◦ Aggregate data since 1990; case level data since 1993.
◦ Encrypted unique child and perpetrator identifiers.

Time Period
◦ Data are collected annually

Mapping and Validation of Data
◦ Data are mapped to national categories
◦ Data are validated for coding, internal consistency, and
analytical accuracy and comprehensiveness
The Second Canadian Roundtable on Child
Welfare Outcomes

Measures of Maltreatment
◦ Six major categories (Physical, neglect, medical
neglect, sexual, emotional, other).

Analytic Environment
◦ Comprehensive Data Quality Assessment Process
◦ OLAP Reporting Tools
◦ Specialty Ad-hoc Research and Analysis Capacity
The Second Canadian Roundtable on Child
Welfare Outcomes

In August 2001, the US Children’s Bureau set
the first national standard for recurrence :
◦ A State meets the national standard if of all children
who were victims of substantiated or indicated child
abuse and/or neglect during the first 6 months of
the period under review, 6.1 percent or fewer
children had another substantiated or indicated
report within 6 months.
The Second Canadian Roundtable on Child
Welfare Outcomes

As of June 2006 the Standard was revised
◦ Of all children who were victims of a substantiated
or indicated maltreatment allegation during the first
6 months of FY 2004, what percent were not
victims of another substantiated or indicated
maltreatment allegation within the 6-months
following that maltreatment incident?
◦ National Standard Based on 45 States: 94.6% or
higher
The Second Canadian Roundtable on Child
Welfare Outcomes
The Second Canadian Roundtable on Child
January
18, 2007 Outcomes
Welfare
EUROPEAN SEMINAR ON MONITORING
SYSTEMS OF CHILD ABUSE


In addition to Indicators Based on Federal Data Collection Programs
a Sample of Cases is Reviewed to Address Other Standards (30 to
50)
States not meeting standards must implement a 3 year Program
Improvement Plan (PIP) approved by the US government
◦ Plans address a range of programmatic and intervention strategies, for
example:







Multiple maltreatment and chronic neglect
Alternative response systems
Safety and risk assessment systems
Required child and family visits for placed children
Large Investment in Federal Resource Centers to Assist States in
Developing and Implementing PIPs
States not in compliance may be subject to penalties
After each three year cycle the US standards are reviewed and
revised.
The Second Canadian Roundtable on Child
Welfare Outcomes

Stages or Transitions
Aggregate Data
to Case
Crosssectional Data
to Longitudinal
•not much can be done with aggregate
data
•transition occurred over many years in
US
•improvements in ID construction and
data management
•sustainability of program yields multiyear data
•still struggling to achieve this in the US
The Second Canadian Roundtable on Child
Welfare Outcomes

Data Analysis/Utilization Stages or Transition
Fiscal to Outcomes Analysis
Program Specific to Cross
Program Analysis
Single to Multiple Agency
Analysis
General Program Information
to Specialized Studies
•includes the incorporation of
special data collection
The Second Canadian Roundtable on Child
Welfare Outcomes
Percent on
Caseload
Workload
(worker)
Disparity Index
+
AA
Only
Skills (AA only)
Removals
+
Community
Resources
(worker)
Family Poverty
+
Family Risk Level
The Second Canadian Roundtable on Child
Welfare Outcomes

Infrastructure
◦ Enabling Legislation and Regulatory Framework
for Data
◦ Strong Information Technology and Operational
Systems Architecture
 Key Analytic Design Components (e.g., OLAP capacity)
◦ Data Quality Improvement Processes
 Research/Evaluation Capacity
 Mechanisms for insuring data integrity
 Documentation
The Second Canadian Roundtable on Child
Welfare Outcomes

Sustainability
◦ Recognition of Long-Term Benefits/Minimal Short
Term Expectations
◦ Long Term Funding
◦ Data use agreements and confidentiality
◦ Workforce

Dissemination
◦ Archives
◦ Reports and Publications
◦ Technical Assistance
The Second Canadian Roundtable on Child
Welfare Outcomes
“Total physical and mental inertia are highly agreeable, much more
so than we allow ourselves to imagine.” – John Kenneth Galbraith

Inertia Avoidance
◦ Key Issues:
Inflexible Regulatory
Frameworks
Improvements in
Knowledge
Obtaining Buy-In
Balancing Input with Action
Data Stability vs. Change
Funding
Unintended Consequences
The Second Canadian Roundtable on Child
Welfare Outcomes

Value of outcome data supporting policy:
◦ Problem Recognition/Definition & some examples
 Re-entry (rereporting, recurrence, recidivism, foster care
reentry, disruptions, etc.)
 Racial Disproportionality and Disparity
◦ Standard Setting
 Child and Family Services Review Standard Setting
 Information System Guidelines (SACWIS)
◦ Monitoring
 Child and Family Services Review
 Alternative Response Implementation
The Second Canadian Roundtable on Child
Welfare Outcomes

Current and
timely policy
support

◦
◦
◦
◦
◦ Data Turnaround
◦ Rapid data quality
assessment

Continuity
◦ Ability to monitor
trends
◦ Outcomes and
performance
Research
Longitudinal design
Special populations
Infrequent events
Decision making
 Simulation
◦ Sampling frames

Assess Resources
◦ Staffing
◦ Workload
◦ Costing/Cost
Effectiveness
Contact
John D. Fluke
Director
Child Protection Research Center
American Humane Association
USA
(303) 810 1934
johnf@americanhumane.org