Identify Research to Support Safety
Assessments.
What are other states doing?
1.
2.
◦
◦
◦
◦
Studies that show state instruments are valid and
reliable?
Have states looked at the “process” or time
involved with safety assessments.
Has safety process effected “face to face” time
with families.
Do states have outcomes on their research?
Research shows workers:
1. May ignore the probability of the abuse.
2. Could be overconfident in our skills.
3. Could have difficulty weighing all the
factors.
4. Have been shown to make different
decisions under the same conditions.
5. Are reluctant to revise their assessments
once made.
6. Display skepticism over new information.
(Evidence for Practice , An Executive Summary. U.C. Berkley, 2005)

Consensus Based Models
used in more than one location).
1. Child at Risk Field (CARF) Action for Child Protection
2. Washington Risk Assessment Matrix (WRAM).
3. California Family Assessment Factor Analysis (Fresno
Model)
4. Child Emergency Response Assessment Protocol (Illinois
Model)
5.

(Included are only those models
◦
*Family Assessment Response (After a safety Assessment)
Actuarial (SDM)
Children’s Research Center – Structured Decision Making.
In all Models Safety Assessments
are treated as part of Risk Assessment
(except PA and NC)

“Safety” represents a point in time which is

Designed to improve clinical judgment &
decisions.

Attempt to “standardize” a process for
decision making.

Observe family behaviors/functioning
though lists of expected behaviors.
valid & reliable only for that moment.


Examine the ability of the caretaker to protect
the child (parents, foster parents, etc…, )
Recognize the role and importance of the
extended family to assist in safety.

Examine current situations that affect safety.

Agree that Safety and Risk Assessments are not

Both see risk and safety assessment as part of an
ongoing problem solving process.
needed with each contact.
Referral
Screen
Investigation
Assessment
Reassess
Safety/Risk
Intervention
Planning
Implementation
Evaluation
Follow-up
Ongoing
reassessment of
problems/issues/
theories.
Other
services
Close Case



Safety is based on
information gathered
during the assessment.

Construct validity

More emphasis on the
caretaker than the extended
family (Action for Child
Protection 2007).

http://www.actionchildprotection.
org/
Consensus Based
Safety is based on
information gathered
during the assessment but
variables proven to be valid
and reliable are given a
score.
Construct and empirical
validity (strong statistical
validity behind the
questions).
Statistically focuses
questions to reduce worker
error. (Shlonskya A. &
Wagner 2005)
Actuarial


Child is “safe” or
“unsafe” (less emphasis
on levels of risk)
Child Safety is
“Dynamic”



Consensus Based
Less subject to bias,
decisions preferences
and ethical concerns.
Levels Safety and risk
can be predicted.
It’s a reliable and valid
checklist
Actuarial Models







Take a comprehensive approach.
Items are based on maltreatment theories.
Items often shared across different
instruments or forms.
Sometimes numerical scores are given.
Tend to use a single tool for all types of
maltreatment reports.
Can structure information for clinical
assessments of risk.
Help document information for decisions.


Some argue that more information equates to
better decisions.
Criticized for:
◦ poorly defined measures (nebulous, ambiguous,
subjective).
◦ Inconsistency in types of variables
◦ Use same variables to predict all types of abuse,
neglect, sexual abuse.
◦ Less weight given to recurrence of maltreatment.
◦ Reliance on variables for which there is no research.
Evidence for Practice , An Executive Summary. U.C. Berkley, 2005)







Less bias (Fontes, L.A. (2008).
Use statistics to weigh factors to predict future.
Often statistical analysis is done in locality where
the instrument is used.
Use fewer factors than Consensus Based.
Factors are scored and summed into an overall
risk score.
Families are rated low, med, high and receive
different service responses.
More reliable and valid questions.
Raelene Freitag, Director Children's Research Center (2011) Email correspondence.

Criticized for:
◦ Not using or curtailing the clinical judgment of the
worker.
◦ Basis for judgment on a factor that is statistically
associated with recurrence of maltreatment, and may
not appear to be causally related to the outcome. This
may cause caseworkers to discount the value because
they can’t understand the theory, math or reason behind
the score.
Evidence for Practice , An Executive Summary. U.C. Berkley, 2005)
The research literature clearly favors
Actuarial Models over Consensus Based.
“Settled Science”




Rittner, B., Children and Youth Service Review, Vol. 24 No. 3, March 2002
pages 189 – 207. (Compared Mich’s actuarial model to Washington’s and
Fresno California’s CB models).
Evidence for Practice, Risk and Safety Assessment in Child Welfare: Instrument
Comparisons, No. 2 July 2005. Argues for more research.
Johnson, W. (2011). The validity and utility of the California Family Risk
Assessment under practice conditions in the field: A prospective study. Child
abuse & neglect, 35, 1, 18-28.
D’Andrade, A., Austin, M.J., & Benton, A. (2008). Risk and Safety Assessment in
Child Welfare: Instrument Comparisons. Journal of Evidence-Based Social Work
5 (102) 31-56.

Alaska – Did have full system but may have switched to Action

Arkansas – Says the are implementing

California – Some counties

Colorado -

Connecticut

DC

Florida

Georgia

Indiana – One of the first states, currently updating their model.
Used in el Paso County only.
– Some communities (not statewide) Family Assessment Response?

Kansas - Two private agencies (Managed Care system?)

Louisiana – Using the model

Massachusetts – Most of system (Union is fighting)

Michigan – One of the first states.

Minnesota – Family Assessment Response State - One of the first states

Missouri – Uses parts of SDM Model

Nebraska (KVC)

New Hampshire

New Jersey

New Mexico - Began several years ago.

New York – Family Assessment Response

Ohio – Incorporated into Competency Based Training.

Rhode Island

Washington – SDM risk assessment only

Wisconsin (some counties)

Utah – Just started

Vermont

Virginia - Family Assessment Response
Raelene Freitag, Director Children's Research Center (2011) Email correspondence.





New Brunswick is just
starting
Manitoba has part of
system
Ontario has risk
assessment and may be
using more
Saskatchewan is starting
British Columbia has risk
and may have picked up
others

Bermuda

Queensland

South Australia

New south wales

Northern Territories (in
roll out)
Raelene Freitag, Director Children's
Research Center (2011) Email
correspondence.








Alabama – Currently doing a pilot of the model.
Alaska (has not been confirmed)
Hawaii
Maryland (at the University of Maryland with a
demonstration project)
Texas
South Dakota
West Virginia
Wisconsin
http://www.actionchildprotection.org/

Arizona

Delaware (Focused Safety Assessment) (PIP)

Georgia

Idaho (Safety Assessment tool) (PIP)

Illinois - Child Emergency Response Assessment Protocol
(CERPA), the model of PA’s risk assessment. Improved
outcomes with use but no reliability and validity. (Evidence for
Practice , An Executive Summary. U.C. Berkley, 2005) and A Comparison of Approaches
to Risk Assessment in Child Protection and Brief Summary of Issues Identified From
Research on Assessment in Related Fields Child Welfare League of America, 2005)


Iowa
Kentucky (Family In Need Assessment) Family Assessment
Response. Very detailed and specific with a great deal
legislated.

Maine

Missouri (Part of Action)

North Carolina - Family Assessment Response. ** Very
detailed Safety Assessment Directions.

South Carolina - Family Assessment Response.
http://www.dss.mo.gov/cd/info/cwmanual/section2/ch9/sec2ch9.pdf
WRAM
FRESNO
CARF
Wash. St. Cal. Model Action
CERAP
Illinois
SDM
Actuarial
Scales
7
Domains
5
Domains
5
Domains
ITEMS
37
23
14
14
20
Predictive
Validity
Poor
Poor
Little
Some
Highest
Convergent
Validity
Poor
No
Research
Poor
No
Research
No
Research
Inter-rater
Reliability
Some
but poor
No
Research
None
No
Research
Yes - ok
Outcomes
No
No
Research Research
None
Yes
Yes –
good.
Racial/
Ethnic
Mixed –
good
No
Research
No
Research
Mixed –
good.
No
Research
3 Scales (1
safety)

SDM and Family Group Decision Making –
Minnesota, SDM News, Issue 24. Feb. 2011.

Massachusetts looked at SDM. About half the
workers thought the processes was helpful, half
did not. Raelene F. & Healy T. (2008) Massachusetts Department of Children
and Families Assessment Field Test Results



CARF (Action Model): Favored by less
experienced workers (Doueck et al. 1993).
Illinois – Found workers inflated scores to get
cases accepted for service. (Lyle & Graham 2000).
Assessment forms often missing.
(Lyle and Graham 2000).




Quality of implementation more important than
actuarial or consensus-based.
Consensus that high quality training is needed.
All models need supervisory and management
support
Need supervisors and workers engaged in planning
and implementation.
(Evidence for Practice , An Executive Summary. U.C. Berkley, 2005)
But in California a county process model must:
1. Determine a response to a referral.
2. Make an initial safety decision.
3. Assesses safety/risk in placements.
4. Make a referral disposition.
5. Be used in ongoing case planning decisions.
6. Be used in reunification
7. Be used in case closure.
(Evidence for Practice , An Executive Summary. U.C. Berkley, 2005)


Action is home to the
National Resource
Center for Child
Protective Services.
Consensus Base
qualitative research
Action for Child
Protection


CRC (SDM) is a nonprofit affiliated by the
National Council on
Crime Delinquency.
Actuarial models
quantitative research.
Children’s Research
Center

What if the baseline isn’t working?
 REASSESS THE MEASURE

Action for Child Protection (no author listed) (2007) The Safety Decision.

http://www.actionchildprotection.org/



http://www.actionchildprotection.org/PDF/May_2007_The_Safety_Decision.pdf
Child Welfare League of America (no author) (2005) A Comparison of Approaches
to Risk Assessment in Child Protection and Brief Summary of Issues Identified
From Research on Assessment in Related Fields, A report prepared for
Pennsylvania.
D’Andrade, A., Austin, M.J., & Benton, A. (2008). Risk and Safety Assessment in Child Welfare:
Instrument Comparisons. Journal of Evidence-Based Social Work 5 (102) 31-56.
Doueck, H.J. English, D. DePanfalis, D. & Moote, G. (1993). Decision-making in
child protective services: A comparison of selected risk assessment systems.
Child Welfare, 72(5), 441-52.

Evidence for Practice , An Executive Summary. U.C. Berkley, 2005.

Fontes, L.A. (2008). Assessment Instruments and Structured Decision Making in Child
Abuse and Culture. New York: Guilford Press.

Freitag, R. & Healy T. (2008) Massachusetts Department of Children and Families

Freitag, Raelene (2011) Director Children's Research Center, email
correspondence.
Assessment Field Test Results


Johnson, W. (2011). The validity and utility of the California Family Risk Assessment under
practice conditions in the field: A prospective study. Child abuse & neglect, 35, 1, 18-28.
Lyle C. G. & Graham E. (2000). Looks can be deceiving: Using a risk assessment
instrument to evaluate the out-comes of child protection services. Children and
Youth Services Review, 22(11/12), 935-49.

New York OCFS Family Service Review
http://www.ocfs.state.ny.us/main/cfsr/far.shtm

North Carolina DSS On-line manual

Rittner, B., Children and Youth Service Review, Vol. 24 No. 3, March 2002 pages
189 – 207.

SDM News, (no author listed) Issue 24. Feb. 2011.

Shlonskya A. & Wagner D. (2005) The next step: Integrating actuarial risk
assessment and clinical judgment into an evidence-based practice framework in
CPS case management, Children and Youth Services Review, 27 (2005) 409– 427.








Baird, C., Wagner, D., Healy, T., & Johnson, K.(1999). Risk assessment in child protective services:
Consensus and actuarial model reliability. Child Welfare, 78(6), 723-748.
Johnson, K., Caskey, R., & Wagner, D. (2003). Addendum to the evaluation of Michigan’s foster care
Structured Decision Making case management system. Madison, WI: Children’s Research Center.
http://www.nccd-crc.org/crc/pubs/mi_2002_fc_eval_addendum.pdf
Johnson, K. (2005). A retrospective support assessment study of foster and relative care providers.
Madison, WI: Children’s Research Center. http://www.nccdcrc.org/crc/pubs/fcrp_support_assmnt_sept05.pdf
Johnson, W. (2004). Effectiveness of California’s child welfare Structured Decision Making® model: A
prospective study of the validity of the California family risk assessment. Sacramento, CA: California
Department of Social Services. http://www.nccd-crc.org/crc/pubs/ca_sdm_model_feb04.pdf
Lee, S., Aos, S., & Marna Miller, M. (2008). Evidence-based programs to prevent children from
entering and remaining in the child welfare system: Benefits and costs for Washington. Olympia:
Washington State Institute for Public Policy, Document No. 08-07-3901.
Loman, L. A., & Siegel, G. L. (2004). An evaluation of the Minnesota SDM Family Risk Assessment.
Institute of Applied Research. www.iarstl.org/papers/FinalFRAReport.pdf
Rossi, P., Schuerman, J., & Budde, S. (1996). Understanding child maltreatment decisions and those
who make them. Chicago: Chapin Hall Center for Children, University of Chicago.
Rycus, J. S., & Hughes, R. C. (2003). Issues in risk assessment in child protective services: Policy white
paper. Columbus, OH: North American Resource Center for Child Welfare. http://www.nccdcrc.org/crc/pubs/ra_issues_whitepaper_2003.pdf









Rycus, J. S., & Hughes, R. C. (2007). Issues in risk assessment in child protective services. Journal of
Public Child Welfare, 1(1), 85-116.
Stewart, A. & Thompson, C. (2004). Comparative Evaluation of Child Protection Assessment Tools.
Queensland: Griffith University.
Wagner, D., Johnson, K., & Caskey, R. (2003). Evaluation of Michigan’s foster care Structured Decision
Making case management system. Madison, WI: Children’s Research Center.
http://www.nccd-crc.org/crc/pubs/mi_2002_fc_eval.pdf
Wagner, D. & Johnson, K. (1999). Using actuarial risk assessment to target service interventions in
pilot California counties. 13th National Roundtable on CPS Risk Assessment. http://www.nccdcrc.org/crc/pubs/13th_roundtable_ca_risk.pdf
Wagner, D. & Meyer, B.L. (1998). Using actuarial risk assessment to identify unsubstantiated cases for
preventative intervention in New Mexico. 12th National Roundtable on CPS Risk Assessment.
http://www.nccd-crc.org/crc/pubs/nm_1998_roundtable_risk.pdf
Wagner, D., Hull, S., & Luttrell, J. (1995). The Michigan Department of Social Services risk based
Structured Decision Making system: An evaluation of its impact on child protection service cases.
Ninth National Round Table on CPS Risk Assessment.
http://www.nccd-crc.org/crc/pubs/mi_1995_cps_eval.pdf
Wood, J.M. (1997, April). Risk predictors for re-abuse or re-neglect in a predominantly Hispanic
population. Child Abuse & Neglect. 21 (4) 379-389.