Child Abuse Review (2019)
Published online in Wiley Online Library
(wileyonlinelibrary.com) DOI: 10.1002/car.2547
Can Predictive
Algorithms Assist
Decision-Making in Social
Work with Children and
Families?
Philip Gillingham*
School of Nursing, Midwifery and
Social Work, University of Queensland,
Brisbane, Queensland, Australia
Decision support systems (DSS) which incorporate algorithms trained using
administrative data have been promoted as the next promising development in initiatives
to improve and assist decision-making in social work with children and families. In this
article, research and grey literature about DSS designed, and in some cases
implemented, in the USA, the Netherlands, Australia and New Zealand are analysed
to assess the extent to which they are currently able to meet this promise. The
challenges of developing DSS for social work are identified and ideas are suggested
to overcome them. © 2019 John Wiley & Sons, Ltd.
KEY PRACTITIONER MESSAGES:
• Decision support systems (DSS) are being developed for social work with children
and families and they may transform how decisions are made with important
consequences for both service users and the profession.
• Current DSS are mostly not very accurate in their predictions and further work is
required in their development and on the data used to train the algorithms.
• The next stage in the development of DSS will require long-term commitment by
social workers to work in collaboration with data scientists and researchers. It will
also be expensive. Social workers need to lead the debate about whether DSS
are worth the investment.
KEY WORDS: decision support systems; big data; predictive analytics
here is a plethora of research about decision-making in social work with
children and families. Much of this research illustrates the influences on
how social workers make decisions, which may be demographic, personal
or organisational, leading to the conclusion that decision-making is a highly
subjective and potentially biased process (Buckley, 2003; Keddell, 2013;
Stanley, 2005). Decision-making tools based on blends of actuarial
and consensus-based approaches to risk assessment were introduced in
T
*Correspondence to: Dr Philip Gillingham, University of Queensland, School of Nursing, Midwifery and
Social Work, St Lucia Campus, Brisbane, Queensland 4067, Australia. E-mail p.gillingham@uq.edu.au
© 2019 John Wiley & Sons, Ltd.
Accepted: 27 October 2018
‘Decision support
systems are being
developed for social
work with children
and families and they
may transform how
decisions are made’
Gillingham
‘Big data approaches
… have been
successful in
informing decisions
about resource
allocation in
healthcare and
health promotion’
© 2019 John Wiley & Sons, Ltd.
many jurisdictions to increase accuracy and consistency in decision-making,
in response to both media and official reports about child deaths and
the perceived failings of child protection services (Reder and Duncan, 2004;
Schwalbe, 2004). While such tools may appear sound theoretically,
in practice their use may not promote consistency and practitioners may
not find them useful (Gillingham, 2009). Whether decisions should be
made ‘subjectively’ on a case-by-case basis or more ‘objectively’ using
tools is a debate that continues, linked as it is to wider debates
about the very nature of social work as a profession (Devlieghere and
Roose, 2018).
Big data approaches, which use algorithms to analyse or mine large
datasets, have been successful in informing decisions about resource
allocation in healthcare and health promotion. It has been proposed that such
approaches to assist decision-making may be the next big development in
child and family social work, following on from actuarial risk assessment
(Macchione et al., 2013). Specifically, big data approaches have been
promoted in the development of decision support systems (DSS). DSS can
be differentiated from actuarial and consensus-based decision-making tools,
such as Structured Decision Making (CRC, 2018) and the Illinois Structured
Decision Support Protocol (Kang and Poertner, 2006). DSS incorporate
algorithms that have been trained to calculate the likelihood of a particular
outcome for a service user and make recommendations to practitioners to
support decision-making. Within the fields of information science or
information technology, this may be referred to as ‘predictive analytics’.
Technological developments such as faster computing power, the
accumulation of data about service users and service delivery in electronic
information systems, and the ability to extract and match data about people
from multiple databases have, over the last ten years, made big data
approaches much more feasible in social work.
The idea of using algorithms to assist and support decision-making in social
work with children and families is not a recent development. For example,
Schoech et al. (1985) explored the potential of what they refer to as ‘artificial
intelligence’ to develop ‘expert systems’ to support professional decisionmaking. Research-based development of DSS continued in the 21st century
and identified particular challenges in their design and implementation such
as making them useful in the contexts in which they are implemented (Fitch,
2006, 2007; Foster and Stiffman, 2009), getting practitioners to use them
(Fitch, 2006, 2007; Foster and Stiffman, 2009) and accounting for the
importance of emotion and bias in human decision-making (MacFadden and
Schoech, 2010).
Despite the challenges that have been identified by research-based
development projects and the identified need for more research (MacFadden
and Schoech, 2010), significant investments have already been made in
implementing DSS in some jurisdictions. It is therefore timely to review where
developments with DSS have led and assess where they might go in future.
Using six examples from North America, Europe and Australasia, this article
provides an overview and critical appraisal of current developments. It is also
important that social workers are aware of these developments and can engage
in both debate about DSS and development processes (Fitch, 2006, 2007). One
aim of the article is therefore to inform social workers and stimulate debate
Child Abuse Rev. (2019)
DOI: 10.1002/car
Decision Support Systems and Child and Family Social Work
within the profession. DSS may have profound consequences for service users
as well as social workers. For example, both social workers and service users
may feel alienated from the decision-making process if DSS are used to
replace, rather than support, human decision-making. Explaining decisions
that are either dictated or even supported by DSS may be more difficult, and
may discourage those affected by decisions from challenging them. Big data
approaches in the social welfare sector, as the examples show, may also be
fallible and unreliable, leading to injustices for service users (see also
Gillingham, 2016a).
The other aim in this article is to use the lessons learned from the examples
provided to suggest strategies for the future development of DSSs. The
language used in this article is deliberately non-technical and aimed at a social
work audience, so for those who are more au fait with the technical and
mathematical bases of big data and machine learning, it may seem overly
simplistic.
‘DSS may have
profound
consequences for
service users as well
as social workers’
Background
The author is engaged in a four-year programme of research supported by an
Australian Research Council Future Fellowship which aims to improve how
electronic information systems are designed and used in the social welfare
sector. An important consideration in this research has been to explore what
can be done constructively with the data that social welfare agencies have
accumulated about service users and service delivery, over and above using
them for reporting purposes. Service evaluation has been one focus
(Gillingham, 2018), but, as stated above, big data approaches now offer other
possibilities.
There is, however, very little published research about DSS and their
application in the social welfare sector. As described above, there are published
articles which describe the process of designing an algorithm, but these
approaches tend to become obsolete very quickly given the speed of
technological developments in big data. DSS are a very recent and emerging
phenomenon in the social welfare sector and using specialist search engines
to identify published academic research yields very little, with the exception
of four out of the six examples analysed in this article. The author had to rely
on ‘insider knowledge’ in finding one of the other examples (Broward County)
and an online network of researchers to find the other (the Rapid Safety
Feedback (RSF) programme). The rationale for including these examples is
that they use a range of different techniques to train algorithms and each, when
analysed and considered alongside the others, provides insights into how DSS
can be improved in the future. The examples may therefore be considered as a
purposive sample (Alston and Bowles, 2012, p. 97).
‘Very little published
research about DSS
and their application
in the social welfare
sector’
Developing and Implementing DSS
Predictive Risk Modelling in New Zealand
The author has published two articles which examine the development of
predictive risk modelling (PRM) in the New Zealand child welfare system
and the following is a summary of the main points raised in them (Gillingham,
© 2019 John Wiley & Sons, Ltd.
Child Abuse Rev. (2019)
DOI: 10.1002/car
Gillingham
‘Concerns were
raised about… the
ethics of using data
collected for one
reason for a
completely different
purpose’
‘It is imperative that
the dataset used to
train an algorithm
accurately
represents what it is
being taught to
predict’
© 2019 John Wiley & Sons, Ltd.
2016a, 2017). PRM was developed by a team at the Centre for Applied
Research in Economics at the University of Auckland (CARE, 2012;
Vaithianathan et al., 2013) as part of a wide range of reforms to the child
welfare system in New Zealand. The team used data from the child welfare,
health, education and public welfare benefit systems to train an algorithm to
identify children at risk of maltreatment at the point at which their parents
began to claim public welfare benefit. Having identified these children, the
aim was to intervene to prevent maltreatment using supportive services.
Further detail about how this was achieved is provided in Gillingham (2016a,
2017) and documents later released by the Ministry of Social Development
(2014a, 2014b) in New Zealand. After much work, the team managed to
develop an algorithm that was 76 per cent accurate and plans were made to trial
and implement PRM.
PRM attracted much media and academic attention. For example, concerns
were raised about: (a) the ethics of using data collected for one reason for a
completely different purpose without the consent of the person who owned
the data; (b) the stigmatisation of children and families identified by PRM;
(c) the legal and moral basis for intervention based on the calculations of
an algorithm; and (d) a lack of transparency about how PRM had been
developed and would operate (Dare, 2013; Keddell, 2015). Eventually, plans
to trial PRM were shelved amid political concerns that children were being
treated as ‘lab rats’. In the meantime, the author had been sent some of
the publicly available documentation about the development of PRM.
Combining knowledge of machine learning with knowledge about the child
welfare system, a serious flaw in how PRM had been trained was identified.
PRM was trained to identify independent variables (child and family traits)
that were associated with the dependent variable of child maltreatment. In
this form of supervised learning, it is imperative that the dataset used to train
an algorithm accurately represents what it is being taught to predict (child
maltreatment). The team at the University of Auckland used data from the
Ministry of Social Development about children who had been in the child
welfare system and whose cases had been ‘substantiated’. The team
understood ‘substantiated’ to mean that maltreatment had been proven to
have occurred. The term ‘substantiated’ though has a different meaning in
child welfare legislation and policy in Australia and New Zealand, and is
part of the threshold criteria for further intervention by welfare services after
an investigation of alleged child maltreatment has been concluded. In New
Zealand, ‘substantiation’ may be applied to children who have been
maltreated, the siblings of those children (whether maltreated or not),
children considered to be at risk of maltreatment and their siblings,
unaccompanied minors seeking asylum and young people needing access
to mental health services. Therefore any dataset of substantiated cases
actually contains many more children who have not been maltreated (those
deemed to be at risk, siblings, young people with mental health problems,
etc.) than who have, making it entirely unsuitable for training an algorithm
to predict maltreatment. In technical language, this mistake amounts to a
‘mislabelling’ of the dataset. It also suggests that the team at the University
of Auckland misunderstood what the dataset represented, and that the
public servants who supplied it lacked knowledge about how supervised
learning works.
Child Abuse Rev. (2019)
DOI: 10.1002/car
Decision Support Systems and Child and Family Social Work
Having noticed this, the author contacted the Ministry of Social
Development in New Zealand and was invited to address senior public
officials; and, in due course, plans to trial PRM were dropped. Subsequently,
full documentation about the development of PRM was released (Ministry of
Social Development, 2014a, 2014b) and the author undertook an analysis of
both the methods and any insights that the development process may have
provided into the child welfare system and those within it. Assuming that
‘substantiation’ (rather than actual maltreatment) is of interest in terms
of prediction, PRM identified three main independent variables that
were considered predictive: poverty, single parenthood and previous
involvement with the child welfare system. None of these factors
are new insights and would be known to anyone working in the child
welfare system.
The development of PRM demonstrated that, at least in the abstract, it is
possible to use administrative data collected routinely for other purposes to
predict, with 76 per cent accuracy, a particular outcome for a group of children.
This means though that PRM would be wrong in one out of every four cases.
To put this another way, the algorithm was trained using data about cases that
were substantiated and these data were created by human decision-making
processes. The use of these data rests on the assumption that the humans
making the decisions made the correct decision every time (100%). At 76
per cent, PRM was only three-quarters as accurate as the human decisionmakers. For human decision-makers in the child protection system, such
inaccuracy in deciding which cases to investigate and at what point children
need to be removed from the care of their parents would not be acceptable. It
also demonstrated the problems that can arise through misunderstandings
about the accuracy of data and what the data actually represent.
‘The use of these
data rests on the
assumption that the
humans making the
decisions made the
correct decision
every time’
The Allegheny Family Screening Tool
The Family Screening Tool (FST) being used in Allegheny (a suburb of
Philadelphia, USA) is another example of a supervised learning algorithm.
Details about its development are scant and the following description is
paraphrased from a frequently asked questions document available on the
internet (Allegheny County, 2017). The FST is designed to help child
protection workers decide whether to screen in, or accept, a notification for
investigation of alleged child maltreatment. Based on data about the child
and his or her family, the FST calculates the likelihood that the child will be
re-notified within two years (the dependent variable) if the notification is
screened in (and investigated) and the likelihood that a child will be re-notified
within two years if the notification is screened out (and not investigated). Again
the emphasis here is on assisting practitioners to target the most vulnerable
children, but alas there is no published research about the extent to which this
has been achieved.
The algorithm is claimed to use over 100 factors about a child, drawing from
a data warehouse that contains data from 29 different sources, including child
protection, mental health, drug and alcohol services, and county court data.
The speed at which it can do this is obviously much faster than any human
could, the implication being that it is much more efficient and accurate than
human decision-making. These claims are of interest because, as mentioned
© 2019 John Wiley & Sons, Ltd.
Child Abuse Rev. (2019)
DOI: 10.1002/car
Gillingham
‘Only three
independent
variables, or factors
about children,
rather than 100 were
found to be
significantly
predictive’
with PRM in the previous section, only three independent variables, or factors
about children, rather than 100 were found to be significantly predictive. As
mentioned below, Wijenayake et al. (2018) found the same finding. While
the amount of data that the FST considers and the speed at which it does
may sound impressive, it may be misleading in that most of the factors it
considers do not contribute to its accuracy. Unfortunately, no information is
publicly available to support or refute this point, but questioning the predictive
ability of different variables, or factors about a child used by an algorithm, is
important in considering its efficacy and usability.
The RSF Programme
‘Overestimated the
likelihood of
maltreatment for
thousands of
children… and
underestimated the
risk in a few cases
where children either
died or were
seriously harmed’
The following is taken from an article by Lindsay Gloor (2017) published in
the Chicago Herald Tribune on the 6 December 2017. Various other mentions
of the RSF programme are made on the worldwide web, but all seem to have
been sourced from the Tribune article. The article reports on how the Illinois
Department of Children and Family Services decided to stop using the RSF
programme because of errors in its predictions. The RSF programme is a
supervised learning algorithm that was trained using child welfare data to
identify children most at risk of harm as they enter the child welfare system,
assigning them a score of up to 100. In short, the RSF programme
overestimated the likelihood of maltreatment for thousands of children, leading
to the department being overwhelmed, and underestimated the risk in a few
cases where children either died or were seriously harmed. For example, the
RSF programme assigned a 100 per cent chance of death or serious injury in
the next two years to 369 children, all under age nine.
The RSF programme was developed by a private company and the algorithm
and exactly how it was developed are not open to public scrutiny, hence it is
difficult to determine why it went wrong. However, given that the RSF
programme appeared to be getting worse at prediction, it would be fair to
speculate that if it used only child welfare data and was periodically (or even
continually) updated with child welfare data, then it may have fallen prey to
sample selection bias. Sample selection bias occurs when the sample of data
being used is overly biased in one or more ways in terms of its characteristics
than what would be expected in the general population (see Berk, 1983).
Within child welfare data, child maltreatment is over-represented as an
outcome and so the algorithm learns to associate all sorts of factors or
characteristics with child maltreatment. In the general population, these
characteristics may have no association with child maltreatment. Over time,
this bias increases, leading to increasingly erroneous estimations of risk.
Predicting Recidivism in Domestic Violence
Wijenayake et al. (2018) describe how they developed an algorithm using
publicly available data to predict the likelihood that men charged with crimes
related to domestic violence will offend again. This example is notable for a
number of reasons. The team used a decision tree approach to develop the
algorithm and provide a clear and accessible explanation of the process, which
is in contrast to some articles that, through their presentation of the details of
quantitative analysis, would only be accessible to academics with considerable
© 2019 John Wiley & Sons, Ltd.
Child Abuse Rev. (2019)
DOI: 10.1002/car
Decision Support Systems and Child and Family Social Work
prior knowledge. More generally, in the Australian context where the algorithm
was developed, both federal and state governments have recently made
significant investments in services and new legislation to deal with what is
perceived to be a rapidly growing, or at least increasingly public, social
problem, and so the research is very timely. In addition to demonstrating
how this can be done, a few other points about big data emerge from the article
that can be considered as problematic, or even emblematic, for the social
welfare sector.
The algorithm is claimed to have an accuracy of just under 70 per cent,
similar to that of PRM, and again this may be impressive in the abstract, but
the question arises as to how useful this is to decision-makers in the real world.
As with risk assessment in child protection mentioned above, the consequences
of a wrong decision could be catastrophic. Human decision makers do make
mistakes which lead to harm in 30 per cent of cases.
The other interesting point that the researchers mention again contradicts
some of the rhetoric that is used to promote big data approaches in social
welfare. They state that the algorithm can achieve almost the same accuracy
with only three, rather than 11, factors about potential recidivists. As with
PRM, introducing other factors, such as demographic and other behavioural
information, did not lead to any significant or real improvement in its
performance. Using only three factors leads to only small trees, which may
be preferable as they are easier for professionals to understand and explain to
others how predictions and decisions have been made.
‘The consequences
of a wrong decision
could be
catastrophic’
The Broward County Experience
In another instance where a big data approach has been developed but has yet
to be applied, Schwartz et al. (2017) demonstrate how both prescriptive and
predictive approaches to machine learning may improve the efficiency of a
child protection system. They describe how Broward County in Fort
Lauderdale, Florida has experienced increases in the numbers of children being
notified, investigated, substantiated and, in particular, returning to the child
protection system leading to overload, a problem shared by many other
jurisdictions across the English-speaking world. They used supervised
machine learning and propensity score matching to develop: predictive models
for determining the likelihood of a child being notified subsequently; and
prescriptive models for determining the type and level of service that were
most likely to prevent the child entering the child protection system again.
Schwartz et al. (2017) acknowledge that using data about subsequent
notifications of maltreatment and substantiation is not ideal as they found that
decision-making between practitioners differed even when based on the same
factors. Put simply, inconsistency in the data makes it harder for an algorithm
to identify patterns and correlations within a dataset. The other problem that they
identified within the dataset was that contact between families and supportive
services did not reduce the likelihood of a child re-entering the child protection
system. This phenomenon has also been found in other quantitative studies of
child protection data (Jenkins et al., 2017) and, as Schwartz et al. (2017) suggest,
it is an area that requires research to determine why this should be so.
Schwartz et al. (2017) conclude that if their system was implemented, it has
the potential to improve outcomes for children and families by up to 30 per cent,
© 2019 John Wiley & Sons, Ltd.
‘Inconsistency in the
data makes it harder
for an algorithm to
identify patterns and
correlations within a
dataset’
Child Abuse Rev. (2019)
DOI: 10.1002/car
Gillingham
with outcomes defined as being referred to the appropriate level of service (from
supportive services to investigative and court interventions). Another point of
interest is that they mention Russell's (2015) and Coohey et al.'s (2013) four
standards for assessing a predictive model: validity, equity, reliability and
usefulness. These standards would provide a useful and clear framework for
empirical research that aimed to evaluate the use of DSS. Though Schwartz
et al. (2017) do not state explicitly, it appears that the approach they have used
means that the algorithm, in making a recommendation to a practitioner, would
also be able to show why it had made that recommendation, that is, show which
factors about a child or his or her family it had considered. As debates about the
need for algorithmic accountability have shown (Binns, 2018), it is not only
practitioners who need to understand how an algorithm works but also those
affected by that decision. This is especially important for service users who
want to question or contest a decision made by a practitioner.
A DSS in Amsterdam
Amrit et al. (2017) developed a decision support system to support
paediatricians and other healthcare workers in Amsterdam to identify children
under their care who are at risk of maltreatment. They used healthcare records,
which they claim were complete, and both structured data (entered into a
specific field, like age, height and weight) and, as they claim uniquely,
unstructured data (the notes made by healthcare workers in free-text fields).
Amrit et al. (2017) used a complex set of procedures to test different types
of algorithms and found that by combining their approaches to both structured
and unstructured data, they produced a tool that was just over 90 per cent
accurate. It is planned that the final tool will be implemented in the healthcare
system towards the end of 2018. The health authority is, however, open to
researchers evaluating its impact on decision-making by health professionals
(personal communication).
Discussion
‘Data scientists and
social workers need
to collaborate to
ensure that both
parties are clear
about what the data
actually represent’
© 2019 John Wiley & Sons, Ltd.
From the examples in the previous section, a few points arise which need to be
considered in the future development of DSS. Data scientists and social
workers need to collaborate to ensure that both parties are clear about what
the data actually represent in order to avoid mislabelling, a point which is more
fully explored below. In terms of accuracy, 70 per cent may be acceptable for
clinical decision-making, according to Zhang et al. (2014), but this value is
too low in child and family social work given the consequences of a bad
decision. Adding more independent variables does not necessarily improve
accuracy. Many variables may have little or no actual correlation with what
an algorithm is trained to predict and there may be only two or three that are
highly correlated. In the examples of PRM and predicting recidivism in
domestic violence, the main independent variables used by the algorithm are
those which have been identified long ago by researchers and paediatricians.
Given that, it is questionable why DSS would be helpful as they are not likely
to ever perform any better than trained practitioners. Lastly, if an algorithm is
updated (and trained) with new data on a regular basis, there is the potential for
Child Abuse Rev. (2019)
DOI: 10.1002/car
Decision Support Systems and Child and Family Social Work
sample selection bias which then undermines the accuracy of DSS. How the
performance of DSS can be improved emerges as a key challenge.
What does improve performance, as Schwartz et al. (2017) demonstrate, is
adding extra statistical procedures like pair matching. The main area for
improvement, though, as hinted at by Schwartz et al. (2017) (see also Amrit
et al., 2017), is the quality, rather than the quantity, of the data used to train
algorithms. Each of the projects mentioned above has used what Salganik
(2018) refers to as ‘readymade’ datasets. He warns that practical (and, of
course, ethical) problems can arise when using data for a purpose other than
that for which they were collected. This may be the usual way that big data
approaches are applied and may be part of their appeal, but there are particular
problems with how data are generated in the child protection field which
undermine the quality. These problems stand in stark contrast to the healthcare
data (both structured and unstructured) used by Amrit et al. (2017).
Child protection data are, obviously, created by humans and, unlike data
from health databases, they constitute a particular way of representing (or
reconstructing, see D'Cruz, 2004) a version of events, decisions and opinions.
These representations are subject to the inaccuracies, biases, prejudices and
inconsistencies that the literature describes about decision-making in child
protection. As Buckley (2003) found, they are part of the ‘official’ version of
practice and might be quite different to the ‘unofficial’ version or what actually
happens in a case. As information is entered into an information system, it has
to be categorised according to the fields built into the information system,
which may or may not fit the circumstances that the practitioner has observed.
This can happen in many ways (see Gillingham, 2015a) but an obvious
example is the level of detail required by the information system. For example,
a common question in risk assessment tools concerns illicit drug use by
caregivers. Ticking a yes/no box in response to such a question is not only
overly simplistic but confounding. In terms of data, caregivers who
occasionally smoke marijuana after the children have gone to sleep are put in
the same category as caregivers who inject heroin two or three times a day
and spend much of their time finding the means to do so. Dick (2017) calls this
the ‘flattening effect’ of categorising data. Clearly, there are different levels of
risk of harm or neglect posed by each scenario.
Case file data, both structured and unstructured, may also be incomplete
(Gillingham, 2009) as busy practitioners are compelled to move on to another
case, and even if they are complete, the version of practice that they convey, as
alluded to above, is only partial (Skotte, 2018). Even when practitioners go to
some lengths to explain the rationales for decisions in case notes, it is not
possible to convey the subtleties and nuances of decision-making that research
about it has uncovered (see, for example, Keddell, 2013). As has been pointed
out, the presence of various factors in a child's life is not the only focus of risk
assessment; it is the interaction between these factors that is important (Reder
et al., 1993). For example, the use of illicit drugs, at any level, may exacerbate
mental illness and vice versa. The presence of a particular factor may also be
positive or negative or somewhere in between and vary from day to day. The
presence of grandparents in a child's life might be a protective factor but it
depends on the grandparents and requires further assessment.
The above is merely an overview of the problems with the data held in child
protection information systems, but it does demonstrate that for DSS to
© 2019 John Wiley & Sons, Ltd.
‘Problems can arise
when using data for a
purpose other than
that for which they
were collected’
Child Abuse Rev. (2019)
DOI: 10.1002/car
Gillingham
‘We may need to
move on from using
ready-made datasets
to ‘custom-made’
datasets’
‘Collaboration needs
to go far beyond a
supposed
participatory design
process that mainly
seeks ‘buy in’ from
end users’
become more accurate and therefore potentially more useful, we may need to
move on from using ready-made datasets to ‘custom-made’ datasets
(Salganik, 2018) to train algorithms. As the authors mentioned in the previous
section have demonstrated, promising methods to develop DSS exist, but the
resulting products are only going to be as good as the data used to train them.
As to what needs to be included in a custom-made dataset is a matter that
requires more research and debate. Going back to Wijenayake et al. (2018)
and PRM in New Zealand, it may be that adding extra factors about a service
user or his or her circumstances may not improve the accuracy of DSS, but
in both cases the factors that could be added were not chosen because there
was reason to believe they might be predictive. They were added because they
were available. Using the extant literature about the antecedents of child
maltreatment and the process of decision-making, it might be possible to
deduce more precisely which factors might improve the accuracy of DSS if
they were included in a dataset. For example, a Delphi study by Powell
(2003) on early indicators of child abuse and neglect found five physical, 13
behavioural/developmental and 16 parenting indicators that may occur
separately or cluster together.
Developing DSS using custom-made datasets would have to involve, as
mentioned above, close collaboration between practitioners, data scientists
and child protection and social work researchers. This collaboration needs to
go far beyond a supposed participatory design process that mainly seeks
‘buy in’ from end users (Gillingham, 2015b). It requires the skills and
knowledge of all parties to develop, as suggested previously, a ‘minimum
dataset’ (Gillingham, 2016b) which involves the collection and entry of what
are considered to be highly relevant factors or predictors of the various forms
of child maltreatment. It would require changes in the way that practitioners
collect and record data. Indeed, using information systems as a tool for
research and the development of DSS would require a very different approach
to how information systems are currently used in child and family social work,
which is mostly to ensure compliance with procedures and standards
(Gillingham and Graham, 2016). Practitioners already complain about having
to spend inordinate amounts of their work time entering data into information
systems (Gillingham, 2016c) and adding additional data entry requirements
may be unsuccessful. A focus on the entry of very specific data, for example,
the indicators of maltreatment found by Powell (2003), with a renewed
emphasis on accuracy and completeness may have to replace some of the more
compliance-driven requirements for data entry. Data must also be retrievable in
a way that best suits the development of the most promising designs of
algorithms and the process will be experimental as different factors, algorithms
and statistical procedures are tested to achieve the highest level of accuracy
possible. Starting again in terms of collecting data means that future
development of DSS will not only be a costly exercise but also a long-term
endeavour.
Conclusion
To address the question in the title of this article, it is possible that DSSs can be
developed to assist decision-making in social work with children and families.
© 2019 John Wiley & Sons, Ltd.
Child Abuse Rev. (2019)
DOI: 10.1002/car
Decision Support Systems and Child and Family Social Work
However, as the examples described above show, current attempts using purely
administrative data, or ready-made datasets, have not led to DSS that are
sufficiently useful in practice. However, the learnings from such attempts have
led to more sophisticated approaches that incorporate extra statistical
procedures and unstructured data. It has been proposed that the most promising
route to developing more accurate and reliable DSS may be achieved by
developing a custom-made dataset, based on research about the factors
associated with child maltreatment.
None of the above discussion addresses the question of whether DSS should
be developed to assist decision-making in social work with children and
families or any other field of social work. How decision-makers in social work
(both do and will) react to DSS based on algorithms and big data has yet to be
evaluated. A critic might argue that DSS, as with actuarial risk assessment
before them, only identify: (a) which potential service users most need a
service, such as children who are most likely to be renotified, re-investigated
and those whose cases are most likely to be re-substantiated; and (b) those at
highest risk of significant harm. It could be argued that professionals are
already good enough at assessing who these service users might be, with or
without actuarial risk assessment tools. Further, the area where social workers
actually need more help and resources is in the development and delivery of
interventions that improve outcomes for service users. Clearly, there needs to
be debate within the profession about whether it is desirable for DSS to be
developed for social work. This debate needs to be informed by the key
contribution of this article, namely, the conclusion that DSS can be developed
to assist decision-making in social work with children and families, but they
will be a long-term and expensive process.
Funding
‘There needs to be
debate within the
profession about
whether it is
desirable for DSS to
be developed for
social work’
Australian Research Council Future Fellowship (FT170100080).
Acknowledgements
This research was supported by the Australian Research Council
(FT170100080).
References
Allegheny County. 2017. Developing Predictive Risk Models to Support Child Maltreatment
Hotline Screening Decisions. Allegheny County Department of Human Services:
Pittsburgh, PA. Available: https://www.alleghenycountyanalytics.us/index.php/2017/04/17/
developing-predictive-risk-models-support-child-maltreatment-hotline-screening-decisions/
[22 November 2018].
Alston M, Bowles W. 2012. Research for Social Workers: An Introduction to Methods, Third
edn. Allen and Unwin: Crows Nest, NSW, Australia.
Amrit C, Paauw T, Aly R, Lavric M. 2017. Identifying child abuse through text mining and
machine learning. Expert Systems with Applications 88: 402–418. https://doi.org/10.1016/j.
eswa.2017.06.035.
Berk RA. 1983. An introduction to sample selection bias in sociological data. American
Sociological Review 48(3): 386–398.
© 2019 John Wiley & Sons, Ltd.
Child Abuse Rev. (2019)
DOI: 10.1002/car
Gillingham
Binns R. 2018. Algorithmic Accountability and Public Reason. Philosophy and Technology.
31(4): 543–556. https://doi.org/10.1007/s13347-017-0263-5.
Buckley H. 2003. Child Protection Work: Beyond the Rhetoric. Jessica Kingsley: London.
CARE. 2012. Vulnerable Children: Can Administrative Data Be Used to Identify Children
at Risk of Adverse Outcomes? Centre for Applied Research in Economics, University
of Auckland: Auckland, New Zealand. Available: https://www.msd.govt.nz/documents/
about-msd-and-our-work/publications-resources/research/vulnerable-children/auckland-uni
versity-can-administrative-data-be-used-to-identify-children-at-risk-of-adverse-outcome.pdf
[22 November 2018].
Coohey C, Johnson K, Renner LM, Easton SD. 2013. Actuarial risk assessment in child
protective services: Construction methodology and performance criteria. Children and Youth
Services Review 35: 151–161.
CRC. 2018. The SDM Model in Child Protection. Children's Research Center, National Council
on Crime and Delinquency: Madison, WI. Available: https://www.nccdglobal.org/assessment/
sdm-structured-decision-making-systems/child-welfare [22 November 2018].
Dare T. 2013. Predictive risk modelling and Child Maltreatment: an ethical review. University
of Auckland: Auckland, New Zealand. Available: https://www.msd.govt.nz/documents/
about-msd-and-our-work/publications-resources/research/predictive-modelling/00-predicitverisk-modelling-and-child-maltreatment-an-ethical-review.pdf [22 November 2018].
D'Cruz H. 2004. Constructing meanings and identities in child protection practice. Tertiary
Press: Melbourne, Australia.
Devlieghere J, Roose R. 2018. A policy, management and practitioners' perspective on social
work's rational turn: there are cracks at every level. European Journal of Social Work.
https://doi.org/10.1080/13691457.2018.1460324.
Dick S. 2017. Algorithmic Accountability. Presented at the Internet Policy Research Initiative,
Massachusetts Institute of Technology, 23 March 2017. Available: https://internetpolicy.mit.
edu/event-angwin-algo-account-2017/ [23 November 2018].
Fitch D. 2006. Examination of the Child Protective Services Decision-Making Context with
Implications for Decision Support System Design. Journal of Social Service Research
32(4): 117–134.
Fitch D. 2007. Structural equation modeling the use of a risk assessment instrument in child
protective services. Decision Support Systems 42: 2137–2152.
Foster KA, Stiffman AR. 2009. Child Welfare Workers' Adoption of Decision Support
Technology. Journal of Technology in Human Services 27(2): 106–126.
Gillingham P. 2009. The use of assessment tools in child protection: An ethnomethodological
study (PhD thesis). University of Melbourne, Australia. Available: http://repository.unimelb.
edu.au/10187/4337 [27 November 2018].
Gillingham P. 2015a. Implementing electronic information systems in human service organizations:
The challenge of categorization. Practice: Social Work in Action 27(3): 163–175.
Gillingham P. 2015b. Electronic information systems and human services organisations: Avoiding
the pitfalls of participatory design. The British Journal of Social Work 45(2): 651–666.
Gillingham P. 2016a. Predictive risk modelling to prevent child maltreatment and other adverse
outcomes for service users: Inside the “black box” of machine learning. The British Journal of
Social Work 46(4): 1044–1058.
Gillingham P. 2016b. Electronic information systems and human service organisations: The
needs of managers. Human Service Organizations: Management, Leadership & Governance
40(1): 51–61.
Gillingham P. 2016c. The use of electronic information systems to guide practice in social
welfare agencies: The perspectives of practitioners as end users. Practice: Social Work in
Action 28(5): 357–372.
Gillingham P. 2017. Predictive risk modelling to prevent child maltreatment: Insights and
implications from Aoteaora/New Zealand. Journal of Public Child Welfare 11(2): 150–165.
Gillingham P. 2018. The evaluation of practice frameworks for social work with children and
families: Exploring the challenges. Journal of Public Child Welfare 12(2): 190–203.
Gillingham P, Graham T. 2016. Designing electronic information systems for the future: Facing
the challenge of New Public Management. Critical Social Policy 36(2): 187–204.
Gloor L. 2017. Data mining program designed to predict child abuse proves unreliable, DCFS
says. Chicago Herald Tribune, 6 December 2017. Available: http://www.chicagotribune.
com/news/watchdog/ct-dcfs-eckerd-met-20171206-story.html [22 November 2018].
© 2019 John Wiley & Sons, Ltd.
Child Abuse Rev. (2019)
DOI: 10.1002/car
Decision Support Systems and Child and Family Social Work
Jenkins BQ, Tilbury C, Mazerolle P, Hayes H. 2017. The complexity of child protection
recurrence: The case for a systems approach. Child Abuse & Neglect 63: 162–171.
Kang H, Poertner J. 2006. Inter-rater reliability of the Illinois Structured Decision Support
Protocol. Child Abuse & Neglect 30: 679–689.
Keddell E. 2013. Beyond care versus control: Decision making discourses and their functions in
child protection social work (PhD thesis). University of Otago, Dunedin, New Zealand. http://
hdl.handle.net/10523/3886 [23 November 2018].
Keddell E. 2015. The ethics of predictive risk modelling in the Aotearoa/New Zealand child
welfare context: Child abuse prevention or neo-liberal tool? Critical Social Policy 35(1):
69–88.
Macchione N, Wooten W, Yphantides N, Howell JR. 2013. Integrated health and human services
information systems to enhance population-based and person-centered service. American
Journal of Preventative Medicine 45(3): 373–374.
MacFadden RJ, Schoech D. 2010. Neuroscience, the Unconscious and Professional Decision
Making: Implications for ICT. Journal of Technology in Human Services 28(4): 282–294.
Ministry of Social Development. 2014a. Final report on feasibility of using predictive risk
modelling. Ministry of Social Development: Wellington, New Zealand. Available: https://
www.msd.govt.nz/documents/about-msd-and-our-work/publications-resources/research/predi
ctive-modelling/00-feasibility-study-report.pdf [23 November 2018].
Ministry of Social Development. 2014b. The feasibility of using predictive risk modelling to
identify new-born children who are high priority for preventive services—companion technical
report. Ministry of Social Development: Wellington, New Zealand. Available: https://www.
msd.govt.nz/documents/about-msd-and-our-work/publications-resources/research/predictivemodelling/00-feasibility-study-report-technical-companion.pdf [23 November 2018].
Powell C. 2003. Early indicators of child abuse and neglect: a multi-professional Delphi study.
Child Abuse Review. 12(1): 25–40 https://doi.org/10.1002/car.778.
Reder P, Duncan S. 2004. Making the most of the Victoria Climbié Inquiry Report. Child Abuse
Review 13(2): 95–114. https://doi.org/10.1002/car.834.
Reder P, Duncan S, Gray M. 1993. Beyond Blame: Child Abuse Tragedies Revisited. Routledge:
London.
Russell J. 2015. Predictive analytics and child protection: Constraints and opportunities. Child
Abuse & Neglect 46: 182–189.
Salganik MJ. 2018. Bit by Bit: Social Research in the Digital Age. Princeton University Press:
Princeton, NJ.
Schoech D, Jennings H, Schkade LL, Hooper-Russell C. 1985. Expert Systems: Artificial
intelligence for professional decisions. Computers in Human Services 1(1): 81–115.
Schwalbe C. 2004. Re-visioning risk assessment for human service decision making. Children
and Youth Services Review 26: 561–576.
Schwartz E, Nowakowski-Sims E, Ramos-Hernandez A, York P. 2017. Predictive and
prescriptive analytics, machine learning and child welfare risk assessment: The Broward
County experience. Children and Youth Services Review 81: 309–320. https://doi.org/
10.1016/j.childyouth.2017.08.020.
Skotte PS. 2018. On caseworkers' writing in child welfare
when less is more. European
Journal of Social Work. https://doi.org/10.1080/13691457.2018.1469474
Stanley T. 2005. Making decisions: Social work processes and the construction of risk(s) in child
protection work (PhD thesis). University of Canterbury, New Zealand.
Vaithianathan R, Maloney T, Putnam-Hornstein E, Jiang N. 2013. Children in the public benefit
system at risk of maltreatment: Identification via predictive modelling. American Journal of
Preventative Medicine 45(3): 354–359.
Wijenayake S, Graham T, Christen P. 2018. A decision tree approach to predicting recidivism in
domestic violence presented at the Big Data Analytics for Social Computing (BDASC)
workshop held at the Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD'18), Melbourne, Australia, June 2018. Available: https://arxiv.org/abs/1803.09862
[23 November 2018].
Zhang SX, Roberts REL, Farabee D. 2014. An Analysis of Prisoner Reentry and Parole Risk
Using COMPAS and Traditional Criminal History Measures. Crime and Delinquency 60(2):
167–192. https://doi.org/10.1177/0011128711426544.
© 2019 John Wiley & Sons, Ltd.
Child Abuse Rev. (2019)
DOI: 10.1002/car
Copyright of Child Abuse Review is the property of John Wiley & Sons, Inc. and its content
may not be copied or emailed to multiple sites or posted to a listserv without the copyright
holder's express written permission. However, users may print, download, or email articles for
individual use.