SFI CAMPBELL – INTERNATIONAL KNOWLEDGE
ABOUT IMPACTS
By Mette Deding, Head of SFI Campbell
SFI Campbell gathers knowledge about effects and impacts of interventions
within welfare policy. In recent years focus has been on effects and on
evidence-based policy and practice, and we contribute to this by producing
systematic reviews of general welfare policy according to the Campbell
Method within topics such as employability enhancement measures for the
unemployed, treatment of substance abusers and consequences for soldiers
of deployment.
A Campbell systematic review is a review which sums up results of all studies
measuring the effects of interventions within welfare policy.
Knowledge about effects, the impact, of an intervention within welfare policy can be
found by seeking to ascertain how people who receive the intervention are coping
compared to how they would have coped if they had not received the intervention.
Methodologically, it is not easy to isolate the effect of a given intervention from other
possible factors influencing the participants at the same time. A classic methodology
to isolate effects of interventions is through completely randomised design, in which
initially some potential participants are randomly designated to receive the
intervention under investigation, and the others are randomly designated to receive
another type of intervention or perhaps be put on a waiting list. This design prevents
systematic differences between the people receiving the intervention, and the people
with whom they are being compared. However, other research designs can be used to
carry out impact assessments, even though these do not include randomised design.
Common for these is the comparison between people receiving the given intervention,
and people not receiving the intervention; i.e. a control group.
This is how it works
The Campbell Method consists of four basic steps: Listing criteria for the scientific
approach to the studies included in the analysis, searching systematically for studies
using a standardised search strategy, assessing all studies found according to the
inclusion criteria and finally analysing the studies which meet the inclusion criteria.
In addition, a fundamental part of the process is that all methodologies, criteria and
choices in the systematic review are documented and made public in order to ensure
transparency. In practice, a protocol is produced (i.e. work plan) for the systematic
review process as well as the actual systematic review. All material is approved by the
independent research group, the Campbell Collaboration, as both the protocol and
systematic review are peer-reviewed by specialists and methodology experts. Finally
the result is published in the Campbell Library.
How can the results be used?
A Campbell systematic review gives an answer to a specific question about the effects
of a specific initiative for a specific intervention measured on specific criteria in an
international perspective. In a time with ever greater resource limitations in welfare
policy, it is increasingly important that resources are used in the best possible way. Of
course, one element in this is how the intervention can be expected to work for the
participants. However, results of this type can also be included in cost-benefit
considerations, where impact studies give insight into the benefits of the intervention.
Therefore, the result of a Campbell systematic review is a contribution to the overall
understanding of a given intervention and is input for deciding which interventions
should be used in a society; decisions which have not been made on the basis of
results about effects alone, but in interaction with other types of knowledge.
There has been protracted and intense debate on the role in the political decisionmaking process of impact evaluation research, and in this connection Campbell
systematic reviews. Results are therefore used by supporters and opponents of
evidence-based policy and practice, sometimes in an unconstructive way.
SFI Campbell believes that knowledge is a non-exact, multifaceted concept, where our
systematic reviews have a clear role in answering the highly relevant questions: does
the intervention work as intended, to what extent does it work and for whom? The
advantage of a Campbell systematic review is that the method is systematic,
documented and transparent, so it is clear how the result has emerged. Conversely,
as mentioned above, the result cannot stand on its own, but must be used as a piece
in the overall knowledge jigsaw puzzle.
SFI Campbell

is a division within SFI - Danish National Centre for Social Research

produces and disseminates systematic reviews of effects according to the
Campbell Method

prepares systematic reviews on e.g. employability enhancement measures for
the unemployed, treatment of substance abusers and deployed soldiers.