How to use the Realist approach to inform policy-making: the example of
a research method on user fees abolition policies in Africa.
Robert E.1, Ridde V.1,2, Fournier P.1, Bicaba A.3
1 CR-CHUM, Canada. 2 IRSS/CNRST, Burkina Faso. 3 SERSAP, RESAO, Burkina Faso.
Background
User fees were implemented in Africa in early 80s. Faced with the inequity of such policies in terms of
access to health care by vulnerable populations, policy-makers started to abolish them. Researchers
have documented these recent policies (Meessen et al., 2009; Ridde & Morestin, 2010). International
stakeholders and African leaders now want to know if the promises of abolition policies are kept.
Objective
The objective is to present the method to be used to synthesize the evidence on user fee exemption
policies in Africa, and to inform policy-making. The review process implies the active participation of a
network of researchers and policy-makers from the West-African region, RESAO (see ‘Stakeholders’
participation’ box for details).
Definition of a user fee exemption policy:
“an official reduction in direct payments for health care, which is targeted by group, area or service”
(Witter, 2009). The national dimension of the policy is also to be considered.
Theoretical framework
Given the complexity of these policies, we will use a Realist perspective, based on the work by Pawson
and Tilley (Pawson, 2006; Pawson & Tilley, 1997). Drawing on theory-driven evaluation, this approach
offers the possibility to go beyond the question of effectiveness, and build up middle-range theories that
explain mechanisms of social change. It aims at identifying how context interacts with intervention
mechanisms to produce outcomes – the so-called CMO configurations. The question to ask is not ‘Is the
intervention working’. Instead, we should ask ‘How does the intervention work? For whom? Under what
circumstance?’
Definition of a middle-range theory
“Theories that lie between the minor but necessary working hypotheses that evolve in abundance during
day-to-day research and the all-inclusive systematic efforts to develop a unified theory that will explain the
observed uniformities of social behavior, social organization and social change.” (Merton, 1968)
Context
Figure 1 – Generative causation
(Pawson & Tilley, 1997)
Mechanism
Outcome
The ingredients of the Realist recipe for systematic reviews
Context: social, cultural, historic or institutional elements that limit or facilitate individual actions.
Mechanism: action or intervention theory taken in a cognitive perspective (human reasoning and
resource).
Outcome: result of the interaction between mechanism and context; they are thus context-dependent.
Research questions
1) What are the intervention theories that underlie user fee exemption policies in African countries?
2) What are the contextual elements that influence how intervention theories are implemented and lead
to expected outcomes?
3) How do mechanisms and contextual factors interact to produce – or not – expected outcomes?
Methods
Quality appraisal – Criteria by Pluye et al. (2009)
Qualitative study : qualitative objective or question / appropriate qualitative approach / description of
the context / description of participants and justification of sampling /description of qualitative data
collection and analysis / discussion of researchers’ reflexivity
Quantitative experimental study: appropriate sequence generation and/or randomization / allocation
concealment and/or blinding / complete outcome date and/or low withdrawal/drop-out
Quantitative observational study : appropriate sampling and sample / justification of measurements
(validity and standards) / control of confounding variables
Mixed methods study : justification of the mixed methods design / combination of qualitative and
quantitative data collection-analysis technique / Integration of qualitative and quantitative data or results
Step 4: Extracting data
An analytic framework will be developed according to the elements from the context, mechanisms and
outcomes making up each theory. This framework will be applied to the primary studies, and will be
completed in an iterative manner, in order to integrate new explicatory elements. We will use N-Vivo for
this purpose.
Step 5: Synthesizing the data
Evidence from the primary studies will then be juxtaposed, reconciled, adjudicated, consolidated and
situated. An inductive analytic process will make possible the specification of constituting elements of the
CMO configurations. Finally the middle-range theories will be reformulated as narrative propositions and
as models in order to emphasize the relation between the context, mechanisms and outcomes. A
two-day workshop will be organized with the stakeholders to present the findings of the review process,
and formulate evidence-based recommendations on exemption policies.
Step 6: Empirically testing the middle-range theories
Considering time and money constraints, only one theory will be empirically tested. The choice will be
made by the stakeholders during the workshop mentioned above and will be made according to their
needs. Consequently the following proposed method may change to take into consideration their
decision. We will carry out a Realist evaluation of the exemption policy of Mali based on a case study
design. Data will be collected through focus groups and semi-structured interviews of key informants
from the national level, and managers of health facilities. The analytic framework developed in step 4 will
be used to analyse the data.
Step 7: Disseminating the findings
Besides the traditional way of diffusing scientific evidence, a message that is clear, short and adjusted to
the audience would make it easier for policy-makers to use the evidence. That is why a policy brief will
also be produced. According to the definition by the European Observatory on Health Systems and
Policies (2010): “Policy briefs emphasize the key policy lessons from […] studies in a brief and concise
format that are easy to read for busy policy-makers”.
Outcomes. Combining realist synthesis and evaluation is an innovative way to understand how abolition
policies work, for whom and in what contexts. Participation of stakeholders will ensure that the needs of
policy-makers are met, and it will facilitate the diffusion and utilization of the results.
Conclusion. This method seems a promising innovation to better understand complex social
interventions.
Figure 2 – The combined logics of realist
synthesis and realist evaluation. Inspired
by Pawson and Tilley
(Pawson, 2006; Pawson & Tilley, 1997).
Initial theory
C
M
O
C1
C2
M2
O1
C3
C4
M3
Primary studies
M4
O3
C5
O4
C6
M5
O5
M6
O6
Cn
Mn
On
Program
specification
Revised theory
Observations
CMO
hypotheses
to be tested
Realist evaluation
Literature search
Search engines, databases and online journals: Ovid Medline, Embase, Popline, HMIC, Web of
Knowledge, African Healthline, AJOL (African Journals On Line), EcomLit, Business Source Premier /
Health Policy and Planning, Bulletin of WHO, Social Science and Medicine / DATAD, Dissertations and
Thesis / books and grey literature
Key words / expressions: [User fee*OR User charge OR Cost sharing OR Cost recovery] AND [Abol*
OR Exempt* OR Waiv* OR Remov* OR End* OR Discontin* OR Free] / Free healthcare OR Free care OR Free health care OR Universal access to healthcare OR Universal access to health care / Health care
costs, Health insurance, Health expenditure*, Health financing, National health programs
Exclusion and inclusion criteria: studies that are not about user fee exemption policies implemented in
African countries at the national level will be excluded
Selection process: first selection based on reading abstracts, second selection based on reading
articles
Snowballing from the bibliography of selected articles + ISI Web of Science to identify articles quoting
selected articles
Step 3: Appraising quality
The quality appraisal in the realist perspective does not aim at assessing the methodological rigor of
primary studies. Instead, it deals with the relevance of information contained in a study to refine theories.
The question is: « Is this study good enough to provide some evidence that will contribute to the
synthesis? » (Pawson, 2006). However, this angle lacks transparency and rigor. That is why we will use
the quality appraisal tool developed by Pluye et al. (2009), which allows the concomitant quality appraisal
of qualitative, quantitative and mixed studies. See ‘Quality appraisal’ box for details.
Realist synthesis
We will first carry out a realist synthesis to find these configurations and develop theories about
abolition policies, and then test the theories using a realistic evaluation. A board of stakeholders will be
involved at different steps of the process.
Step 1: identifying intervention theories of user fee exemption policies
We will rebuild the causal model of the inequalities of access to health care, and the theoretical model of
exemption policies. The specific objectives are: 1) to understand how exemption policies are supposed to
work, for which beneficiaries, and with what expected outcomes; 2) to identify contextual elements
influencing the way exemption policies are supposed to produce outcomes; 3) to explore the influence of
other health policies on the implementation of exemption policies and expected outcomes. The board of
stakeholders will meet for a one-day workshop to refine these models according to their field experience.
Step 2: searching for primary studies
A comprehensive literature search will be carried out. Besides scientific literature, grey literature will
also be reviewed. The network of stakeholders and researchers from RESAO will help find the grey
literature. See ‘Literature search’ box for more details.
Réseau Équité et Santé
en Afrique de L'Ouest
Stakeholders’ participation
Why involve stakeholders?
According to the knowledge translation literature, personal
contact between researchers and decision-makers, as well as
participation of decision-makers facilitate the use of evidence.
Who to involve?
The RESAO (Réseau Equité et Santé en Afrique de l’Ouest) is a
regional network of West-African health researchers and
policy-makers whose objective is to foster formulation of
evidence-based health policies in the partnering countries,
specifically pro-equity policies (http://reseao-sante.org/). It has
been partnering with Montreal University for several research
projects.
How to involve the stakeholders?
AB who is the coordinator of RESAO is a partner in the research
project. Furthermore, a board of key policy-makers from Burkina
Faso, Mali and Niger is being set up. This board will meet twice
during the review process, as mentioned in the method, and will
be regularly informed of the progress.Snowballing from the
bibliography of selected articles + ISI Web of Science to identify
articles quoting selected articles
ER holds a scholarship from the Global Health Research Capacity Strengthening Program (GHR-CAPS).
Contact: emilie.robert.3@umontreal.ca
VR is a CIHR New Investigator.
VR, PF and AB hold a grant from the CIHR Knowledge Translation Branch.
References: