UNIVERSITY OF TASMANIA
University Department of Rural Health
A Logic Model Tool
for Evidence Informed
Policy Making
Dr Peter Orpin
for the NHMRC Partnership Grant
Team
The Team – NHMRC Partnership Grant
• Industry Partners
– Department of Health and Human Services
– Department of Premier and Cabinet
• Chief Investigators
–
–
–
–
–
Prof. Judi Walker – Monash University
Dr Peter Orpin – University of Tasmania
Prof. John Humphries – Monash University
Dr Matthew Carroll – Monash University
Kim Boyer – University of Tasmania
• Associate Investigators
– Prof Helen Bartlett – Monash University
– Prof. Andrew Robinson – University of Tasmania
– Mr Brendon Davidson – Department of Health & Human Services
www.utas.edu.au/ruralhealth
2
The Project
Initial Aim: To develop a tool to support policy makers to increase the
evidence base of rural ageing well policy and practice.
An evolution: To a more generic tool to facilitate evidence informed
policy making – using a rural ageing well case example:
– Very limited specific, quality ageing-well intervention evidence let alone
rural ageing well evidence
– Considerable, more generic, QOL support intervention evidence but
dispersed widely across disciplines, professional practices and
literature and, therefore, difficult to find, collate and operationalise
– Issues around evidence proliferation, dating, translation and
incorporation into practice
– Restricted translation capacity of social science evidence
What appeared to be needed was not a packaged suite of rural ageing
well evidence but a tool to ‘sensitise’ policy makers to the potential
need for evidence and to facilitate their sourcing of that evidence.
www.utas.edu.au/ruralhealth
3
Evidence and The Policy Process
• Continuing strong support for increased use of evidence in
policy, however:
– Evidence based policy making (EBPM) recognised as flawed both
empirically and normatively SANDERSON, I. 2011. Evidence-based policy or policy-based
evidence? Reflections on Scottish experience. Evidence & Policy: A Journal of Research, Debate and Practice, 7, 5976.
– Not exactly ‘muddling through’ (Lindblom 1959) - some (variable)
rational linear evidence based elements - but also ‘values, interests,
personalities, timing, circumstances and happenstance’ BANKS, G. 2009.
Evidence-based policy-making: What is it? How do we get it? ANZSOG/ANU Public Lecture Series P.4
– Not an event but a social dialogue/discourse – ‘a formal [and perhaps
informal] struggle over ideas and values.’ RUSSELL, J., GREENHALGH, T., BYRNE, E. &
MCDONNELL, J. 2008. Recognizing rhetoric in health care policy analysis. Journal of Health Services Research & Policy, 13,
40-46.p. 40
– Role for evidence in informing and influencing discourse ‘change[ing]
mental models and minds DAVIES, H. T. O. & POWELL, A. E. 2012. Communicating social research
findings more effectively: what can we learn from other fields? Evidence & Policy: A Journal of Research, Debate and Practice,
8, 213-233.
– Best hope is evidence informed policy processes (EIPM)
www.utas.edu.au/ruralhealth
4
Facilitating EIPM
• Policy-makers receptive if it assists them to do their job LOMAS, J.
& BROWN, A. D. 2009. Research And Advice Giving: A Functional View of Evidence Informed Policy Advice in a Canadian Ministry
of health. The Millbank Quarterly, 87, 903-926.
• However, also poses disincentives – can make life more difficult BANKS, G.
2009. ibid
• EIPM theorisation and strategizing largely ‘push’ focussed– researchers
seeking to understand the policy process from the outside in order to
identify openings, barriers and opportunities to push evidence into the
process (see BELL, E. 2010. Research for Health Policy, New York, Oxford University Press for an example).
• Much less attention to policy maker evidence ‘pull’ – that is, what prompts
policy makers to actively seek out evidence BANKS, G. 2009. ibid
• ‘Evidence pull’ starts with policy makers having questions that need
answers which, in turn, has its roots in a critical reflective approach to
policy making
A revised project aim: a tool that assists policy makers to identify those
elements within the process that require critical examination and testing
through further information (including evidence) seeking
www.utas.edu.au/ruralhealth
5
A Logic Model Based Approach
• Program Logic Modelling already well established within
program evaluation as a tool for making program
assumptions explicit and open to testing through information
seeking. W.K. KELLOGG FOUNDATION 2004. Logic Model Development Guide: Using Logic Models to Bring Together
Planning, Evaluation, and Action. Battle Creek, Michigan: W.K. Kellog Foundation.
• The generic program logic model looks something like this:
• As generally operationalised it is used to identify and target some critical
points at which the performance of a program may be tested:
– Were the inputs or resources adequate?
– Did the activities take place as intended?
– Did they produce the outcomes as expected?
• It is also often notable for lack of questions addressing
underlying assumption
Logic modelling, however, has potential utility at all stages of the policy or
program development process, from initial planning to post hoc evaluation.
www.utas.edu.au/ruralhealth
6
The ‘if we, then’ logic
Program logic models have at their heart one, or more, ‘if we,
then’ logic models – that is, if we put these resources into these
actions then we will achieve these outcome
Such causal linkage models underlie and shape policy
discourses and their ‘struggles over ideas and values’ – but are
rarely explicitly modelled, particularly at any level of detail.
‘If we, then’ logic implies a range of assumptions – usually
implicit and unreflected upon - about current states and the
associated causal mechanisms either underlying those current
states or the achievement of change within those states
www.utas.edu.au/ruralhealth
7
A logic model allows those assumptions to be made explicit and facilitates their
testing through critical reflection and application of information
The nature of the ‘problematic’ state – and its root causes
The range of possible pathways from the problematic to the unproblematic
The appropriateness, feasibility and actual implementation of the intervention(s)
The mechanisms by which these interventions might have their effect
What exactly a more desirable state would/should look like and acceptable
alternatives
6. The (potentially changing) environment(s) in which this is all taking place
1.
2.
3.
4.
5.
www.utas.edu.au/ruralhealth
8
By formally modelling the policy or practice aspiration we:
• Make all assumptions and positions fully explicit and open
to testing
• Precipitate critical reflection and discourse around these
issues
• Increase the imperative to seek clarifying information
(evidence ‘pull’) whether that be formal evidence, informal
intelligence or a discourse around values and ideas.
• Provide an anchoring or re-orienting point to return to when
the discourse loses its way or hits an impasse.
www.utas.edu.au/ruralhealth
9