Building a logic model of reinforced
models of palliative care
Anke Rohwer, Andrew Booth, Lisa Pfadenhauer, Louise
Brereton, Ansgar Gerhardus, Kati Mozygemba, Wija
Oortwijn, Marcia Tummers, Gert Jan van der Wilt, Eva
Rehfuess
The INTEGRATE-HTA project is co-funded by the European Union under the Seventh Framework Programme (Grant Agreement No. 306141)
The logic of logic models
The Logic of Logic Modelling
Complexity of health technologies
and systems
Describes components of complex
interventions and relationships
between them
Strong assumptions about the Logic Models
Makes underlying theories of
a graphic
structure, content and “…
objectives
of description of a system …
change and assumptions about
to identify important
a health technology asdesigned
well as its
causal
pathways
elements and relationships
within
that explicit
implementation
system”.
Negligence of system within which
the intervention is intended to act
and the potential interplay and coevolution
Describes interactions between the
intervention and the system within
which it is implemented
Added value of using logic models in
systematic reviews
Scoping
the review
Defining
and
conducting
the review
Making
the review
relevant to
policy and
practice
• Refining review question
• Deciding on lumping or splitting a review topic
• Identifying intervention components
•
•
•
•
Identifying relevant study inclusion/exclusion criteria
Guiding the literature search strategy
Explaining the rationale behind surrogate outcomes used in the review
Justifying need for subgroup analyses (e.g. age, sex/gender,
socioeconomic status)
•
•
•
•
Structuring reporting of results
Interpreting results based on intervention theory and systems thinking
Illustrating how harms and feasibility are connected with interventions
Interpreting results based on intervention theory and systems thinking
(Anderson et al, 2011)
Method of development of logic models:
four step process
• (i) systematic searches for published examples of
logic models
• (ii) searches for existing guidance on the use of logic
models in primary research, SRs and HTAs
• (iii) development of two draft templates for systembased and process-orientated logic models and
• (iv) application of these templates in three SRs and
one HTA of different complex health technologies.
Logic models in HTA and Systematic
Reviews
HTA/SR research question
Check literature for published logic models
Adoption of published
logic model
Adaptation of logic model
De novo logic model
Type of logic model
A-priori logic model
Iterative logic model
Staged logic model
Subtype of logic model
Process-based logic model
System-based logic model
Three types of logic model
Label
Meaning
A priori logic A logic model that is specified as close to the inception of an HTA or SR as scoping the
model
literature and/or stakeholder consultation permit and that remains unchanged during the
Type
HTA/SR process.
Iterative
A logic model that is subject to continual modification and revision throughout the course
logic model
of an HTA or SR.
Staged logic
A type of iterative logic model that pre-specifies points at which major data inputs are
model
anticipated to prompt a subsequent version of the logic model, thereby increasing
transparency and minimising problems with version control.
Subtype: System-based logic model
Subtype: Process-oriented logic model
Populating the logic model template
Literature
review
SAP and
expert
input
Discussion
within team
Logic models in the INTEGRATEHTA Process
INTEGRATE-HTA model
Step 1: HTA Objectives and Technology
Decision-making body, HTA
commissioning agency
Selection of theme for
assessment e.g. palliative
care
Definition of
functional
requirements of the
decision-making body
Definition of
stakeholder
advisory
panel (SAP)
Create logic model
architecture and attributes
for specific technologies
according to a generic
logic model template
Create specific logic
model regarding the
theme e.g. palliative care
based on the data from
step 1
Identify and
assess patient
preferences,
moderators,
context and
implementation
Specific requirements and evidence
needs according to the specific logic
model, context, implemantation and
patient groups (moderators/
preferences), relevant issues
Literature
review, SAP
consultations
Evidence collection for all assessed
aspects (effectiveness, economics,
ethical, legal, cultural, and social
aspects, relevant issues)
Step 4: Mapping of the evidence
Step 5: HTA decision-making
Evidence summaries about different
assessment aspects (e.g.
effectiveness, ethics)
Presentation of HTA results
obtained from steps 3 and 4 to a
decision committee comprising
stakeholders/decision-makers
Selecting a tool to structure a
deliberative discussion (in
cooperation with the decision
committee)
Summarizing and structuring the
assessment results into specific
assessment criteria of the HTA
research question
HTA researchers
Scoping
literature
overview
Specific
scoping
procedures
for each
assessment
aspect
considered
Definition of relevant issues and
assessment criteria regarding the
assessment theme (e.g. access,
continuity)
Result
Step 3: Evidence assessment
Step 2: Logic Model to define evidence needs
Definition of HTA research question,
assessment criteria and preliminary
definition of technologies e.g. models
of care and relevant issues
Scientific assessment of evidence
Review and adaptaptation of specific
model by SAPs and HTA researchers
Review of the assessment results by
HTA researchers and SAPs
Refinement of A,B,C,D, E:
A) Definition of specific technologies
B) Relevant issues
C) Outcome parameter
D) Relevant patient characteristics (preferences,
moderators)
E) Context and implementation issues
Result
Specific logic model to start evidence
collection including A,B,C,D,E
Construction of the ELMMAR
(Extended Logic Model to Map the
Assessment Results) Model:
Integration of the assessment
results (effectiveness, ethics etc.)
Deliberative reflections of
stakeholders/decision-makers
about unanswered issues/
uncertainty/
limitations of the assessment
process (steps 1- 4)
Review by stakeholders (HTA
researchers, SAPs, decision-maker)
Completing evidence
summary templates
about different
assessment aspects
(e.g. effectiveness,
ethics)
Result
Evidence reports and evidence
summaries for each assessment
aspect
Deriving conclusions from the
ELMMAR Model (HTA researchers,
SAPs, decision-maker)
Result
ELMMAR Model and
synthesised evidence according
to the HTA research question
Result
HTA decision/ recommendation
Ideal application of Logic Models in
INTEGRATE-HTA: Staged system-based logic
model
1
Based on the agreed
HTA question and a
combination of team
discussions, literature
and SAPs.
Initial Logic
Model
2
3
4
Based on an analysis
of context and
implementation,
patient preferences
and moderators of
treatment.
Based on the results of
the effectiveness
review and economic
assessment, as well as
insights generated
through the sociocultural, ethical and
legal assessments.
Based on any
additional findings
generated through an
assessment of context
and implementation.
Iteration 2
Final Logic Model
Iteration 1
Systembased logic
model of
reinforced
and nonreinforced
home-based
palliative
care
(Brereton et
al. 2015)
Discussion
• Added value
– Getting team on same page
– Integrating various types of evidence/knowledge
– Structuring subsequent SR/HTA process
– Graphical means of communication
• Strenghts and limitations:
– parallel development of methods and application in
demonstration HTA
– Empirical testing in various SR
– Each type and subtype of logic models has its specific
strengths and limitations
Conflict of Interest
• The authors have no conflicts of interest to declare
<DISCLAIMER: The sole responsibility for the content of this presentation lies with the authors. It does not necessarily reflect
the opinion of the European Union. The European Commission is not responsible for any use that may be made of the
information contained therein.>
References
• Anderson LM, Petticrew M, Rehfuess E, Armstrong R,
Ueffing E, Baker P, et al. Using logic models to
capture complexity in SRs. Research Synthesis
Methods. 2011;2(1):33-42.
• Rohwer, A., Booth, A., Pfadenhauer, L., Brereton, L.,
Gerhardus, A., Mozygemba, K., Oortwijn, W.,
Tummers, M., Van Der Wilt, G.J., Rehfuess, E. (2015)
Guidance on the use of logic models in health
technology assessments of complex interventions
[Online]. Available from: http://www.integratehta.eu/downloads/