Academic excellence for business and the professions
Using framework methods to
analyse qualitative data
Reflections from a study of users’ experiences of a
community-based intervention for overweight children 7-13
years
Dr Katherine Curtis Tyler
What is the contribution of
qualitative methods to evidence?
Understanding stakeholder views
and experiences, and wider
process provides “insight into why
an intervention fails unexpectedly or
has unanticipated consequences,
or why a successful intervention
works and how it can be optimised”
MRC, 2008
http://www.mrc.ac.uk/Utilities/Documentrecord/index
.htm?d=MRC004871
My background
• Reviews of effectiveness and
implementation studies in
health and social care – how
and why interventions work
(or not) the way they do
• Children not just the object of
concern – with expertise
about their own lives crucial
to the developmetn of
effective interventions
A population-level evaluation of a family-based community
intervention for childhood overweight and obesity (MEND 7-13)
• A multi-component community intervention to support families of
overweight/obese children to adopt and maintain healthy lifestyles
• 10 weeks, education, skills, motivation
• MEND provides materials, local staff
• Shown moderate effect at one year under research conditions
• Delivered at scale, who does it work for? (Who gets to go?) What
are the health and social care costs?
• Catherine Law, UCL Institute of Child Health
• NIHR HS&DR
The families and commissioner studies
Helen Roberts, Institute of Child Health
Lisa Arai Teeside University; Patricia Lucas Bristol University; KCT City
Children and families
To find out the salience and acceptability for participants
To explore the types of costs when participating and in sustaining a
healthy lifestyle afterwards.
30 group interviews + 30 one to one
Commissioners and delivery partners
What is the salience and acceptability of MEND? - especially explore
tension between programme fidelity and responsiveness to local context
Telephone / face to face interviews 30 commissioners
Recruitment via MEND records/contract holders N/E, S/W, London
Interview questions
Children and families
• Reasons for participation, non-participation,
drop-out
• What liked best/least and why
• Problems and costs?
• Maintained healthy living? Helped/hindered?
Commissioners
• Local obesity profile/demographics
• How funding flows – flat or by completion
• What’s worked more/less well
• What tends to inform commissioning weight
management programmes
• Commission again? Why?
Approaches to qual analysis in applied policy
research
•
Variations on thematic analysis
•
Constant comparative method – comparing data,
categorising similar/different
•
Themes arising “solely” from data (Glaser and Strauss
1967)
•
Pre-existing categories (existing evidence)
•
Actively search for negative cases
•
Double code and discuss (Thomas et al 2003)
•
Data management - software, or coding on transcripts
with notes/ ‘memos’ re ideas for higher levels of
abstraction (Glaser and Strauss 1967
Glaser BG and Strauss AL (1967) The Discovery of Grounded Theory. Chicago: Aldine
Thomas J, Sutcliffe K, Harden A et al. Children and healthy eating: a systematic review of barriers and facilitators. 2003.
London, Eppi-Centre, Social Science Research Unit, Institute of Education, University of London.
Concerns and how Framework might address
them
•
Individual voices and patterns across
groups of voices lost
•
Eg different experiences for families
who attended less than 2 sessions,
who attended approx half, who
attended all sessions
•
list of themes which gives you little
insight into the ‘how’ and ‘why’
questions under investigation
•
Need to develop higher levels of
abstraction – conceptual analysis:
analyses should be more than the sum
of their parts
•
Systematic and transparent
•
Framework - explicit process to
guide movement up ‘analytical
ladder’ from descriptive to
conceptual categories
•
Systematic/transparent – of
course not necessarily “reliable”
Ritchie J, Spencer L & O’Connor W
(2003) Carrying out qualitative analysis in
J Ritchie and L Lewis (eds) London: Sage
Ritchie J and Spencer L Qualitative data
analysis for applied policy research in A
Bryman and RG Burgess (eds) “Analyzing
qualitative data” pp.173-194. London:
Routledge.
Framework
Data management stage (iterative)
• Familiarisation
• Identifying low level themes
• Categorising or ‘charting’ summarising in case/category
matrix – page numbers to facilitate
cross ref with interview
• Facilitates comparing across and
within cases
Interpretation stage
• Development of conceptual
categories / higher order themes
• Assess relationships between
categories ie using patterns to
search for explanations (not
universal deterministic causes)
• Typologies
• Existing research
• Theory
• Discussion/ brainstorming/delphi
Families: data management - charting low level
categories by case
Excel Sheet 1 Interview details
2 Attending MEND
3 Barriers to taking part /dropping out
4 Levers to taking part or sticking with it
5 Behaviour change - During programme/ views of content
6 Types of costs of programme
7 Behaviour change – after the programme
Families: charting low level categories by case
Excel Sheet 1 Interview details
1.1 Family
Number MEND
2 Attending
1.1 Family
number
1.2.
Interview
p/pants
1.3
Transcript
file
number
1.4 Setting
1.5 Mend
records of
attendanc
1.2 Participants
2.1How sessions many attended (please note where
3 Barriers
to taking part /dropping out
1.3 Transcript
number
different file
from
1.5)
3.1Practical factors (time, location, travel,Family
childcare)
1 Mum
Home
High
1.4 Setting
2.2Who attended
(kids
and
adults)part or sticking with it
4
Levers
to
taking
3.2
Social factors
(eg other people)
1.5 MEND
recorded
attendance
and son
2.3Why
did
each
member
attend?
4.1
Practical
factors
3.3 Factors
related
to the programme
(eg
content,
delivery)
1.6 Interviewer
observations
5
Behaviour
change
During
programme/
2.4What was
each
member’s
involvement?
Family 2 Mum, views of content
Home
Low
4.2 Social
factors
3.4
Family
factors
1.7 Individual
interview
following?
5.1
Did
you
change
any
of
the
things
you
spend
time/money
on
as
a
result
2.5Main memory
of sessions
4.3 Factors
toof
thediet
programme
6related
Types
costs
of programmedad,
3.5 Factors
related
to health,
or weight
5.2
Change
other
things
you
do Why?
2.6 How were
family
referred
to
MEND
4.4 Related
toyou
health,
dietand
or weight
6.1
Time
costs
constraints of involvement
gran, in programme
3.5 Anything
else
disliked
5.3
Any
weight
loss
during
MEND
2.7 Whose4.5
decision
to
attend?
7
Behaviour
change
–
after
theencouraged
programme
Things
you
liked,
or problematic
enjoyed
6.2
Time
costs
and constraints of behaviours
by MEND
3.6 Anything
else
you
found
daugh5.4
Experience
of
local
activities/clubs/services
discussed
by
MEND
leader
2.8 How did
child
feel
about
attending?
7.2
Any
changes
begun
in
MEND
maintained?
Eg
food
habits
or
4.6 Other
costs and
constraints (eg Probe
food
expenditure/transport
etc
3.8 What
would6.3
youFinancial
change about
MEND??
ter
5.5 What was
the most difficult thing
youcosts
wereeg
asked
to do? pool/sports; bike ownership)
2.9 Other
expenditure/transport/
sports
swimming
6.4 Impact on other activities not related to programme (eg activities for others in the family
5.6 Anything
was easy
to do?
7.3that
Changes
to fitness
since MEND
Family
3 Mum,
Home
Low
6.5 Other
5.7 Most useful
thing about
MEND?
7.4 Changes
toaweight
7.1 Recommend
to
friend?since MENDdaugh5.8 Other 7.5 Anything that has helped to maintain changes
ter and
7.6 Anything made it harder to maintain changes
sister
7.7 Other
Families – finding answers to the research question
What is the
salience and
acceptability and
types of costs
associated with
attendance and
sustaining
behaviours
afterwards?
Assess relationships for higher order dimensions?
On-going generation of higher order dimensions
•
•
•
•
Developing insight within cases: reading and re-reading interviews so familiar
with individual narratives
Developing insight across cases: assigning informal names to units of analysis
eg ‘gym’ family, ‘nurse’ family, single dad – and summarising/memorising broad
narratives of each (like study summaries in narrative synthesis – Popay et al )
“Tinkering” with data displays (like Miles and Huberman 1994 suggestions to
group cases displays by timing, role or even chart by theme and timing or theme
and role) – see eg over with data removed
Juxtaposing data from different categories to explore how they relate and
seeking typologies eg people who did X, or people who felt Y.
Miles MB and Huberman AM (1994) Qualitative data analysis. London: Sage
Popay J, Roberts H, Sowden A, Petticrew M, Britten N, Arai L, Roen K and Rodgers M (2003) NARRATIVE
SYNTHESIS: A METHODOLOGY REVIEW: Developing methods for the narrative synthesis of quantitative and
qualitative data in systematic reviews of effects. http://www.york.ac.uk/inst/crd/projects/narrative_synthesis.htm
(accessed 10/11/13)
Families’ understandings of their experiences
Salience (trigger)
Perception of changes in weight/behaviour
Happy with small changes (a few)only (a few)
F53/54
xxx
xxx
mum
It’s up to you (one)
P27 mum + xxx
xxx
daughter
Feelings about change or lack of it
xxx
xxx
It’s all part of a wider picture (5+)
F51 son,
xxx
xxx
xxx
daug + dad
Programme a ‘safe space’, away from those pressures, a unique opportunity to exercise I(5+)n a safe environment
F68 mum
xxx
xxx
xxx
and son
Feelings of about not facing down wider pressures (5+)
21 mum +
xxx
xxx
xxx
daughter
On reflection
• Coding onto transcriptions (plus summary chart with examples from
each category with counter examples?)
• Memos on emerging ideas
• Summarising by unit of analysis
• Charting selectively to explore/test theories/typologies (Miles and
Huberman 1994)
• Does our anxiety about being systematic and transparent –
managing data – detract from attention from knowing / “learning” it reading, re-reading, summarising – should be less ‘familiarisation’
and more ‘immersion’?