User Involvement in Mental
Health Research
Diana Rose and Jenny Walke
Service User Research Enterprise (SURE)
Institute of Psychiatry
King’s College London
Purpose



To demonstrate how service userfocused research can add something to
the methods and findings of
conventional research
To show why service user researchers
are best-placed to do this
Involving service users at all stages in
the research process
Overview



A little bit about one of us (DR)
Some introductory slides
Service user research





User-focussed monitoring
Systematic reviews
Outcome measures
Criticisms and answers
The NIHR continuum
Having a double identity (DR)

Service user all my adult life

1972-1986 academic career


1986 medically retired – ‘living in the
community’
1996 two identities come together to do userled research
Colney Hatch Lunatic Asylum (est 1854)
Friern Hospital corridor 1976 – I walked the
line
Friern Hospital patient (1984)
Camden Mental Health Consortium
Resettling Friern Patients





Camden – central and north London
Friern – patients to be resettled to this area
Camden Mental Health Consortium – one of
first local user groups
Formed to make sure the resettlement was
done well
Not user-led at this point – that came later
Service user research - history


1988 – CMHC and GPMH project on new
inpatient provisions in District General
Hospital
1996 – two user-led projects based in NGOs




Strategies for Living (Mental Health Foundation)
User-Focused Monitoring (Sainsbury Centre for
Mental Health)
Political: workers members of the user
movement and took research questions from
the movement
To make a contribution – our ‘evidence’ for
the movement
User research moves on (or
backwards)



Early 2000s two groups in universities
 SURESearch (Birmingham)
 Service User Research Enterprise (SURE)
 Service user researchers have ‘insider
knowledge’
SURE has ‘collaborative’ management structure – one
conventional university researcher and one user
researcher
SURESearch more complex but ‘head’ is conventional
university researcher
Activity
Now test your knowledge - correct answers revealed on final slide.
1. What was the original name of Friern Hospital?
a)
Bedlam
b)
Colney Priory
c)
Colney Hatch Lunatic Asylum
2. How many patients did Friern accommodate by the 1950s?
a)
500
b)
2000
c)
5000
3. What does CMHC stand for?
a)
Community Mental Health Clinic
b)
Centralised Mental Hospital Care
c)
Camden Mental Health Consortium
4. When were two reports from service user-led projects published?
a)
2000
b)
2002
c)
2005
Further Reading

Taylor B: The Last Asylum. London:
Penguin; 2014.

Survivors’ History Group, Mental health
and survivors' movements and context:
http://studymore.org.uk/mpu.htm
User-Focussed Monitoring
Method of Peer-Review of
Mental Health Services



Started in Central London in 1996
 Community services
 Hospital services
Expanded across UK and across service types
Now also used in Nordic countries
 Norway “User ask user”
UFM is User-Led



Co-ordinator who is a user but also with
research skills
Visits local area to identify service users
interested in project
Get together over lunch to make final
decision about whether they want to be
involved
Some UFM projects
questionnaire-based


Group devises questionnaire on the
basis of their experience of the service
to be assessed
Takes many meetings as co-ordinator
synthesises ideas and then brings back
to group for amendment
Training



Some involved in UFM projects have
never done an interview
Some basic training
But mostly role-play: opportunity to
experience answering as well as asking
the questions
More qualitative projects



Focus groups have been used
Topic guide devised in the same way as
for questionnaires but shorter as want
to give group scope to expand on their
views
Some training for this too
Collecting the information



Interviewers go to many different venues:
CMHTs, hospitals, participants’ homes,
charities
Focus groups not usually held in NHS
venues – more neutral
De-briefing – co-ordinator speak with
interviewer immediately after each
interview to check no problems
Analysing the data



For questionnaires there is some
statistical analysis
Problematic from involvement
perspective as few service users have
these skills
Easier for members of UFM team to be
involved in analysing qualitative data

Can bring their experience to bear
What happens to the report?



UFM projects are locally commissioned e.g.
Trusts, local authorities, charities
Report goes to commissioners
Up to them what they do with it



Some take very seriously and results in
measurable changes
Some take not so seriously!
Sometimes have feedback days for staff
Activity
5. When did peer review of mental health services start in Central
London?
a)
1983
b)
1990
c)
1996
6. Why aren’t focus groups typically held in NHS venues?
a)
Lack of space
b)
Lack of neutrality
c)
Lack of insurance
7. At what level are UFM projects commissioned?
a)
Local
b)
National
c)
International
Further Reading
Rose D, Fleischmann P, Schofield P:
Perceptions of User Involvement: a
User-Led Study. International Journal of
Social Psychiatry 2010, 56(4):389-401.
New method – patient-centred
systematic reviews
Example - ECT





Consumers’ perspectives on ECT
Two main researchers had experienced ECT – “insider
knowledge”
Adapted method of systematic review to make it user-focused
 Included peer-reviewed literature as normal in systematic
reviews
 Included the ‘grey’ literature
 Included qualitative data – ‘testimonies’
Assembled 26 clinical papers and 9 authored by service user
groups
Testimonies sourced from a video archive and the internet
Main themes

Perceived benefit

Retrograde memory loss

Information

Consent and perceived coercion
Perceived benefit



Conventional research showed much higher levels of
satisfaction with ECT than user research – no overlap
in the estimates of benefit between the two groups
Critique of method in grey literature
We answered with critique of method in clinical
papers
ECT continued




BOTH conventional and grey literature showed high
levels of memory loss though conventional did not
discuss this or even said it was not important
BOTH showed people did not feel informed but again
this absent from discussion in clinical papers
Perceived coercion – testimonies showed some
people felt coerced into signing form
Same results but different interpretations – not like
the finding on perceived benefit where there were
different results
The Dispute



Paper published in a high-profile
medical journal
Royal College of Psychiatrists issued
press release disputing what we had
said about memory loss
Mendacious – they mis-quoted
themselves
Policy




NICE new guidelines on ECT as our
research work being done
Consumer review influenced
Especially around information and
consent including about risk of memory
loss
User research can have an effect on
national policy
Activity
8. Which two of the following are omitted from orthodox systematic reviews?
a)
Grey literature
b)
Randomised controlled trials (RCTs)
c)
Peer-reviewed literature
d)
Qualitative testimonies
9. Issues of consent and memory loss are associated which of these treatments?
a)
Cognitive Behavioural Therapy (CBT)
b)
Occupational therapy
c)
Mindfulness
d)
Electro-convulsive therapy (ECT)
Further Reading
Rose D, Fleischmann P, Wykes T: Consumers'
views of electroconvulsive therapy: A
qualitative analysis. Journal of Mental Health
2004, 13(3):285-293.
Patient Generated Patient
Reported Outcome Measures
(PG-PROMs)
The claim of neutrality in
Randomised Controlled Trials (RCTs)

RCTs considered the ‘gold standard’ in medicine

Neutrality depends on blinding (not knowing certain details)

But is everything in an RCT neutral?

Outcome measures devised by clinicians and academics

May not be the outcomes that matter to service users

Try to develop measures that are valued by service users
(and others) in mental health
Patient Reported Outcome
Measures (PROMs)



Much talk of PROMs
But only filled out by patients – no say
in which questions are asked
Patient-Generated PROMs (PG-PROMs)


Medication side-effects
Experiences of inpatient care
Method



Participatory research
Attempts to reduce the power relations between
researcher and researched
In user-focused research, researchers have the same or
similar experiences as the participants

All are mental health service users

A new development even within participatory research

Potential influences on treatment and policy
Procedure








Focus Groups which meet twice
Recruited because they have experience of what the measure is
attempting to tap
Facilitators/researchers have experience of the treatment or
service that is being evaluated
On basis of focus group discussions, researchers draw up draft
measure
Taken to Expert Panels for amendment and refinement and that
the language is their own language
Feasibility study to make sure it is easy to complete –
refinements all the way
Psychometric testing
About 200 participants involved in all
Example – in-patient care





Much anecdotal evidence and evidence in the
‘grey’ literature that profoundly disliked
My experience too
Wanted to do something more rigorous
Finally became the main outcome measure in
an RCT evaluating the introduction of
psychological therapies on acute wards
Collaborative but our part is user-led
User Measure: VOICE
Participants


People who had been in-patients in the
local Trust within the previous two
years
One group specifically made of
participants who had been detained
Focus groups

6-8 people: 4 groups

One facilitator with experience of in-patient
care and other also a service user

Meet twice to make sure we have accurately
captured their views

Thematic analysis using Nvivo software after
both 1st and 2nd wave
Drafting the measure

Done by the researchers on the basis of
the qualitative analysis

Quantitative and qualitative questions

Researchers also use own experience
Expert Panels

One drawn from focus group members and
one independent

Also been inpatients in previous two years


Tasked with amending and refining the
measure and making sure that language and
layout are appropriate
Usually quite a few changes made at this
stage
Feasibility study




Final stage of constructing measure
~50 people complete the measure and
we find out which parts are easy to
complete and which not
Iterative process
In this project the participants were
actually in hospital
Psychometrics



Mixed methods research
We do this because it is appropriate but also
to show that user-led research can be
rigorous
One mainstream researcher said users could
never produce measures because they would
never understand factor analysis
Psychometrics Continued





Test-retest reliability: the same people fill in the
measure twice with an interval of a week in between
Are the scores the same the 2nd time? This would
mean the measure is stable.
VOICE very stable
People in this exercise mostly a diagnosis of
psychosis which might compromise stability.
Good test-retest reliability because measure
developed by service users?
Psychometrics cont


Criterion validity: assess measure against an existing
one
 Expect some differences because of means of
development: close relationship but there were
differences
Acceptability: how does it feel to fill it in
 Enjoyable?
 Distressing?

Right length?
Activity
10. Who compiles most existing outcome measures?
a)
Patients’ families and carers
b)
Clinicians and academics
c)
Hospital administrators
11. What is ‘blinding’?
a)
Conducting research without obtaining consent
b)
Assigning participants to different groups
c)
Temporarily withholding information to reduce bias
12. What name is given to techniques for finding out whether a
measure is stable and valid?
a)
Feasibility study
b)
Psychometrics
c)
Expert panels
d)
Quantitative methods
13. What is the main purpose of a feasibility study?
a)
To ensure a measure is easy to complete
b)
To check cost-effectiveness of a measure
c)
To advertise your research
Further Reading
Evans J, Rose D, Flach C, Csipke E, Glossop H,
McCrone P, Craig T, Wykes T: VOICE: Developing a
new measure of service users' perceptions of
inpatient care, using a participatory methodology.
Journal of Mental Health 2012, 21(1):57-71.
Challenges to user-led
research and user-produced
knowledge
Frank Scepticism

Peter Tyrer, past editor of the British
Journal of Psychiatry, writes:
“The engine of user involvement, while
welcome in principle,……….may drive
mental health research into the sand.”
Power



Most of the projects we have been involved with are
‘collaborative’
Nearly always headed up by professor(s) of psychiatry or
psychology
Not just status or naked power although that exists– more
subtle

Are you a researcher or are you a patient?

“I wonder what your diagnosis is, then.”

Undermining user-produced knowledge
Hierarchies of Evidence



RCT is the ‘gold standard’
But RCTs are not neutral – outcome measures devised by
clinicians
‘Expert opinion’ counts as evidence but only as the
weakest form

Experts are psychiatrists

Users as experts – a different knowledge perspective

The Cochrane Hierarchy needs revisiting
The charge of bias 1






Said, mostly implicitly, that user-research is biased,
anecdotal and carried out by people who are overinvolved – ENMESH conference
We make no pretence of neutrality
But all research comes from a certain standpoint
Conventional researchers think what they do is
‘obvious’ – nothing is obvious
User researchers more explicit about this than
mainstream researchers
In my opinion the word ‘bias’ should be banished
from research discourse and all researchers should
clearly say where they are coming from.
Bias 2




Although implicit some seem to think that
irrational people (the mad) cannot engage in
the supremely rational activity of science
We epitomise ‘unreason’ (Foucault)
So more difficult to find legitimacy and
credibility than consumer researchers in other
medical disciplines
Paradox – user research in mental health
ahead of that in other specialties
Some answers?
A new epistemology for user-led
research – learning from feminism

‘Timeless oppositions’:





Reason / unreason
Culture / nature
Intellect / emotion
First are male attributes and valorised
So have women been excluded from
science and science not attentive to the
concerns of women
Timeless oppositions and madness


The mad positioned as nature, emotion
and crucially unreason
The Enlightenment valorised reason and
so positioned the mad as its antithesis:



Unable to reason
Unable to labour
The Great Confinement
Standpoint epistemology and
strong objectivity




Standpoint epistemology has been seen as
essentialist
Sandra Harding and ‘strong objectivity’
We have access to two discourses
 Our own
 That of conventional science
A more complete picture – even a challenging
one
Implications


User-led research can add to the jig
saw or it can challenge conventional
research
But needs to critique the methodologies
and epistemologies of conventional
research to do this successfully
Other ways service users may
become involved in research







Everybody in SURE employee of IoP
BRC Service User Advisory Group (SUAG)
Reference groups for projects e.g. The
Interface Study
Sitting on Steering Groups
Problem of tokenism although in SURE also
Should be parity of esteem and payment
Training resources for service users
INVOLVE (NIHR) Continuum
Consultation Collaboration User-control
Collaboration split
Researcher-initiated
Jointly initiated
User-initiated
Activity
14. The Cochrane Collaboration is associated with which type of evidence?
a)
Systematic reviews
b)
Focus groups
c)
Ethnography
d)
Service user research
15. Which two of the following statements are true?
a)
RCTs are neutral
b)
RCTs are considered the ‘gold standard’ of evidence
c)
Service user research is neutral
d)
User-led research can reduce the power imbalance in the research process.
16. Sandra Harding used which term to describe research by groups traditionally excluded
from knowledge production?
a)
Enlightenment
b)
Strong objectivity
c)
Unreason
d)
Evidence-based medicine
17. Which is NOT a level of user engagement on the INVOLVE continuum?
a)
Consultation
b)
Collaboration
c)
Facilitation
d)
User-control
Further Reading
Sweeney A, Beresford P, Faulkner A, Nettle
M, Rose D (eds.): This is Survivor Research.
Ross-on-Wye: PCCS Books; 2009.
Activity Answers
Q.
ANSWER(S)
Q.
ANSWER(S)
1.
c
10.
b
2.
b
11.
c
3.
c
12.
b
4.
a
13.
a
5.
c
14.
a
6.
b
15.
b, d
7.
a
16.
b
8.
a, d
17.
c
9.
d