National Centre for Research Methods (NCRM)
Networks for Methodological Innovation
Dancing with new partners: developing novel research methods to establish and
monitor impacts of user engagement in times of austerity
Report of Workshop 3: Engaging ‘Hard to Reach’ Users
23rd March 2012, Gallery North, Squires Building, Northumbria University
Welcome and introduction - Purpose of NCRM Network
Chair for the morning session: Ann Clark, University of the Highlands and Islands
Professor Irene Hardill, Programme Co-ordinator, Northumbria University
Two interlinked imperatives give this Network its focus:
•
•
methods for engaging non-academic users - to date user engagement has placed a particular
emphasis on engagement with policy and practice, we also wish to consider methods of
engagement with people who, in our experience, are often resistant to engagement;
methods for measuring the impact of research on policy and practice
User engagement and impact is very much to the forefront of academic debates in the UK
But as we know research that aims to engage with research users and have an impact is not new,
indeed within the social sciences there are a number of epistemological traditions - feminism,
participatory research, disability studies and the more recently emerging field of sustainability science
- that foreground user engagement, while mainstream social scientists have traditionally preserved
their ‘distance’ from their research subjects.
The NCRM Network brings together three existing networks.
•
•
•
Engaging Scottish Local Authorities scheme which was funded by ESRC/SFC, and comprises
five project teams, plus Network Co-ordinator (Dr Rehema White, St Andrew’s University)
The New Dynamics of Ageing (NDA) funded project, Sustaining IT use by older people to
promote autonomy and independence Sus-IT, which is led by Professor Leela Damodaran (of
Loughborough University) along with a team at Memorial University, led by Professor Wendy
Young who have funding from CIHR for Sus-IT Canada.
The third network is the Academy of Social Sciences (AcSS), which is composed of over 700
Academicians and most of the UK’s social science learned societies, which reaches a
significant proportion of UK social science community.
The Network has five aims:
1. to support the growth of capacity across UK practice-related social science disciplines to
undertake engaged research which is of benefit to research users, thereby and enhancing the
quality of Knowledge Exchange (KE) and demonstrating the wider value of social science
1
2. To Facilitate methodological developments in user engagement with: ‘hard to reach’ users,
private sector companies and ‘front line’ professionals engaged in service delivery in economic
austerity drawing on ESLA and Sus-IT, maximising the value of the recent ERSC, SFC, and
LARCII investments
3. To establish an international network of existing and new researchers with a shared interest in
advancing thinking and improving practice;
4. To target specific activities to involve early career researchers and doctoral students in
innovative ways of working with research users;
5. to produce and share attractive, accessible resources in various media that will inform and
help to guide practice.
In addition to attending events, the Network reaches other participants via Twitter
(@newpartnerskt # hardtoreach)
Stephen Anderson, Executive Director of the Academy of Social Sciences
The notion of dancing brings the ideas of linking, coming together, partnering. Often one partner is
hesitant. Trust is essential for good dance. Producers and users of research often find it difficult to find
the right partners and communicate. If you link up to dance you have to dance to the same tune and
hence agree what the tunes are – yet users, and researchers, often have set agendas/tunes.
Choreography and timing are critical; need to be in step with themes, talk at the right time, and join
with people in time.
The Academy of Social Science has 750 Academicians plus 43 learned societies, making 85,000
members. They will hold a conference in early July on the practice of engagement, and will look at
lines of communication and processes. Users such as government departments also find this
communication difficult.
Keynote Speaker: Professor Rachel Pain, Durham University
Experiences of user engagement, including ethical and practical challenges
Striking a blow or walking together?
Institutional influences are complex. It is difficult to scale what we are supposed to do in user
engagement and impact. For example, we can work regionally and locally, but drivers such as the REF
push for global impact and scale of use.
Participatory Action Research is a specialisation of the Centre for Social Justice and Community Action
at the University of Durham. In this approach there is a focus on co-production of knowledge and the
distinctions between users and producers of knowledge are less clear.
The Centre has an Arts and Humanities Research Council Connected Communities grant, led by
Professor Sarah Banks, which is developing principles and guidelines for community research and is at
consultation stage.
Participatory Action Research (PAR) is a contested concept. Fine et al (2007) argue that PAR can invert
who constructs the research questions and contest traditional hierarchies, but Cooke and Kothari
(2001) argue that participatory processes can be a way of maintaining existing hierarchies and power
structures.
Participatory methods try to allow more open engagement with participants including GIS, mapping,
diagramming, storytelling, drama, peer interviewing. But methods don’t make the user engagement
2
or participation. The ethos, ethics and collaboration are more central (Kindon et al 2007). The
question is not so much how to access hard to reach users but more do I want to open up my research
in a collaboration with others?
Ethical and practical challenges include:
 Participation as a multiply positioned, contested object
 The impact agenda may be further evidence of neoliberalism but also provides opportunities
to counter it (Gregson et al 2011)
 It may be a welcome recognition of the validity of relevant, engaged research
 But critics defend what they see as academic freedom and autonomy
Disjuncture
What are we about? Who shares our goals?
The particular form of impact requested by processes such as the Research Excellence Framework
reproduces certain knowledge hierarchies and could squeeze out new ways of thinking.
A feminist lens offers an example to critique. We can develop a dichotomy of masculine versus
feminine forms of impact. The current mode is masculine – it focuses around university power and
large scale, significance, striking, a blow, a linear process of doing research and seeing outcomes.
Which universities will be rewarded by the impact agenda and which researchers will benefit? Not the
casual, part time researchers probably. On the other hand PAR is often long term, circular, offers
shared, diffuse, distributed benefits. This is more of a feminine form of impact.
3 principles for participatory impact
1. impact is a 2 way relationship
2. scale of engagement does not equate to quality of impact
3. impact can occur throughout research processes, not just from research outputs
(Pain et al 2011)
Currently formal, policy and economic impacts are being supported, and participation with grassroots
or activist movements is diminished. Currently research and impact are separated in time and
concept, but in reality impact may occur during the process and in an iterative manner. Universities
will reorganise to try to reproduce the form of impact that is being rewarded.
Participatory processes examples
 PAR on young people, fear and geopolitics. The young people focused on racism, bullying,
stereotyping. The process used arts and diagramming and other methods to develop the
themes for focus. Research outputs were co-authored with the organisations or young people
themselves.

What shall we do with the rivers research? PAR meets REF – the rivers trust wanted to
develop a way of using local knowledge to influence policy makers. It was a long term process
to develop goals and collect mostly quantitative data. This process gave ownership to the
trust and developed an innovative model, but is not deemed wide ranging impact for REF.
Towards a transformative quiet politics (Kye Askins)
Knowledge co-production more diverse, porous, - representative perhaps of a more feminine mode of
impact. In this sense small transformative interactions occur, and knowledge production is
acknowledged to be relational. Changes outside the university can be maintained when they are
embedded in people, organisations and places involved in the research.
3
The communiversity
Beyond individual projects
Scaling up participatory research
Conclusions
 Evidence calls reflect a deflection from deep accountability (Fine 2012)
 Alternatives to what? become more important in times of austerity
 Be optimistic about critical mass growing
 Keep producing alternatives through practice
 Support yourself and others through networks
 Remember failure is normal and surprise inevitable
 Participation as a multiply positioned , contested object in these debates – opportunities for
socially just collaborative research as well as threats.
Research users’ perspective on engagement
Lyn Russell, Independent Service User Adviser
Lyn has worked for the NHS in an advisory capacity in many roles including setting up and working in
day services for clients with personality disorder (PD) & lately with research at Sheffield University.
She was a research team member in a study looking at acceptable and unacceptable aspects of
psychological therapy. She was able to influence the design and management of the project,
challenged jargon and was able to present and interpret the results to others. Service users felt the
benefits of involvement – feeling valued, gaining confidence. The collaborative approach led to a
more honest and easier engagement for service users. It was hard to leave the users after the research
sessions. Hearing distressing stories was difficult. In the long term they hope to influence local users
and NHS.
Laura McIntyre, Cyrenians
The Cyrenians work with people in Newcastle often excluded from society, e.g. addictions,
homelessness, victims of trauma. Laura works mostly with adult women involved in sex work or
exposed to sexual exploitation. Laura has previously worked with the University of Northumbria on
peer research and also in 2007 she worked in partnership The Voices Heard Group on the “Hidden for
survival” peer research project into the lives of women sex workers in Tyne and Wear, which is a
hidden activity in Newcastle. Through peer research this allowed women to develop the
questionnaires and help deliver the project. This resulted in a change in service provision and
disclosure where women can now access health services etc.
Lately Laura has been working with Northumbria University on a new piece of peer research which will
contribute towards influencing a local strategy on prostitution. Other areas of focus are on promoting
education, working with a different client base, including women who are working actively as local
escorts or trading sex for favours or trafficked. Their aim is to interview 60 women and 15
stakeholders to identify how to overcome barriers to services.
It is an ethical challenge to work with women active in sex work. But these women can be empowered
to build their confidence and self worth which enables them to go on to access services. So
participation helps the women change as well as changing the services themselves.
4
Group discussions on issues raised
Facilitators: Wendy Olphert (Loughborough University), Dr Wendy Young (Memorial University),
Mark Smith (Edinburgh University), Tom Wakeford (Edinburgh University), Ann Clark (UHI))




Opportunity to reflect on the morning session
Discuss how own experiences are same/different
Develop 2 key points to reflect back to plenary on hard to reach and methods
Suggest alternatives to hard to reach
Group 1
Often participants had high expectations or expectations raised by researchers. to be honest about
who is object and subject in participatory research. Participants need to be permitted to influence the
research questions. In some cases people refuse to be reached e.g. because of their bad experiences
in the past.
Group 2
Challenges of language can be a barrier to developing trust and sense of mutual ownership, for
example how to explain research policies. Need support for researchers and participants, for example
when dealing with difficult ethical issues, vulnerable groups or where issues of personal safety for
researchers. Gatekeepers can be hard to deal with People can be hard to reach but not necessarily
vulnerable, for example people involved in organised crime. People are free to exercise a choice
about whether to engage with research….they may have other ways to spend their time and energy
which is more important for them.
Group 3
Practical methods to engage – stop thinking of transactional relationships and instead open up
dialogue for different areas of research. Hard to reach should be used contextually. Academics hard
to reach for ‘users’.
The group discussed the merits of the term ‘hard to reach’, and came to the conclusion that it was an
inappropriate term, as different groups can be hard to reach in different contexts. Academics need to
move away from seeing relationships with external stakeholders as transactional, and towards
focusing on the best outcomes for all beneficiaries. The best way to engage the ‘hard to reach’ is by
ensuring there is value for all users. The opportunities to engage are often related to this value. Space
and time were identified as the main obstructions to engaging ‘hard to reach’ users. A potential
solution is to focus on the transfer of existing academic knowledge rather than the creation of new
knowledge. The group discussed a variety of engagement methods including, but not exclusively,
‘speed dating’ and ‘slow dating’, trustee programmes, and collaborative relationships.
Group 4
One participant was surprised by the quantitative mode of impact described by Rachel, because in
English Literature impact tends to be mostly collective, cumulative, long term and influenced by
exhibitions etc. How do we measure contact with others and development of relationships? We can
offer examples but can’t easily quantify.
The reality is that impact at universities is expected to be at least national standard and very much the
model of wider scale impact Rachel described. Current university supported notions of impact favour
large projects against individual narratives. Interpretations of REF guidelines might vary across
universities – we need to challenge the ‘safe’ university favourite mode. It depends on the disciplinary
assessment unit within which you operate. Deep, rich research is disadvantaged. If we don’t challenge
these rules, they are self perpetuating. Hard to know who to challenge. Hard to measure people’s
5
capacity growth. Adapting measures for user group is hard. Publishing this kind of research can be
hard – some departments don’t support this kind of work. We end up having to do parallel lines of
research for REF journals and research that we want to do/participatory/engaging; legitimate versus
real research. Moral obligations to tell the stories you are provided. but at the same time loss of
academic freedom because Universities force you into REF based channels. ESRC and AHRC etc now
requiring engagement research but Universities have not caught up with this. Public service unit can
help facilitate the time taking establishment of partnerships and events. Civic university is a driver at
Sheffield – new Public Services Academy changed to Knowledge Exchange for Social Sciences. Should
this be separated from academic role? The position ‘impact fellow’ has now been created and
knowledge exchange brokers.
Challenges of engaging ‘hard to reach’ groups – need to invest in personal relationships and trust.
Need to know about culture, acceptable process, lots of preparation and gatekeepers then be alert to
changes and issues. The research process needs a lot of thought. This time and effort is
hidden/unappreciated by funders, universities. Would be good to have a contingency fund for
community research. Hard to cost this time in project. Hard to determine participant incentives.
Now becoming acceptable to include participant fees in project proposals – or is it?
The group preferred the term ‘Easily ignored ’rather than ‘hard to reach’.
Group 5
Is research a good use of time for participants? Language is important. Methods (detailed methods)
are different from methodology (overarching theoretical perspective). What is driving the current
innovation for new method?
Group preferred ‘people whose voices are seldom heard’ to ‘hard to reach’.
Challenges:
1) Users can simply be more interested in other activities rather than getting involved in research
whether this be significant things happening in their lives or that they feel that devoting their
resources to other tasks such as campaigning for change is more important than research. This
is not simply an issue of time but also that the arguments for the importance, meaning and
relevance of research still need to be made (and won) for many users;
2) The researcher(s) and the users may have very different agendas for the research. This needs
to be recognised, negotiated and managed;
3) Accessing “hard to reach” groups can be difficult. Users with certain “hard to reach”
characteristics may be forthcoming but are these individuals simply the “usual suspects”
within these groups (e.g. confident, articulate etc). Are these users representative of a wider
group?
4) Managing identities in such research e.g. users could be put off by labels such as “government
researchers”;
5) The language within this debate can be problematic. “Hard to reach” still has a certain
positionality and also implies that researchers have tried but that it is too “hard” to reach
them. “Chronically excluded” keeps some aspects of positionality but it is less loaded,
however, some felt that it sounded too much like a disease. The group offered “people whose
voices are seldom heard”. The group liked this because the core of the label is “people” who
are active but are ignored by those who should be listening. Whilst recognising that language
is important it was also acknowledged that all language can be problematic and that it is okay
to have different terms in different contexts i.e. positionality is not such a problem if all the
people using the term are from a certain position.
6
Methods:
1) The key point on methods is that they should be directed as part of the participative process
of user engagement;
2) An important distinction between methodology (overarching theoretical perspective) and
methods (the details of data collection) was made;
3) Whilst acknowledging the need to develop new methods for specific contexts, the group
questioned the drive for innovation in methods, which some argued has become slightly
fetishised. In particular, some felt the drive for innovation in methods can be driven by the
quest of the researcher or funder to be a pioneer rather than by the needs of users.
Innovative Dissemination Methods
Chair of afternoon session: Professor Irene Hardill
Innovative Dissemination Methods: Bright Club Session, James Lord
http://brightclub.org/
“Where funny meets brains” – where researchers provide information for the general public via
stand-up comedy. Bright Club is a network of local comedy clubs. It has sold out shows at 500 tickets
a time. It is designed for researchers to enthuse people about their research. There is a comedian in
residence that helps develop a routine and acts as a compere on the night.
James gave a short example of Bright Club in action by delivering a routine based on his own research
as a PhD student looking at how to improve the effectiveness of hip replacements.
Engagement in Practice Workshops
Engaging Young People and Parents: Professor Angie Hart, University of Brighton
Angie Hart doesn't use the term 'hard to reach'. Academics are hard to reach. People she works with
are often the most in need with the fewest chances.
Example of collaborative work with young people:
The Mental Health and the Resilient Therapy Toolkit is written by young people, for any parent or
carer who is at all concerned about the mental health of their child
(free to download from www.boingboing.org.uk) See www.boingboing.org.uk for other examples
of collaborative work.
Easy ways to involve people in collaborative work with academics:






Become useful to a local organisation
Brainstorm your research questions with community partners
Have YP or parents as part of your supervision team (advisors)
Ask them to comment on information sheets etc. and ethics submission
Apprentice yourself to someone else
Give a talk on your work and strategically invite people
7


Make things as easy as possible for people to join in
Be careful about how much support you can give
Engaging Practitioners and Service Users:
Mark Smith, University of Edinburgh and
Ann Clark, University of the Highlands and Islands
Mark and Ann gave presentations about two projects from the Engaging Scottish Local Authorities
Programme, a knowledge exchange pilot supported by the Economic and Social Research Council, the
Scottish Funding Council and the Local Authorities Research Council Initiative.
Ann Clark’s project involved researchers from two Higher Education Institutions working with staff in
two rural Scottish local authorities to: search for, generate and apply knowledge about best practice in
dementia care at home in rural and remote areas; and to build institutional capacity for future
evidence based practice. Key challenges of KE with staff included: the impact of the operational
demands of social services delivery on the ability of LA staff to protect time for KE, differences
between academic and LA paradigms – LA staff found evidence from peers most useful, lack of
research of practical use to LA staff, difficulties in building and sustaining interactive relationships
within a rural context using ICTs. Engaging people with dementia in research is possible but needs
time, resources and understanding of people’s different capacities. Research literature on
engagement methods was useful and experience of project supported its findings. Useful methods
recognise that people with dementia not a homogenous group, use qualitative, narrative methods
that seek feelings and stories not facts. Need to use triangulation and variety of methods to confirm
findings. Communication aids and methods involving drama, art etc may be successful.
Relationships are important in developing effective KE. Discussion of skill set one requires for this
work – is it practical for all researchers to develop this relationship development ability as well as
other demands? Mainstream research funding and assessment does not ‘fit’ with doing engaged
research. A lot of the funding around is for doing and disseminating rather than planning. Perhaps we
need a specialist funding stream for this. Sometimes lit reviews might be more useful than new
research. You have to be flexible in your dealings with service users.
Engaging Older People:
Professor Leela Damodaran, University of Loughborough
Older people are very diverse. There is a paradox where on the one hand they are diverse but also
cohort effects such as experiences of war, rationing, decimalisation so also some shared aspects. How
do you influence policy makers etc? Individual stories can be translated into better design or retail and
other sectors.
Engaging Beyond Methods:
Tom Wakeford, University of Edinburgh
In times of austerity, is participation difficult to do democratically because society is even less equal?
Contradictions within universities – on the one hand there is a stated commitment to community
embeddedness and engagement but on the other there is an over-riding push towards universities
becoming corporate entities that are accountable to the free market.
Should we pay participants? Depends on whether research questions are developed by community or
by researchers; hard to compete with market companies; need to value time of people; consider roles
of people (eg community consultants). If people get recurring payments, benefits are affected, but you
8
can give a one off up to £2000 payment without benefits being cut. Communities could be paid in kind
-
depends on the project itself. People can be disenfranchised by the bad research experiences;
or frustrated that their contributions don’t lead to change immediately and visibly.
In health, INVOLVE have a good report guidance on payment. - http://www.invo.org.uk/
What is research for? What is in it for publics who get involved in collaborations? Financial motive (eg
project officer funding); interest in the subject. People could have process (coffee, conversation,
empowerment) or outcome (change on policy, change in practice, community wins) benefits from
participation (or both).
Timescales of interactions – process can take a long time. We are driven by processes with 3 year
funding cycles. But long term relationships enable projects to happen when funding comes or a need
is identified. Timescale depends on project aims, existing relationships, frequency of meeting,
whether you live in the community or not etc, expectations and needs of participants. Need to also
allow new people into the loop. It can take 2 / 7 years (from examples) to work with service users or
communities to develop the questions before you can even submit the proposal. The truly
participatory part of the research is thus often not funded! Then there is a time lag before you hear
about funding. Business consultancy expects to wine and dine to get funding eventually. You can
maintain relationships to some extent by touching base occasionally.
Who owns the data? We can have collaborative approach to interpretation but researchers tend to
keep an editorial role. A funder can try to manipulate or not report particular findings.
What is a genuine participatory approach? Is it flavour of the moment and thus in danger of becoming
diluted misunderstood? Does it all have to be participatory? Can we be honest about when we should
just consult? Practitioners are sometimes better than researchers at being explicit about when it is
inform, consult, involve, empower. It is important to select the appropriate level of participation for
the context of the project. You should remember the participation principles.
Difficult to get people to participate in some areas and around some issues e.g. looking at coastal
management with people from urban areas. Rosemary.barber@sheffield.ac.uk says see Telford 2004
paper on principles.
Plenary
Critical reflections on user engagement:
Rosemary Barber, University of Sheffield
Language is a key topic, as has been demonstrated today. Many different terms are used to describe
‘users’, including: ‘service users’, ‘the public’, ‘consumers’, ‘patients’, ‘lay people’ and even ‘PPI
agents’. Perhaps more contentious is what to call the process of actively involving or engaging users.
‘Engagement’, ‘involvement’, ‘participation’ and ‘user controlled’ have different connotations.
Language matters as it reflects ways of seeing and portraying people and their relationship with health
and social care systems (Bastian, 1994), and is associated with conceptual, ethical and political issues
e.g. consumerist vs. democratic approaches (Telford et al., 2001).
Why evaluate impact? Very little is known about the impact of user engagement, and it is suggested
that there is a need to seek better ways to do it, to share good practice and to be aware of possible
deleterious effects on research and on individuals. However, it has also been argued that user
engagement is a moral issue, and therefore not amenable to evaluation. Others argue that it is too
9
difficult to separate contextual and organisational factors and therefore too costly and not feasible to
undertake.
User engagement is about relationships, and it is challenging to evaluate a process that is subjective
and socially constructed (Smith et al., 2008). Possible methods of evaluation could draw on the
following: principles of successful consumer involvement (Telford et al., 2004); developing and
evaluating complex interventions (Craig et al., 2008); realist evaluation (Pawson and Tilley, 2008) and
the quality involvement and questionnaire (Morrow et al., 2010).
Website for INVOLVE: http://www.invo.org.uk
Winners of the postgraduate poster competition were:
Roni Naheed Nawazesh, PhD Student, Manchester Metropolitan University and Emilia Pietka, PhD
Student, University of Strathclyde who both received a year’s Associate Membership of the British
Academy of Social Scientists and book tokens as a prize.
The next Workshop is to take place on 10 May 2012 at Leeds University, and will address the theme
of research education.
Contact: Paul Ellwood, P.Ellwood@lubs.leeds.ac.uk
10