BOD 14/2012
For Publication
BOARD OF DIRECTORS
MEETING – 11 JANUARY 2012
MYSTERY SHOPPING PROJECT
EXECUTIVE SUMMARY
This paper provides a brief description of the Mystery Shopping project undertaken
in October and November 2011 and the emergent findings and recommendations
THE BOARD IS ASKED
To note the actions taken and approve the recommendations for further action
Regulatory framework
This report reflects priorities set by the Board of Directors stated as Objective 7:
“Engage meaningfully with service users, carers and the local community and align
our services to their needs”
DATE:
LEAD DIRECTOR:
23 December 2011
Andy Mattin - Director of Nursing & Operations
1
1.
Introduction
1.1
This paper provides an overview of the methodology used for the Mystery
Shopping project, the findings from the various shopping “expeditions”, the
outcomes of the project and the recommendations for further action.
2.
Background
2.1
Mystery Shopping is a customer feedback tool quite commonly used in the
commercial sector but which is beginning to attract support in health care
environments. Research into its usage in the NHS and particularly mental
health settings showed that it is rarely used, but it has been adopted more
readily in social services settings to test the suitability of support services.
2.2
The Mystery Shopping project has been running at CNWL since 2009 when a
pilot project showed that this was a viable method for gaining valuable
feedback about the way in which our services are experienced.
Since then there has been a steady expression of interest in this project from
other NHS organisations.
2.3
2.4
The overall driver is to identify areas for service quality improvement. In
October / November 2011 the third phase of this project continued to look at
reception areas across the Trust, plus the PALS services, and a first attempt
was made to recruit carers as mystery shoppers during their visits to patients
on the wards and residential units.
3.
Methodology used
3.1
The pilot project had identified that the methodology for mystery shopping
visits to Trust reception areas and telephone contact maintained the
anonymity of the mystery shoppers and provided a good measure of a real
response. It was therefore decided to continue using this methodology but with
a revised scenario and identity for the “shoppers” visiting the reception areas
and making telephone calls to PALS Linkworkers. For the third element the
shopping team were asked to recruit carers and visitors at inpatient sites for
them to act as mystery shoppers at their next visit.
3.2
The focus of the mystery shopping expeditions where to test the degree to
which CNWL staff respond to the enquiries made by the shoppers. The
mystery shoppers assessed whether services provide appropriate information
and advice, and in the case of reception areas made observations on access,
signage and the availability of information in the form of leaflets and posters.
The choice of the carer focused element was to try to get a perspective of the
experiences and observations of “real” carers when visiting relatives and
friends in the inpatient services. A concern often cited by carers is that they do
not receive a very welcoming response when they go to visit on inpatient
wards, and are often not provided with sufficient information.
2
3.3
Mystery Shoppers for this phase of the project were recruited from previous
participants in the scheme, plus a few new members who responded to an
open invitation to Foundation Trust members who have identified an interest to
be involved in service monitoring. A team of 23 service user and carer
shoppers each selected areas where they would undertake their shopping
expeditions.
4.
Specific findings from the Shopping areas
The results of the project have only just been collated and are yet to be fed
back to the services. There is some variance in the experiences of the
shoppers. However it is also worth attaching a caveat to this because it was
noted that the individual subjective standards or expectations of the
“shoppers” will have influenced their assessment of the service. An example of
this is the assessment of the signage of toilets in reception areas – a fairly
factual observation - where separate shoppers at the same reception area
have assessed these as being both well signed and poorly signed. It is also
recognised that the experiences of mystery shoppers will be strongly
influenced by the behaviour of the CNWL staff who they encountered.
4.1
Reception areas - the mystery shoppers made a total of 128 visits across 88
Trust sites.
4.1.1 69% of the sites were assessed as being clean and tidy which is an
improvement from the 65% score in the last mystery shopping exercise
in February – March 2011. 4 mystery shoppers described the reception
areas they visited as being dirty, and 2 of the mystery shoppers could
not get past the intercom.
4.1.2 59% of the reception and waiting areas were described as having a
pleasant and welcoming décor, which is an improvement from the less
than 50% score at the last mystery shopping exercise.
4.1.3 62% agreed that the reception areas were clearly signposted.
4.1.4 67% of the shoppers said that the receptionists were pleasant, polite
and helpful, which is a considerable improvement on the 53% score in
the last survey.
4.1.5 55% agreed that the toilets were clearly signposted, which is an
improvement of the previous score of 40% but obviously still not good
enough.
4.1.6 58% of the shoppers reported that CNWL PALS information was clearly
visible, but only 49% (compared to 62% in the previous survey) said
that there was a good selection of other information leaflets available.
53% (compared to 26% in the last survey) said that there was
information about local support services available. 39% of the mystery
shoppers were given extra written information which they found helpful
and 52% (compared to 43% in the last survey) found the verbal advice
given to them as being helpful.
4.1.7 75% of the mystery shoppers agreed that the receptionists
acknowledged them quickly on arrival, 83% felt that they were greeted
appropriately, and 57% of the receptionists had made every effort to
deal with their (fictitious) problems. 49% of the mystery shoppers said
they would be happy to use the service again.
3
4.2
PALS Linkworkers
4.2.1 Overall 71% of the mystery shoppers rated the responses and manner
of the PALS linkworkers as either excellent or good.
4.2.2 A total of 51 PALS Linkworkers were contacted by the mystery
shoppers but only 29% were able to get through to the PALS linkworker
immediately.
4.2.3 Of those who did get through to the PALS linkworker only 47% of the
PALS workers gave their name, 36% were asked to call back at a
different time and only 23% were asked to leave a number where they
could be called back at a more convenient time.
4.2.4 Once the mystery shoppers began talking to the PALS linkworkers and
had explained their scenario, 54% were satisfied with the information
provided by the PALS workers and 67% felt that the PALS workers
answered all their questions.
4.2.5 41% of the PALS linkworkers offered suggestions of other sources of
information that the mystery shoppers might want to follow up, but only
15% offered to send additional information.
4.3
Carers visiting wards
4.3.1 The mystery shopper team managed to recruit a total of 22 carers to
undertake a role of mystery shopping while visiting their friends and
relatives in residential care. These visits took place over 8 sites. The
volume of visits per site ranged from 8 on one site to 1 each on 3 sites.
The volume of visits in total is small and some caution is needed in
analysing the results on the basis of percentage scores, however the
results and the additional comments provided by carers give some
indications of where improvements could be made. There was a
variance of responses from carers about their experiences ranging from
excellent to poor.
4.3.2 The majority (68%) of carers were female and white (73%), although
there was an even spread of carers across the Black, Asian and Other
ethnicity groups.
4.3.3 100% of carers reported that they had no difficulties in getting to see
the person they wanted to visit. One carer reported some difficulties in
parking.
4.3.4 91% of carers reported that they were made to feel welcome on the
ward, however 2 (9%) did not experience their welcome positively.
4.3.5 19 of the 22 carers (86%) reported that there was a suitable place for
them to make their visit in private
4.3.6 Only 10 (45%) said that the ward staff introduced themselves to the
carers, although a further 5 (23%) said this was not necessary as they
were already known. However this still left 7 (32%) who had not been
introduced to staff.
4.3.7 90% of the carers said that the ward staff were available for them to talk
to, and 77% said the ward staff were approachable and easy to talk to
4.3.8 Only 7 (33%) of the carers were able to talk to the patient’s named
nurse, although a further 9 (43%) said they did not need to talk to the
named nurse.
4
4.3.9 13 (62%) carers reported that ward staff answered any questions they
had to their satisfaction and 4 (19%) did not ask any questions.
4.3.10 9 (41%) of the carers reported that staff asked them questions about
the patient, leaving 13 (59%) who were not able to share their
information.
4.3.11 91% of carers felt safe on the ward
4.3.12 Only 45% were invited to attend a care planning meeting
4.3.13 Only 41% saw or were offered any information that might be relevant to
them eg visiting times, support services for carers
5.
Outcomes of the project
5.1
Reception areas Whilst many of the mystery shoppers’ experiences of the
Trust reception areas showed an improvement from the last exercise, there
are still some aspects of these service where further improvements can be
made. In particular this relates to the availability and provision of information
about local services. In a few isolated cases the mystery shoppers found the
reception/security staff noticeably unhelpful in responding to their enquiries,
and in 17 cases there was no receptionist even present to receive enquiries. It
is suggested that all reception staff are reminded that they are the front line of
the service and need to represent a positive image to all visitors and that they
take responsibility for ensuring that the supply of information leaflets in their
area is kept fully up to date. It would also be helpful as part of the induction
and supervision of reception staff that they are reminded where they can look
for additional information about services and to proactively supply this
information to visitors.
5.2
PALS Linkworkers This is the first time that PALS Linkworkers have been
reviewed using the mystery shopping methodology. The main issues that have
arisen also relate to the provision of information. However, more importantly a
concern arose over the availability of PALS Linkworkers to answer the
shoppers’ enquiries. PALS linkworkers function as a first port of call for
patients and carers who have concerns or might be considering making a
complaint, and undertake this role in addition to their main job role, but if they
are to be absent for whatever reason it is important that there is someone
available to deputise for them to ensure that this service is maintained. The
mystery shoppers were reasonably satisfied with the response of the PALS
workers but the results of this exercise will be used in the regular PALS
network meetings to identify areas for general service improvement.
5.3
Carers Despite the low number of carers who were recruited to this mystery
shopping programme it has proved to be a useful exercise. The main finding is
that visiting times are a missed opportunity for ward staff to share information
with carers and to engage with them as key elements of the care team. Carers
want to be engaged but are often not acknowledged for the support they
provide to patients, and they need to be supported in this role. Working with
carers on acute wards has been identified as a key objective in some hospital
units. The lessons from this must be shared across the Trust through the Trust
Acute Care Forum and Service Line management teams.
5
5.4
The mystery shoppers were keenly interested where the results of this
exercise would be discussed and they are aware that a report is to be
presented to the Board. They are really interested in what impact their work
will have on service improvements. The results of this mystery shopping
exercise will be shared with the services visited by forwarding a copy of this
Board report and the detailed results for their service. The services will be
encouraged to formulate an action plan to address any areas where services
can be improved and these will be evaluated through subsequent surveys and
mystery shopping ventures.
6.
Recommendations for future mystery shopping
6.1
It is recommended that the Trust continues to use mystery shopping as a
quality measurement tool. A key requirement from service users and carers is
for information to enable them to make informed choices and to access
various sources of support, and to be engaged in a dialogue with services.
These will occupy a central position of all new mystery shopping exercises.
Chris Bumstead
Head of Patient & Public Involvement
6