Report: Handheld Technologies in the Ward
A Human Factors Evaluation of eObs deployment and uptake in
Nottingham Universities Hospital Trust
February 2016
Human Factors Research Group, University of Nottingham
Feb 2016
Report Produced by
Dr Alexandra Lang, Dr James Pinchin, Dr Michael Brown and Professor Sarah Sharples
Human Factors Research Group & Horizon Digital Economy Research Institute
University of Nottingham
Nottingham
NG7 2RD, UK
Lead contact and for further information – alexandra.lang@nottingham.ac.uk
For
ICT Services
Nottingham Universities Hospitals NHS Trust
Queen’s Medical Centre & City Hospital
Nottingham, UK
2
Human Factors Research Group, University of Nottingham
Feb 2016
Contents
Executive Summary............................................................................................... 4
Human Factors Approach ....................................................................................... 6
Observations ..................................................................................................... 6
Interviews ......................................................................................................... 6
Observation Findings ............................................................................................. 7
Nurses .............................................................................................................. 7
Doctors ............................................................................................................. 8
........................................................................................................................ 9
Summary of Observation Findings ........................................................................ 9
Interview Findings ................................................................................................. 9
Common Findings Nurses and Doctors ................................................................ 10
Nurses ............................................................................................................ 11
Doctors ........................................................................................................... 12
Future Investigations ........................................................................................... 14
Concluding statements ......................................................................................... 15
Acknowledgements .............................................................................................. 15
Appendices ......................................................................................................... 16
3
Human Factors Research Group, University of Nottingham
Feb 2016
Executive Summary
This report summarises the findings of a Human Factors (HF) evaluation of the deployment
of new technology via handheld mobile devices in a secondary care setting. The evaluation
was commissioned by Nottingham Universities Hospital Trust (NUHT) and part funded by
the ‘NHS England Safer Hospitals, Safer Wards Technology Fund’.
The primary focus of the report is the rollout of the Electronic Observations (eObs) package
by Nervecentre Software Ltd in Queens Medical Centre and City Hospital Nottingham in
2015. This technology deployment involved a significant change in clinical practice. The
recording of patient observations and early warning scores (EWS) were previously manually
calculated and inputted onto a paper chart. The new technology involved a step change in
practice involving the inputting of observation data onto a handheld mobile device, with the
eObs software automatically calculating EWS and generating clinical alerts when necessary.
The aims of the evaluation were to:




Evaluate the impact of handheld technologies on activities within the wards and staff
practice.
Improve understanding of the way in which handheld technology is adopted by staff
in their clinical practice.
Report on staff satisfaction with change in technology and practice.
Provide guidance for future implementation of technology.
This evaluation employed two methods: observation of staff activity; and face to face
interviews with staff. Over eighty five hours of direct observations (n=89) and forty staff
involved in interviews and four focus groups in 19 wards were carried out to elicit data
about clinical practice and staff experience.
After the eObs package was introduced, observations demonstrated a reduction in the time
spent interacting with paper notes and talking on the phone. There was also a small increase
in the time spent using handheld mobile devices. This change was smaller than the
reductions in time recorded for other activities. Both medical and nursing staff groups spent
less time in office spaces and more time visible to other staff, patients and visitors. The time
spent with patients increased for both nurses and doctors, in the case of doctors that time
was doubled.
Interview data revealed a range of anticipated and unanticipated benefits associated with
the eObs rollout. Anticipated benefits included, but were not limited to: accessibility of
information and utilisation of the device as a tool for distributed working; improved
communication within teams; and facilitation of personal time management. There was
general praise for the Clinical ICT team which was set up to support the technology roll out.
Unanticipated benefits included: junior staff providing informal mentoring of eObs system
use; visibility of team capabilities; and improved communications between clinical staff and
patient relatives. There is opportunity if these benefits are communicated to the workforce
more widely for the deployment to improve working practices further.
4
Human Factors Research Group, University of Nottingham
Feb 2016
Challenges were identified in association with the introduction of the new technology, these
included but were not limited to, integration and infrastructure; the absence of feedback
mechanisms for staff use and information; management of staff expectations; and training
requirements. These requirements provide commentary for organisational learning with
respect to future technology deployments.
5
Human Factors Research Group, University of Nottingham
Feb 2016
Human Factors Approach
Two methods were used to elicit data on the working practices, behaviour and experiences
of staff. Over 85 hours of observation were recorded over 89 observation sessions and 18
hours 30 minutes of interview data with 40 participants, involving representatives from both
medical and nursing disciplines.
Observations
Direct structured observations of clinical staff were carried out before (pre) and after (post)
the deployment of handheld technologies to record staff activities, tasks and location within
the ward. This method was selected to quantify changes in the distribution of activities for
each staff group.
The number of observations sessions (n) was 24 Pre-deployment and 65 post-deployment.
Observation sessions lasted between 15 minutes and 2 hours. Observers ‘shadowed’ staff,
using a bespoke application running on a tablet computer to record activities and locations
from exhaustive lists (see Appendix 1).1 Data were collected by researchers and
postgraduate researchers at the University of Nottingham and by clinical staff managing the
technology rollout.
Pre-deployment data were collected on ward B3 QMC
(n=11), a short stay admissions ward, in November 2014 and
on D58 (n=13) a Health Care of Older Person ward, in
October 2015.
Post-deployment data was collected on a wider range of
wards in April, May and June 2015. Of these 65 ward based
observation sessions, the distribution was across the
following disciplines 49% (n=37) Acute Medical Admissions,
17% (n=12) Medical, 29% (n=15) Surgical).
Activities observed on the
ward included,
- Time spent interacting with
technology on the ward.
- Time spent interacting with
other artefacts in the ward.
- Time spent interacting with
other people, either directly
face to face or remotely via
technology.
Registered Nurses were observed for 17.0 hours pre-deployment (n=16) and 23.3 hrs postdeployment (n=18). Doctors (from F1 to consultant) were observed for 10.0 hours predeployment (n=7) and 35.1 hours post-deployment (n=47).
Interviews
Semi-structured interviews and focus groups were undertaken to elicit staff experiences of
the eObs system deployment and feedback about their use of the software and handheld
technologies in clinical practice. Forty participants were recruited across a range of nursing
and medical job roles and seniority to take part in the interviews. All interviews were carried
out post- system deployment, with staff experience of the system ranging from 1 week to 5
months.
1
Observation sessions were divided into 30 second time bins. If an activity was observed in a 30 second bin it
was recorded as one observation even if multiple instances of the activity occurred. This method makes the
observation of multitasking or rapid task switching possible and provides a measure of the relative distribution
of different activities during the observation period.
6
Human Factors Research Group, University of Nottingham
Feb 2016
Interviews durations ranged from 12 to 65 minutes with a total of 18.5 hours of staff
interview data. The interview schedule was developed to encourage interviewees to
consider the impact of the technology deployment on their own personal working practices
and job satisfaction and also reflecting on the wider impact on teams, environments and
organisation.
Observation Findings
Nurses
The change in observed activity frequencies pre and post-deployment is shown in figure 1.
It was expected that there would be an observed increase in smartphone use postdeployment and that is reflected in the observation data. The case example presents an
illustration of how the deployment and use of eObs and handheld mobile devices has
changed clinical nursing practice. This example uses a one hour exemplar and assumes that
each observed activity spanned the entire 30 second observation bin.
The ‘Using Smartphone’ activity was observed in only
2.2% of the pre-deployment data, increasing to 6.4%
in the post-deployment dataset.
This change is small however when compared to the
decrease in observations of activities which we would
expect to move totally or partially to the smartphone.
There were observed reductions in ‘Using PC/COW’
(23.3% to 5.1%), reduction in ‘Looking at Notes’
(36.2% to 22.3%) and reduction in ‘Writing on Notes’
(26.3% to 16.0%). Observations of ‘Talking on Phone’
more than halved from 8.4% of 30 second bins to 4%.
An increase in time spent at the nurses’ station and
corresponding decrease in time spent in the office was
observed. Observations of ‘In ward – Office’ decreased
(40.8% to 16.2%) while more time was spent at the
nurse’s station (13.3% to 35.1%).
Nurse example: average change in
nursing practice over one hour due to
eObs implementation;
TASK
- Increase in average smartphone use
from just over 1 minute to nearly 4
minutes.
- Use of PC/COW would reduce from
14 minutes down to 3 minutes.
- ‘Looking at notes’ would reduce from
over 21 minutes down to less than 14
minutes.
- ‘Writing in notes’ would reduce from
nearly 16 minutes down to under 10
minutes
- ‘Talking on phone’ would reduce by
more than half from 5 minutes down
to just over 2 minutes.
- Searching tasks would reduce by 3.5
minutes.
LOCATION
- Time spent in office would reduce
from over 24 minutes to less than 10
minutes.
- Time spent at the nursing station
would increase from 8 to 21 minutes.
Observations of ‘searching’ also decreased, from
12.5% of bins to 6.7% of bins, relating to a drop of
approximately 3.5 minutes in an hour. The searching
category can be decomposed into searching for
‘place’, ‘equipment’, ‘staff member’, ‘patient’ and ‘paperwork’, all of which saw decreases.
Searching for staff member dropped from 2.3% to 0.5%, searching for patient dropped from
1.6% to 0.4% and searching for paperwork dropped from 4.9% to 3.1% of bins.
7
Human Factors Research Group, University of Nottingham
Feb 2016
Figure 1: A comparison of observed activity frequency for nurses before (pre) and after (post) the deployment of
the eObs package / handheld technologies. Error bars indicate the 95% bootstraped confidence interval.
A decease in the number of activities observed in each 30s bin was observed. This decrease
is closely related to a decrease in rapid task switching. The mean number of activities in
each 30 second bin decreased from 1.99±0.04 to 1.66±0.03.
Doctors
The observed changes in the way doctors spend their time were very similar to those
observed for nurses. The full results are given in figure 2. The case example presents an
illustration of how the deployment and use of eObs and handheld mobile devices has
changed clinical practice for doctors. This example uses a one hour exemplar and assumes
that each observed activity spanned the entire 30 second observation bin.
Smartphone use increased (3.7% to 8.3%) while
remaining low relative to the frequency with
which interacting with paper notes or desktop PCs
was observed.
Doctors were also observed spending less time in
the office (68.7% to 25.6%). More time was spent
at the nurses’ station (6.6% to 41.7%). Patient
contact time more than doubled (2.9% to 7.3%).
8
Doctor example: change in medical
practice over one hour due to eObs
implementation;
TASK
- Increase in smartphone use from 2
minutes to 5 minutes in an hour of clinical
practice.
LOCATION
- Time spent in office would reduce from
over 40 minutes down to less than 16
minutes.
- Time spent at the nursing station would
increase to 21 minutes.
- Increase in time spent with patient, from
less than 2 minutes to over 4 minutes in an
hour of care.
Human Factors Research Group, University of Nottingham
Feb 2016
Figure 2 comparison of observed activity frequency for doctors before and after the deployment of the eObs
package / handheld technologies. Error bars indicate the 95% bootstrapped confidence interval.
A decrease in the number of activities observed in each 30 second bin, related to rapid task
switching, was also observed for doctors. The mean number of activities in each bin
decreased from 1.86±0.05 to 1.45±0.02.
Summary of Observation Findings
The direct observation data confirm that while more staff time is spent interacting with
handheld devices (smartphones) this is outweighed by the decrease in interactions with
paper notes, desktop PCs and time spent talking on the phone.
Both nursing and medical staff were spending more time away from the office and more
time at the nurses’ station. This made them more visible to other staff, patients and visitors
and perhaps goes some way to explaining the observed reduction in time spent searching
for other staff members.
Both staff categories were observed spending more time with patients, in the case of
doctors this time was doubled. Since the nature of the work undertaken by a ‘doctor’ varies
largely with grade, more observations are required to ensure that this effect is not an
artefact of the doctor grades observed.
Interview Findings
Forty staff provided subjective feedback about their experiences of the eObs and Handheld
devices deployment process. This provided detailed accounts of staff experience in relation
to the deployment and also technology uptake in the months following.
9
Human Factors Research Group, University of Nottingham
Feb 2016
Common Findings Nurses and Doctors
The application of a specialist clinical team who were trained as ambassadors and ICT
technicians to lead the deployment was universally lauded as the reason for a successful
technology roll out. This Clinical ICT team was praised for their skills and capacity to assist
people whilst administering clinical care.
During the deployment and early use of the technology it
was reported by staff that the intervention had increased
stress and workload. It became evident that staff
expectations of the technology deployment were not well
managed and that identification of training requirements
and provision were considered a weakness of the
deployment. Where heightened stress levels persisted
after the involvement of the Clinical ICT team it appeared
to be due to the lack of support from ICT help services and
staff requirement for a sustainable feedback procedure
between the general workforce and ICT services.
“Not all of us had physically got
our phone in time so it were all
faffing, trying to get the phones
charged up and all that kind of
technical stuff…we’d not really
had time to play with them…I
think every one of us felt nervous
the morning of it coming and I
don’t think we needed to” –
Senior Nurse
The accessibility of information provided by eObs on the handheld mobile devices appears
to have improved communications within teams, whereby clinical staff do not have to
report and repeat observation data but can fast track discussions to address treatment
pathways and actions. The availability of eObs on the handheld mobile devices have
facilitated remote decision making and distributed working for both nursing and medical
staff. This was reported in relation to benefits individually and also in regards to team
working.
There was frustration expressed by both nursing staff and medical staff at the lack of
engagement with the new system by senior medical personnel (specifically consultants). It
was considered that this issue was one of the main barriers to realising the potential
benefits in a ward setting. It was also reported that when communication problems arose
around the use of the new system, these were often due to the lack of buy-in and
understanding of the system by senior medical consultants.
Junior personnel (medics and nurses) provided an
important source of informal device use support to
individuals who were struggling to implement the software
and hardware in their working practices. This support was
provided in the immediate situation of the deployment
and during the weeks and months following the departure
of the Clinical ICT from the ward environments.
“If you break it [eObs or phone],
even now, it’s a standard joke if
one of us breaks one of the
phones, we get one of the young
staff to fix it” – Senior nurse
‘Word of mouth’ or ‘heard it through the grapevine’ communications often perpetuated
information about eObs and device use throughout the workforce. Staff believed that if
good practice use of eObs and devices could be formally captured and disseminated it could
speed up the rate at which staff experienced benefits from the new system.
10
Human Factors Research Group, University of Nottingham
Feb 2016
There was also concern expressed by both staff groups about the inflexibility of the system
and the potential for unnecessary observations being carried out on patients. Staff accounts
reflected on the potential conflict of interest between hospital policy and the
implementation of eObs alongside patient quality of life.
Nurses
Interview data revealed a largely positive response from nursing staff, reporting added value
in the form of reassurance of patient health state from the eObs software, due to the realtime accessibility of information. However there was also an initial perception that the new
technology resulted in a loss of control for the nursing staff and promoted a ‘Big Brother’
culture.
It was evident that during and after the deployment there was a period of ‘paper
persistence’ whereby nursing staff would write down notes and observations on paper, with
the aim of entering data into eObs all in one go. Staff reports indicated that this issue
subsided as experience and use of the system increased. Additionally following the initial
peak in workload during and after ‘go live’ periods, the
process of familiarisation and use enabled nursing staff to “It is just about making the
identify the potential for reduction in stress, particularly at device work for you and I
suppose, as I’ve got more
busy times when information would be more readily confident with the device, I have
available to them. After a few months of use, nurses began said to my staff, you know, just
to describe the mobile devices as their own ‘personal tool’ don’t let it rule your shift and you
for management of workload and improving situation get it to work for yourself” –
Deputy Sister
awareness of team capacity.
Nursing staff also described the value of handheld devices as a communication tool for use
with patients and specifically relatives, whereby the request for information could often be
responded to more quickly without having to search out a paper chart or colleague.
Concerns were raised in regard to cognitive workload, with nurses reporting alarm fatigue
and information overload. Whilst this effect reduced following the ‘go live’ period there
was a persistent concern and reports of ‘accepting bleeps’ to turn it off.
Another challenge identified is that of clinical skills learning; senior nurses were concerned
about individual staff ability to recognise a deteriorating patient through clinical assessment
outside of the eObs system. They felt that staff understanding of the Early Warning Score
(EWS) parameters could be detrimentally impacted due to
the automatic nature of the eObs system. Additional “You need to have people who
are allowed to nurse and do the
concerns were raised about how to empower nurses to doctor bit independent of the
challenge the eObs system on occasions when their clinical phone and I think that’s what
expertise and judgement was suggesting that the EWS we’re losing also for those senior
score is not representative of the patient health state. The nursing staff who are very good
reliance on the system was also discussed in relation to the they will come to me and say I
have to do this because the
resilience of the organisation and concerns about failure in phone says I have to do this” technology infrastructure.
Consultant
11
Human Factors Research Group, University of Nottingham
Feb 2016
The nurses raised concerns about resources, specifically the availability of personal devices
to ‘bank nurses’ and student nurses. In some wards it was perceived that the shortage of
devices and long delays in providing devices to individuals meant that their use of the
system was not as efficient as it could be. Senior nursing staff also reported increased
workload associated with the task of providing devices to temporary staff i.e. when a ward
is heavily reliant on agency nurses.
Doctors
Lack of engagement in the eObs project by senior medical staff sometimes provided a
barrier to team communications associated with the deployment and use of the system.
This was especially true where the relationship between consultants and registrars was put
under pressure due to the permissions and access programmed within the system initially.
There were several rationales offered by medical staff of all grades (including consultants) to
explain the lack of engagement by senior staff, including the perceived loss of expertise due
to the ‘step change’ in practice, the potential for embarrassment associated with naivety of
the new system or reluctance to embrace change. It was also suggested that the baton
device system and issue of personnel having to float between wards were practical barriers
to their use of a mobile device. There was evidence of senior medics relying on registrars
and junior doctors and ‘borrowing’ phones in situ.
Permissions and restricted access within the eObs system
were at odds with current practice in terms of the
perception about ‘consultant led practice’ versus the
reality of registrars working independently and accessing
consultant advice as required. Clinicians understood the
need for policy to underpin the system, however there was
disruption to working practices as these issues were
experienced.
“…who delivers the cardiac arrest
process and decides, well it’s the
registrar. …so if you are allowing
them to make those decisions
then to say they can’t alter the
parameters I think is patronising.
And it’s again where the trust
says oh no all our decisions are
consultant made but the reality is
that’s not true” - Consultant
The uptake of eHandover in the wards was varied. Frustrations were expressed that the
software offered electronic handover but that there was little guidance associated with its
use. It was perceived that this variability led to duplication of tasks and inconsistency
between different working environments. This issue was beginning to lesson as interviewees
experienced use of the device over a longer period of time however it was felt that the
change of practice in relation to this would have benefitted from better management.
Medical staff also disclosed their use of the device to ‘check
up’ on patients that they had treated. This was described as
checking for clinical and personal reassurance. This use of
the device whilst providing support to the clinicians needs
to be monitored to ensure that the device does not
promote unhealthy practices with regard to work-life
balance.
12
“I think it is mainly when I see
somebody sick in the ward when
on acute medicine or something
and I just sneak a peek to make
sure they are getting better
instead of worse... that is
probably my main sort of
benefit.” - Registrar
Human Factors Research Group, University of Nottingham
Feb 2016
There was a perception from medical staff that the mobile devices and eObs system had
provided benefit in regard to reduced time for searching for people and information and
task and time management. This was related not only to their working practices but also in
their reflection of nursing practices and where the system might benefit both the activities
of healthcare professionals more generally.
13
Human Factors Research Group, University of Nottingham
Feb 2016
Future Investigations
This human factors evaluation has consulted with and observed NUHT medical staff. It has
provided insight into the impact on working practice and elicited experiential data from the
staff regarding their use of eObs and the handheld devices. From these data sets a range of
benefits to NUHT and the work force have been discovered. It has also unearthed areas
where additional research could further benefit staff and the patient experience. These
include but are not limited to the following recommendations:





Study the impact of further appropriation and expansion of technology on hospital
working practices.
Investigate methods of disseminating formal and informal user information about
eObs and use of technology for improved practice (with consideration for
‘information overload’).
Explore the roles of senior and junior medical staff to understand cultural impact
when there is a change in practice or technology intervention.
Analyse and measure the impact of improved situation awareness and system status
e.g. visibility of ward capacity for both team and individuals.
Investigate further how mobile devices are being used on a personal level and where
different clinical roles find utility in the technology.
The following bullet points provide recommendation for short term actions which could be
applied to alleviate some of the current issues and concerns associated with eObs and
mobile device use on the wards. These are practical suggestions which could be actioned in
response to the findings of the human factors evaluation.




Consider of a sustainable PR approach to inform patients about change in practice
and technology innovation in NUHT.
Consider the training requirements associated with eObs and mobile devices in
clinical practice to promote confidence amongst workforce that clinical judgement
and expertise is not being lost through the utilisation of the new system.
Capture requirements from workforce for a ‘fit for purpose’ feedback system
between healthcare professionals and the NUHT ICT service.
Investigate the current situation with regard to ‘alert fatigue’ for different staff
groups.
There were also calls from staff for:
- Improved information about hygiene policy associated with the devices;
- Communication about the long term plans for phone maintenance and
replacement;
- Clarity about phone use outside of clinical purposes ;
- Mechanisms to ensure competency of use by transient staff, agency nurses and
locum doctors;
- Ability to select speciality of medical consultant who needs to be informed about
escalation (specifically requested within the children’s hospital).
14
Human Factors Research Group, University of Nottingham
Feb 2016
Concluding statements

Mobile tools to support clinical observation have the potential to be beneficial for
doctors and nurses.

Deployment of this technology takes time must involve working directly with users
and must be supported by a specialist technology deployment team.

More junior staff adapt to the technology particularly well.

Clinicians find ways of using this technology in conjunction with other tools to
manage their work.

Embedded algorithms must take account of different clinical specialities.

The technology can support clinical and patient communications.

It is vital that there is integration of new IT systems with existing systems.

The technology is only as good as the infrastructure that supports it.
Acknowledgements
This body of work is thanks to the collaborative efforts of Nottingham Universities Hospital
Trust Clinical ICT and the eObs project team, with special acknowledgments to Caron
Swinscoe, Mark Simmonds, Sue Clarke and Lorrayne Dunn. The authors acknowledge the
support of Health & Social Care Information Centre (HSCIC) in supporting the contractual
obligations and realisation of benefits associated with the ‘NHS England Safer Hospitals,
Safer Wards Technology Fund’ award (Ref: SHSW-0087). The authors of this report are also
supported by the Horizon Digital Economy Hub EP/G065802/1.
15
Human Factors Research Group, University of Nottingham
Feb 2016
Appendices
Appendix 1. Structured Observation Tablet Application (screenshot illustrates scenario
where three activities – talking face-to-face, talking on smartphone and using smartphone
are all noted within a single 30 second period or ‘bin’)
16