An evaluation of the Paediatric
Observation Priority Score in
predicting appropriate admission and
safe discharge from a Children's
Emergency Department
1 Project summary
Six million children use Urgent Care Services every year. National scoring systems such as the
Modified Early Warning Score (MEWS) exist to recognise sick and deteriorating adults. No validated
tool exists for children in emergency departments. The Confidential Enquiry into Maternal and
Childhood Health (CEMACH) report “Why Children Die” indicated death could be prevented if
doctors were better at recognising ill children, with the National Patient Safety Agency (NPSA) also
reporting similar findings. Both identified illness scoring systems as a key area of development. The
pilot of a locally developed “Paediatric Observation Priority Score” (POPS) showed it allows
healthcare professionals of any level of experience to assess the acuity of children. POPS was popular,
easy to use, and discriminated between children of low and high acuity and is now the standard
assessment tool in the Leicester Royal Infirmary children's Emergency Department. This study will
determine appropriate cut off points for the POPS score to predict appropriate admission and safe
discharge.
Once this work has been performed we hypothesise POPS will enable a quantification of the level of
resources a particular child requires as those being admitted are by definition more ill than those being
discharged. This would susbsequently allow decisions on the number of health care professional staff
and their skill mix requirements to be made based on the intensity of acuity of all children in a
department. This will ultimately be used to facilitate resource management and would be the basis for
a future study.
2 Project description:
2.1 Rationale
Illness scoring systems such as the Modified Early Warning Score (MEWS) exist to recognise sick
and deteriorating adults. These translate commonly recorded physiological parameters (e.g. heart rate)
into an integer (whole number) score. On reaching a predefined value the score would trigger a
response such as a medical review or admission to a high dependency area. Such systems exist in
children but have not been validated in a Paediatric Emergency Department to help aid decision
making processes.
The pilot of a locally developed “Paediatric Observation Priority Score” (POPS) showed it allows
healthcare professionals of any level of experience to assess the acuity of children. The POPS score,
defined in more detail below, is a score from 0-16. This research project is to define the which scores
in the POPS system best predict appropriate admission and safe discharge
Dr. Damian Roland Version 1 160911 |11/EM/0351
1
2.2 Objectives
2.2.1
Primary
What are appropriate cut off points on the POPS score to determine appropriate admisison to the
childrens hospital and safe discharge.
2.2.2
Secondary
Does the POPS system predict resource use (that is the time of staff or the use of medical
interventions) in the Emergency Department?
Whether POPS is predictive of the presence of a serious bacterial illness?
Does POPS predict which children require admission for greater than 24 hours?
2.3 Methodology
This study involves creating a predictive model to link POPS to the eventual decision on admission or
discharge for the patient.
The following definitions are used:
POPS - POPS is the Paediatric Observation Priority Score. It is a checklist of variables which include
heart rate, respiratory rate and temperature and also some comments on the appearance of the child
(such as work of breathing and level of alertness). Each of the variables has a score between 0-2
assigned to it (i.e a normal heart rate for the child’s age would score 0, a very high rate would score
2). There are 8 variables which are considered leading to a score between 0-16.
The POPS system is employed when a child initially presents to the Children’s Emergency
Department of the Leicester Royal Infirmary. The POPS system has been awarded a Da Vinci Health
Care Technology award.
Appropriate Admission - If a child is admitted to the Children’s Assessment Unit they are not
discharged within 4 hours of arrival. The Children's Assesment Unit at the Leicester Royal Infirmary
is a specialist unit run by paediatricians which also takes referrals direct from GPs and Urgent Care
centres throughout Leicester City.
Safe Discharge - If a child is discharged they do not return to the Emergency Department and be
subsequently admitted within 7 days. This duration is has been chosen as it has recently been accepted
by the College of Emergency Medicine as a quality indicator for Emergency Departments nationally.
It is currently not possible to determine if the child represents to another health care provider however
admission back to the Leicester Royal Infirmary Children's Assessment Unit following discharge
from the ED can be determined.
On a prospective basis the physiological observations physiological (e.g. Heart Rate), observational
features (e.g. such as appears unwell to a health care professional) and other relevant non-identifiable
patient data of the those children presenting to the Leicester Royal Infirmary Emergency Department
will be recorded in an electronic form.
The following system has been agreed by the NIGB as not requiring consent from patients. The POPS
system is currently in use in the Emergency Department - no information is collected which is not
currently occurring.
Dr. Damian Roland Version 1 160911 |11/EM/0351
2
1) A patient books into the Emergency Department at the Leicester Royal Infirmary.
2) They have an assessment within 15 minutes of arriving in the department (current departmental
standard) which includes a POPS score. This is the current standard of care. This occurs via the triage
or assessment nurse on duty.
3) This score is put on an electronic proforma and data relevant to the research (which does not
include the patients name or any demographic information) and stored in secure encyrpted computer
database. The electronic proforma is printed off as per current practice and this is used in the patients
normal clinical care. The serial number of the patient is coded and scrambled and then linked to the
information in the encrypted database. It can then be used to match the outcomes required via the
Emergency Department Information System. This process occurs automatically.
4) Data analysis which will not include identifiable data will be analysed by the principal Investigator
(who is not part of the patients care team).
To determine the primary outcome measure the overall POPS score will be plotted against the
outcomes of appropriate admission and safe discharge and relevant cut off points selected. Secondary
outcomes will also be calculated using the following criteria:
1) Whether POPS is predictive of the presence of a serious bacterial illness as defined by a growth of
an organism in the urine, blood stream or cerebral spinal fluid (fluid around the brain)?
2) Whether there is any relationship between overall POPS and the total resource utilisation based on
the following each having a unit score of 1?
- require an early medical intervention (defined as immediate cannula insertion, intravenous antibiotic
prescribing, immediate delivery of an inhaled or nebulised medication, request for a portable CXR,
transfer to the resuscitation area))
- require senior clinician involvement
- to be admitted to Children’s Intensive Care
- to have longer stay in the ED (over 3 hours)
3) Whether POPS predicts which children require admission for greater than 24 hours?
2.4 − Data management and analysis
Data will be placed on an electronic database held on an NHS password protected computer in locked
office and will be analysed on site by the prinicipal investigator. The ethics committee will be
informed if either the dataset is no longer required for any future meta-analysis or the prinicipal
investigator changes his location of work.
The Emergency Department sees over 30000 patients per year of which at least 10000 have a medical
problem for which they would be assessed with a POPS score. A sample size of 5000 patients
(discussed with a medical statistician) would make this the largest series of its kind and enable model
development and external validation. Because of technical reasons (computer shut downs and
downtime) and use of the electronic recording by staff it is not feasible to obtain data on all 10000
potential subjects although it is feasible at least 80% of attendees will have data sets recorded.
In regard to determining appropriate admission:
Dr. Damian Roland Version 1 160911 |11/EM/0351
3
The feasibility study of POPS (942 patients) has demonstrated a 36% admission rate from those with
assigned POPS scores. This is a much higher than average admission rate although for this work those
with POPS scores were a self selecting high risk group. For this study the admission rate is likely to
be around 10%.
In regard to determining safe discharge:
In a population of 5000 a 3% re-attendance rate would detect the sensitivity of a POPS score greater
than zero to within a precision of 8% either way with reference to the 95% Confidence Interval for
this estimate.
This sample size is would also make it the largest published series of its kind.
ROC curves will be used to examine the effect that different choices for the cut point would have on
the sensitivity and specificity to predict appropriate admission and safe discharge (admission for
greater than fours hours to the Children's Assessment Unit and discharge without return within 7
days) as outcomes. The final decision will be based on appropriate best practice in paediatric
emergency care. This work will be performed with the assistance of a medical statistician.
• Ethical considerations
Data will be collected prospectively by inputting patient de-identified information (as above) with
resource utilisation and outcome into an NHS password protected computer located in a locked office.
This data will be used to create prediction models to map the POPS score against the selected proxies
for resource allocation. Data collection is in keeping with the Caldicott Principles and has been
confirmed by the Ethics and Confidentiality Committee of the NIGB that section 251 support is not
required. No new data, technique or intervention is being utilised as part of this study so specific
patient consent is not being requested.
During the study period all children who present to the Leicester Royal Infirmary Emergency
Department will have a POPS recorded and stored electronically (i.e the data fields are inputted
directly into a computer). This recording system does not record patient identifiable data but is able to
also record the eventual outcome (disposition) of the patient according the admission and discharge
definitions above.
The POPS system is utilized by staff at the Leicester Royal Infirmary who all have training in the
recording of observations in children are currently routinely collected as part of standard care within
Emergency Departments. This process is no different to current practice at the Leicester Royal
Infirmary Emergency Department except from the electronic recording of data. No data is being
collected that would not already be obtained from patients.
Dr. Damian Roland Version 1 160911 |11/EM/0351
4