Outcome measures for acute general paediatrics
Consultation on proposed shortlist
The RCPCH has launched a project to define 5-10 universally applicable and measureable outcome
measures for acute paediatric services that can be used to indicate changes in the quality of care
over time, which may result from service change. It is intended that these will be relevant to clinicians,
children and young people, their parents and carers, and health care commissioners and service
planners, and be applicable across the four UK nations.
In autumn 2014, we conducted an initial call for evidence and a literature review. We collated these to
produce a long list of 271 measures, covering 5 domains in line with recommendations from the
Children and Young People’s Outcomes Forum (CYPHOF). Although the work of the Forum relates to
England only, we believe the principles apply across all four UK nations.
This long list was shortened to 19 measures by an expert reference group.
We are now seeking your views on the shortlist. We invite you to respond by providing comment
using the form below. Please return this to outcomes@rcpch.ac.uk by Monday 29th February 2016. If
you have any questions please contact Rachel Winch on 0207 092 6156, or email
outcomes@rcpch.ac.uk.
Why are we doing this?
The RCPCH Facing the Future: Standards for Acute General Paediatric Services aimed to build
consensus on service standards for acute paediatrics. The evidence base upon which to develop
such standards is limited so the standards were founded upon expert opinion. Currently, robust
evidence of good outcomes resulting from service change is lacking. In 2012 the CYPHOF, in relation
to health services in England, recommended that “Colleges and specialist societies develop robust
and evidence based outcome measures and indicators for the key conditions within their remit, and
that these are transparently reported by service provider organisations for the information of children,
young people and their families, as well as for commissioners and regulators.” 1
One of the recommendations in the Chief Medical Officer’s 2012 annual report, “Our Children
Deserve Better” was that the CYPHOF annual summit should provide an opportunity to review health
outcomes that are relevant to children, and to examine regional variation 2. The CYPHOF annual
report 2014-2015 reviews progress on improving child health outcomes, highlighting key challenges
and areas for the development of indicators, and the publication of transparent, regularly updated
surveillance of trends, outcomes and variation in performance3.
The NHS Outcomes Framework for England contains a set of high level, overarching outcome
indicators and improvement areas. In the 2015/16 Framework, 6 out of 30 improvement areas which
sit under the overarching indicators are dedicated to infants, children and/or young people.
Children and young people’s health outcomes strategy. Report of the children and young people’s
health outcomes forum. Department of Health. 2012. Available at:
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/216852/CYP-report.pdf
2 Davies, S. Chief Medical Officer's annual report 2012: Our Children Deserve Better: Prevention
Pays. Department of Health. 2013. Available at: https://www.gov.uk/government/publications/chiefmedical-officers-annual-report-2012-our-children-deserve-better-prevention-pays
3 Cheung, R, Duncan, H. and Viner, R. Report of the children and young people’s health outcomes
forum 2014/15 – Health Outcomes Theme Group.
1
RCPCH outcome measures consultation
September 2015
A recent NIHR report into drivers for reconfiguration and the underpinning evidence base concluded
that there was strong evidence that some service reconfiguration can significantly improve outcomes,
but that there were evidence gaps, including around the interplay between staff numbers, skill mix and
outcomes4.
There is a clear lack of outcome measures specifically for acute general paediatrics. The RCPCH
aims to provide tools for child health services to monitor the effects of service interventions in the
future. The first step is to develop a set of outcome measures.
What do we mean by “outcome”?
Outcome measures address a service or clinical intervention to see if a desired effect has been
achieved.
Process measures address whether a particular intervention or activity has been completed.
Adherence with process measures is typically assessed by clinical audit or assessment of adherence
to service standards.
Experience outcomes address a particular aspect of patient, parent or staff experience to see if the
desired experience has been achieved.
Scope and domains
The scope of this project is acute general paediatrics in the UK, including infants, children and young
people up to the age of 18. It does not include children and young people seen as outpatients or by
specialist services, including neonatology. This scope aligns with the Facing the Future: standards for
acute general paediatric services5, and with the Facing the Future: Together for Child Health
standards6.
To ensure that the chosen outcome measures are representative of all aspects of acute paediatric
care, we have considered measures falling under the following domains, in line with the
recommendations of the CYPHOF:
1.
2.
3.
4.
5.
Overall acute healthcare utilisation
Acute healthcare utilisation for long term conditions, including mental health conditions
Safety
Process markers of care quality proven to have impact of outcomes
Measures of acute healthcare access & experience
Some of the measures included in the shortlist are already being recorded, for example through
Health and Social Care Information Centre Hospital Episode Statistics in England, and equivalent
organisations in Northern Ireland, Scotland and Wales. Availability of data will be assessed through
the consultation and subsequent feasibility assessment.
What isn’t included?
There were many outcome measures highlighted during the evidence gathering stage that the expert
reference group felt were important, but it is not possible to collect data on these at the moment. We
4
National Institute for Health Research. Insights from the clinical assurance of service reconfiguration
in the NHS: the drivers of reconfiguration and the evidence that underpins it – a mixed-methods study.
Health Services and Delivery Research. 2015; 3(9). Available at:
http://www.journalslibrary.nihr.ac.uk/hsdr/volume-3/issue-9#abstract
5
RCPCH (2014) Facing the Future. Available at: http://www.rcpch.ac.uk/facingthefuture
RCPCH (2015) Facing the Future Together for Child Health. Available at: http://www.rcpch.ac.uk/improving-childhealth/better-nhs-children/service-standards-and-planning/general-paediatrics/faci-0
6
RCPCH outcome measures consultation
September 2015
aim to highlight these measures in our final report as aspirational and for future consideration. We
have limited our consideration of process measures to those where a link to an outcome has been
established (e.g. timely antibiotic administration in children with sepsis).
We also did not include mortality as a domain for consideration. This is because our aim is to provide
a set of measures that can be recorded on an individual service basis to aid local monitoring and
service improvement. Child deaths are rare events, and would not occur frequently enough to provide
sufficient statistical power to detect changes in quality of care. We have, however, included measures
that relate to adverse events and deterioration of the sick child. Assessment of the statistical power of
the proposed measures will form part of the feasibility stage.
The development of outcomes for acute general paediatrics sits alongside other work being carried
out to evaluate child mortality and inequalities, including the joint RCPCH and NCB Why Children
Die? report7, launched in May 2014. More information about Why Children Die? can be found here:
http://www.rcpch.ac.uk/child-mortality
Next steps
The shortlist will be reviewed in the light of the consultation. We will work with research partners to
carry out a feasibility assessment, develop data acquisition methodologies, and pilot with child health
units.
The feasibility assessment will include consideration of the following:







Statistical power
Acceptability and relevance
Case-mix adjustment protocol
Identification and management of outliers
Availability of required data
Usefulness and limitations of existing data sources
Time needed to collect and analyse data, and who will carry out this work.
7
Wolfe, I. et. Al. (2014) Why children die: death in infants, children and young people in the UK, Part A. Royal College of
Paediatrics and Child Health and National Children’s Bureau. Available at:
http://www.rcpch.ac.uk/sites/default/files/page/Death%20in%20infants,%20children%20and%20young%20people%20in%20the
%20UK.pdf
RCPCH outcome measures consultation
September 2015
1. Your details:
Name
Click here to enter text.
Job title
Click here to enter text.
Organisation
Click here to enter text.
Email
Click here to enter text.
Please review Table 1 and provide your responses in Table 2.
RCPCH outcome measures consultation
September 2015
Table 1: Shortlist and evidence table
Domain
Overall acute healthcare utilisation
Measure
Administration of intravenous antibiotics within 1 hour of presentation to acute paediatric services of infants, children and young people with
suspected sepsis.
Early antibiotic administration in adults presenting with sepsis has been linked to lower mortality. Fewer studies are available in children, but suggest
similar benefits with earlier antibiotic administration.
Reason for
consideration
Additional
information
Domain
Measure
Reason for
consideration
Additional
information
Population
Episodes of infants,
children and young
people aged 0-17
admitted with suspected
sepsis.
Type of outcome
Process measure related
to clinical outcome
Potential data source
England – CQUIN reporting?
Associated standards or guidelines
Indicator 2, 2015/16 National
Commissioning for Quality and Innovation
Goals, NHS England.
https://www.england.nhs.uk/wpcontent/uploads/2015/03/9-cquin-guid2015-16.pdf
Meningitis (bacterial) and meningococcal
septicaemia in under 16s: recognition,
diagnosis and management. NICE
guidelines CG102.
Acute healthcare utilisation for long term conditions
Management of acute exacerbations of asthma in infants, children and young people according to British Thoracic Society/Scottish Intercollegiate
Guidelines Network Guidelines. For example:

Correct recognition of the severity of presentation (mild, moderate or severe) in children presenting with acute exacerbations of asthma

In children >5 presenting with severe and/or life threatening acute exacerbation of asthma receive oral or intravenous steroids within 1 hour
of presentation.

In children presenting with severe and/or life threatening asthma regular administration of inhaled/nebulised beta2-agonist, with assessment
of response

Children unresponsive to initial beta2-agonist treatment receive repeated doses of ipratropium bromide

Children with SpO2 <94% should receive high flow oxygen at sufficient flow rates to achieve normal saturations of 94-98%
Asthma is a common presentation for children to the emergency department. In the UK, guidelines for the management of acute asthma were
produced by BTS/SIGN in 2014. The 2014 Confidential Enquiry report “Why Asthma Still Kills – The National Review of Asthma Deaths” identified a
number of factors contributing to potentially avoidable asthma deaths. One such factor was the failure to recognise the level of severity of asthma
exacerbation at the time of acute presentation.
Population
Type of outcome
Potential data source
Associated standards or guidelines
Presentations to
Process measure related
New data source
BTS/SIGN guidelines for the management of
emergency department
to clinical outcome
acute asthma.
with acute exacerbation of
National Review of Asthma Deaths:
asthma aged 2-17.
https://www.rcplondon.ac.uk/projects/nationalreview-asthma-deaths.
RCPCH outcome measures consultation
September 2015
Asthma. NICE Quality Standard QS25
Measure
Reason for
consideration
Adherence to the Advanced Paediatric Life Support (APLS) guidance (or personalised Epilepsy care plan if in place) for the treatment of infants,
children and young people presenting in or developing status epilepticus whilst an inpatient.
Status epilepticus is a common presentation to acute paediatric services, with established guidelines for treatment.
Population
Episodes of children and
young people aged 0-17
presenting with or
developing status
epilepticus.
Type of outcome
Process measure related
to clinical outcome
Potential data source
New data source
Associated standards or guidelines
Advanced Paediatric Life Support (APLS)
guidance.
Royal College of Emergency Medicine,
National Clinical Audit - Initial management of
the fitting child:
http://www.rcem.ac.uk/ShopFloor/Clinical%20Audit/Previous%20Audits
Royal College of Paediatrics and Child
Health, Epilepsy12 audit:
http://www.rcpch.ac.uk/epilepsy12
Additional
information
Royal College of Paediatrics and Child
Health, Child Health Reviews UK – Clinical
Outcome Review Programme. Coordinating
epilepsy care, a UK–wide review of health care
in cases of mortality and prolonged seizures in
children and young people with epilepsies.
Epilepsy in Children and Young People. NICE
Quality Standard QS27.
Epilepsies: diagnosis and management: NICE
Guidelines CG137.
Domain
Safety
Measure
Number of serious incidents reported (as defined by the NHS England Serious Incident Reporting Framework) relating to physical or mental health
affecting infants, children and young people admitted to inpatient acute paediatric services.
An adverse event rate of between 2.1% and 10.8% of hospital admissions for patients aged 0-17 years of age was reported by a National Patient
Safety Agency Review in 2009, highlighting how common place such incidents are.
Reason for
consideration
Whilst adverse events in children are unfortunately common, serious harm arises in only a minority of incidents. Monitoring of all such events is
important, particularly if changes in the culture of reporting events lead to increases in the overall reporting rating.
RCPCH outcome measures consultation
September 2015
Additional
information
Measure
Reason for
consideration
Additional
information
Measure
Reason for
consideration
Additional
information
Domain
Measure
Reason for
consideration
Additional
information
Population
Number of admissions to
the paediatric department
of children and young
people aged 0-17 years.
Type of outcome
Clinical outcome
Potential data source
National Reporting and Learning
System, NHS England and Scotland,
Wales and NI equivalent datasets.
Associated standards or guidelines
Children and Young People’s Health Outcomes
Forum recommended indicator.
NHS England Serious Incident Reporting
Framework
Important patient information lost during shift change (e.g. important investigation results, or need to chase such results, or arrange a time
dependent investigation or clinical intervention).
With changes in the working patterns of medical staff since the introduction of the European Working Time Directive (EWTD), handover between
incoming and outgoing shifts has become more frequent. This has led to concerns about the potential for failure to communicate important patient
information (e.g. outstanding investigation results which could change management). This was highlighted by the recent Taskforce report on the
impact of the European Working Time Directive (2014).
Population
Type of outcome
Potential data source
Associated standards or guidelines
Number of admissions to
Process measure
New data source
Facing the Future standard 4
the paediatric department
of children and young
people aged 0-17 years.
Medication and treatment errors (Including incorrect drugs given, prescribed and/or dispensed).
Medication errors are the most commonly reported adverse incident type in infants, children and young people, and, as such, have been a particular
focus for improving patient safety.
Population
Type of outcome
Potential data source
Number of admissions to
Clinical outcome
National Reporting and Learning
the paediatric department
System, National Patient Safety Agency
of children and young
in England and Scotland, Wales and NI
people aged 0-17 years.
equivalent datasets.
Process markers of care quality proven to have impact of outcomes
Associated standards or guidelines
Length of stay in the emergency department:
Median time for infants, children and young people to be seen and/or discharged/admitted from department
Interquartile range for time for infants, children and young people to be seen and/or discharged/admitted from department
The maximum 4 hour wait for patients presenting to A&E has been a target for NHS hospitals since 2004, and was a particular focus of media
reporting in the winter of 2014/2015. Data regarding A&E waiting times is already collected routinely. There has been much criticism of the 4 hour
target as chosen. We would support consideration of a more continuous metric around waiting times
Population
Type of outcome
Potential data source
Associated standards or guidelines
Number of attendances at Clinical outcome
Hospital episode statistics.
A&E 4 hour target
the emergency
department of children
and young people aged 017 years.
RCPCH outcome measures consultation
September 2015
Measure
Reason for
consideration
Additional
information
Measure
Reason for
consideration
Additional
information
Measure
Reason for
consideration
Additional
information
Measure
Reason for
consideration
Additional
information
Unscheduled re-attendance at the emergency department within 48-72 hours of attendance.
In recent years admission rates to acute paediatric services have increased, with a decrease in the length of stay, and, in particular, a significant
increase in the proportion of admissions <1 day (see Gill et al 2013). Data around admission rates, admission lengths, re-attendance rates to the
Emergency department, and readmission within 48 hours of discharge are already routinely collected (through HES data). Interpreting changes in
any one of these metrics in isolation is extremely difficult, and so measurement of all 4 is likely to be more informative, e.g. a decrease in admission
rate could be a positive change, but not if this was accompanied by an increase in unscheduled re-attendances.
Population
Type of outcome
Potential data source
Associated standards or guidelines
Total number of children
Clinical outcome
and young people ages 0
to 17 attending the
emergency department.
Overall rate of admission to paediatric inpatient services.
See above under “Unscheduled re-attendance at the emergency department within 48-72 hours of attendance.”
Population
Total number of children
and young people aged 017.
Type of outcome
Clinical outcome
Potential data source
Hospital episode statistics.
Associated standards or guidelines
Facing the Future
Length of admission to paediatric inpatient services.
See above under “Unscheduled re-attendance at the emergency department within 48-72 hours of attendance.”
Population
Number of admissions to
the paediatric department
of children and young
people aged 0-17 years.
Type of outcome
Clinical outcome
Potential data source
Hospital episode statistics
Associated standards or guidelines
Facing the Future standards 1, 2, 3 and 7
Emergency readmission within 24 hours/48 hours/7 days of discharge from hospital for infants children and young people.
See above under “Unscheduled re-attendance at the emergency department within 48-72 hours of attendance.”
Population
Number of admissions to
the paediatric department
of children and young
people aged 0-17 years.
RCPCH outcome measures consultation
Type of outcome
Clinical outcome
Potential data source
Hospital episode statistics
Associated standards or guidelines
Facing the Future standards
Children and Young People’s Health Outcomes
Forum recommended indicator
September 2015
Measure
Reason for
consideration
Additional
information
Measure
Reason for
consideration
Additional
information
Measure
Reason for
consideration
Additional
information
Domain
Measure
Reason for
consideration
Additional
information
Following admission to paediatric acute inpatient services, timely clinical response to infants children and young people triggering Paediatric Early
Warning System (PEWS).
Adoption of Paediatric Early Warning Systems (PEWS) has been recommended for acute paediatric inpatient services, to enable early recognition of
deteriorating children and young people.
Population
All children and young
people requiring acute
medical assessment.
Type of outcome
Clinical outcome
Potential data source
New data source
Associated standards or guidelines
Rates of emergency admission for selected long term conditions, e.g. diabetes, asthma and epilepsy, for clinical issues which could have been
managed in the community.
Unplanned hospital admissions for these conditions may be an indicator of suboptimal management of the condition.
Population
Total number of children
and young people ages 0
to 17 attending the
emergency department.
Type of outcome
Clinical outcome
Potential data source
Hospital episode statistics
Associated standards or guidelines

Rates of emergency transfer to paediatric intensive care and

Level of ventilator and circulatory support required at transfer of CAYP to paediatric intensive care.
Rates of unplanned transfer of infants children and young people to paediatric intensive care services is highly variable between acute paediatric
services, and likely to be more strongly determined by clinicodemographic factors relating to the composition of the local population. A better
indicator of quality of care for these cases is possibly management prior to transfer, including early recognition of level of illness, early discussion
with PICM services and level of intensive care support required at transfer. Data is already routinely collected by PICM services (through the
PICANET database).
Population
Type of outcome
Potential data source
Associated standards or guidelines
Total number of children
Process measure
PICANet
PIC Society, draft Quality Standards Care of
and young people ages 0
Critically Ill & Critically Injured Children.
to 17 attending the
http://picsociety.uk/wpemergency department.
content/uploads/2015/08/PICS-CICstandardsV5-D24-20150716-PICS-VERSION.pdf
Measures of acute healthcare access & experience
Measure of staff physical and psychological health (e.g. levels of sickness, levels of absence, turnover).
The health and wellbeing of the workforce are essential to the delivery of good care to children and young people and families. Service change may
improve patient orientated outcomes, but this may come to the detriment of staff wellbeing.
Population
RCPCH outcome measures consultation
Type of outcome
Experience outcome
Potential data source
NHS Staff Survey
September 2015
Associated standards or guidelines
Measure
Reason for
consideration
A measure of staff satisfaction or attitudes toward safety (for example with the intervention, with care provided, with the level of education provided
about family-centred care).
One consistent conclusion from high profile reports into the failings of NHS care in recent years (e.g. the Berwick report), is the increased recognition
of the importance of all staff members taking ownership of the quality of care a service delivers.
Population
Type of outcome
Experience outcome
Additional
information
Measure
Reason for
consideration
Additional
information
Measure
Potential data source
NHS Staff Survey.
Associated standards or guidelines
Safety attitudes and safety climate
questionnaire:
https://med.uth.edu/chqs/surveys/safetyattitudes-and-safety-climatequestionnaire/
A measure of patient/parent satisfaction.
The experience of children, their parents and carers must remain at the centre of carer child health services provide.
The recent Care Quality Commission national children’s hospital survey highlighted the generally positive experience children report of hospital stay,
with 87% of children happy with their care. Room for improvement was also highlighted, e.g. 41% of parents and carers felt that members of staff
were not always aware of their child’s medical history.
Population
Type of outcome
Potential data source
Associated standards or guidelines
Experience outcome
CQC national children’s hospital survey.
Additional
information
A measure of patient stress/anxiety levels and emotional wellbeing whilst in hospital.
Services delivering “good” healthcare to children and young people must be able to attend to more than physical symptoms. Truly patient orientated
services must also address the emotional wellbeing of children. The Care Quality Commission national children’s hospital survey did not specifically
ask questions regarding emotional wellbeing or stress.
Population
Type of outcome
Potential data source
Associated standards or guidelines
Experience outcome
New data source
Measure
Reason for
consideration
All infants children and young people requiring pain relief receive effective multimodal pain relief with documentation of efficacy.
Pain is a common symptom in children admitted to hospital. The recent Care Quality Commissions’ national children’s hospital survey demonstrated
the potential for improvement in pain control for children and young people admitted to hospital.
Reason for
consideration
Population
Type of outcome
Process outcome
Potential data source
New data source
Additional
information
RCPCH outcome measures consultation
Associated standards or guidelines
Association of Paediatric Anaesthetists of Great
Britain and Ireland guideline.
Royal College of Emergency Medicine,
Management of Pain in Children:
file:///C:/Users/rachelw/Downloads/CEM4682Pain-in-children-(REV-July-2013).pdf
September 2015
*Hospital episode statistics refers to data published by the Health and Social Care Information Centre in England, Information Services Division in Scotland,
NHS Wales Informatics Service and the Department of Health, Social Services and Public Safety in Northern Ireland.
Further work will be undertaken to look at the appropriateness of all potential data sources.
RCPCH outcome measures consultation
September 2015
Table 2: Feedback on shortlist
Measure
Measure focus
Acceptability
Relevance
Data collection
What particular
measure should we
focus on? For
example, “Rates of
admission for
ambulatory sensitive
conditions”, which
condition should be
used?
How acceptable
would this measure
to be to you as an
indicator of child
health outcomes?
1. Completely
acceptable
2. Moderately
acceptable
3. Not at all
acceptable
How feasible do you
think it would be to
improve this aspect
of care through
service change?
1. Completely
feasible
2. Moderately
feasible
3. Not at all
feasible
Based on your local
service, how feasible
do you think it would
be to collect these
data? Do you
already collect it?
Administration of intravenous
antibiotics within 1 hour of triage
to acute paediatric services of
infants, children and young
people with suspected sepsis.
Management of acute
exacerbations of asthma in
infants, children and young
people according to British
Thoracic Society/Scottish
Intercollegiate Guidelines
Network Guidelines.
Adherence to the Advanced
Paediatric Life Support (APLS)
guidance (or personalised
Epilepsy care plan if in place)
for the treatment of infants,
children and young people
presenting in or developing
status epilepticus whilst an
inpatient.
Number of serious incidents (as
defined by the NHS England
Serious Incident Reporting
Framework) relating to physical
or mental health reported.
RCPCH outcome measures consultation
September 2015
Sensitivity to
change
In your opinion,
would these events
occur frequently
enough to detect
changes in quality of
care?
Additional
comments
Please add any
additional comments
relating to your
responses, and any
general comments
on this measure.
Please also note if
there are data
sources or
guidelines we have
missed.
Important patient information
lost during shift change (e.g.
important investigation results,
or need to chase such results,
or arrange a time dependent
investigation or clinical
intervention).
Medication and treatment errors
(Including incorrect drugs given,
prescribed and/or dispensed).
Length of stay in the emergency
department:
Median time for infants,
children and young people
to be seen and/or
discharged/admitted from
department
- Interquartile range for time
for infants, children and
young people to be seen
and/or discharged/admitted
from department
Unscheduled re-attendance at
the emergency department
within 48-72 hours of
attendance.
Overall rate of admission to
paediatric inpatient services.
Length of admission to
paediatric inpatient services.
Emergency readmission within
24 hours/48 hours/7 days of
discharge from hospital for
infants children and young
people.
Following admission to
paediatric acute inpatient
services, timely clinical
response to infants children and
young people triggering
Paediatric Early Warning
System (PEWS) system.
RCPCH outcome measures consultation
September 2015
Rates of admission for
ambulatory sensitive conditions,
for example diabetes, asthma or
epilepsy.
Rates of emergency transfer to
paediatric intensive care, and
level of ventilator and circulatory
support required at transfer of
CAYP to paediatric intensive
care.
Measure of staff physical and
psychological health (e.g. levels
of sickness, levels of absence,
turnover).
A measure of staff satisfaction
(for example with the
intervention, with care provided,
with the level of education
provided about family-centred
care).
A measure of patient/parent
satisfaction.
A measure of patient
stress/anxiety levels and
emotional wellbeing whilst in
hospital.
All infants children and young
people requiring pain relief
receive effective multimodal
pain relief with documentation of
efficacy.
RCPCH outcome measures consultation
September 2015
2. Would you be interested in your organisation taking part in the piloting stage?
☐
Yes
☐
No
3. Are you already collecting/collating outcomes for acute general paediatrics that are not included in the list above? If so, please
provide details.
4. Additional comments:
Please use the box below to provide any additional comments on the shortlist or on the project in general.
RCPCH outcome measures consultation
September 2015