Evaluation of the use of a parent questionnaire to provide later health status
data: The Panda Study
David Field, Edi Spata, Thomas Davies, Brad Manktelow, Samantha Johnson,
Elaine Boyle and Elizabeth Draper.
All authors are from the Department of Health Sciences, University of Leicester,
Leicester, UK.
Corresponding Author:
Prof D Field
Department of Health Sciences
University of Leicester
22-28 Princess Road West
LEICESTER
LE1 6TP
United Kingdom
Phone: +44 (0)116 252 5468
Fax: +44 (0)116 252 3272
Email: df63@leicester.ac.uk
Word count: 2641
Key words: Neonatology, outcomes research.
ABSTRACT (248 words)
Background: Routine comparable outcome data collection relating to the later
health status of babies born very preterm has long been considered important but
has not been achieved in the UK.
Aim: To test the potential for a parental questionnaire to provide this data for all
eligible babies from a geographical population.
Methods: Consent for follow up by questionnaire (using the PARCA-R combined
with questions derived from the Oxford minimum dataset) was sought for all babies
≤30 weeks of gestation discharged from a hospital in the East Midlands and
Yorkshire Regions of the UK having been born between 1/1/2007 and 31/12/2011.
Results:
The rate of consent to participate in follow-up showed a steady increase over time to
83.1% in 2011. However the response rate in terms of completion and return of the
questionnaire at two years, as a proportion of those eligible, showed little change
overtime varying between 42% and 46%.
Among those children where a questionnaire was returned the rate of disability was
broadly consistent over time: lowest 2009 21.0% (95%C.I. 16.8% to 25.6%); highest
2011 25.5% (95%C.I.21.5% to 31.2%). The instruments used appeared effective
with the capability of discriminating between children with physical and or cognitive
disability.
Conclusion
The overall response rate in terms of returned questionnaires was disappointing and
inadequate to recommend for implementation. It is possible that response rates
would have been higher had clinical follow up been linked to the data obtained from
the questionnaires rather than rather than running as a parallel process.
INTRODUCTION
There has been a longstanding acceptance by the neonatal clinical community that
increased survival of the sickest and most immature babies might be associated with
significant rates of long-term morbidity. As a consequence there are large numbers
of published reports providing snapshots of later outcomes following preterm birth.
However, lack of standardisation of approach in these studies has made it difficult to
establish a clear perception of what constitutes good or poor performance in terms of
the overall outcomes for graduates of neonatal intensive care.
In contrast, interest in the concept of formal care quality assessment is relatively
recent in the UK having largely followed the reorganisation and separation of the
processes of commissioning and delivery of care in the 1990s. Whilst this has
increased the desire for on-going data about later health status of high risk
newborns1 both the cost of introducing formal assessments of health and
development in childhood and the lack of standardisation in defining outcomes have
hindered progress.
In 1993, an Audit Commission report2 highlighted the lack of outcome data for babies
discharged home from neonatal care. In response, a major initiative was launched to
establish a simple standardised system for measuring later health status in high-risk
babies3. The subsequent report aimed to define, for each of the main areas of health
and development, characteristics that represented severe disability. For assessing
cognitive impairment it recommended a cut-off of 2 standard deviations for defining
impaired function and below 3 standard deviations for the definition of severe
cognitive impairment, using any well validated test. During the 1990s it became clear
that a simple, inexpensive and effective means of assessing cognitive development
was required for population screening and for research-based outcome evaluations.
This led to the development of parental questionnaires as cost-efficient methods of
first-line screening. To date, the Parent Report of Children’s Abilities-Revised
(PARCA-R), designed for use at two years corrected age, has been the most widely
used in preterm populations, mainly as an outcome measure in randomised trials4 5.
Assessment at two years is widely accepted as the first opportunity at which overall
health and cognitive status can be reliably measured.
Our study explored the use of a parent report version of the Oxford minimum dataset
and PARCA-R as a cost-effective means of providing routine follow-up data at two
years corrected age. Specifically, we wished to know whether:
(i)
A postal questionnaire to parents would be an effective way of obtaining
routine data on neurodevelopmental outcomes in babies from high risk
categories that would be suitable for quality improvement purposes.
(ii)
A postal questionnaire to parents would be an effective way of screening
babies from high risk categories for evidence of impaired health status at 2
years.
METHOD
The PANDA (Preterm AND After) study was based in the East Midlands and
Yorkshire regions of England, which account for approximately 15% of UK births. All
neonatal intensive care admissions in these regions were captured by The Neonatal
Survey between 2004 and 20146. In 2009, the Commissioning Groups for these
regions agreed to fund an exploratory investigation of the use of a parent
questionnaire, which combined the PARCA-R and additional questions to classify
motor and sensory disability based on the Oxford minimum dataset, for clinical
outcome assessment in all infants born at ≤30 weeks of gestation. This gestational
age group was chosen because it would include babies at greatest risk of adverse
outcome but also ensured that the number of babies included (approximately 1% of
births) remained manageable.
The Trent Multi-centre Research Ethics Committee gave approval, designating the
project as meeting the criteria for research. It was therefore necessary to obtain
written consent from the parents of all eligible babies.
Sole reliance on prospective data collection would have incurred a delay of over
three years before data on outcomes at two years (based on a full year of births ≤30
weeks of gestation) could be reported to commissioning bodies. We therefore used
two different approaches in parallel:
1) Retrospective identification. Funding commenced in 2009. All cases born in 2007
and 2008 and eligible for inclusion in terms of birth address and gestation were
identified from The Neonatal Survey. After checking the vital status of all babies with
the NHS central register, consultants were contacted in the hospitals from which
these babies were discharged following neonatal care. They were asked to send a
standard letter to parents seeking their consent to be involved in the questionnaire
follow-up. One reminder was sent to non-respondents.
2) Prospective identification. During 2009 to 2011, neonatal units discharging eligible
babies were asked to obtain consent from parents for participation in the PANDA
follow-up project. Although we encouraged clinical teams to gain consent before
discharge, some preferred to use their neonatal outreach service to gain consent
after discharge. During the final year of data collection this work was included in the
National Institute of Health Research (NIHR) research portfolio (as part of a larger
EU FP7 project), which meant that hospitals received a financial benefit for
recruitment.
A single part time administrator ran the project from The Infant Mortality and
Morbidity Studies (TIMMS) office at the University of Leicester.
As all eligible babies were born preterm their second birthday fell before a corrected
age of two years and a note informing parents that the assessment forms would
soon arrive was sent with a 2nd birthday card where possible. The study
questionnaire and a reply paid envelope were then sent with instructions asking
parents to complete the questionnaire as close as possible to the identified date on
which the child would reach two years corrected age. Paper forms were used for
data collection and one reminder was sent by post if the forms were not returned
after three to six weeks.
Measures
The questionnaire consisted of two sections. The first was a parent report adapted
from the Oxford minimum dataset (Table 1). Parents were asked to rate their child’s
health and development using forced-choice questions corresponding with the
Oxford minimum dataset domains so that severe disability could be classified in each
domain.
To assess cognitive development, we used the PARCA-R parent questionnaire. This
is designed to assess non-verbal cognitive and language development in preterm
infants at two years corrected age. From this, the total Parent Report Composite
(PRC) score, which combines non-verbal cognition and the linguistic skills sub-scale
scores, was used to assess cognitive function. PRC scores can range from 0 to 158,
with lower scores representing poorer performance, and a PRC score of less than 49
was originally recommended to identify moderate/severe delay, equivalent to test
scores < -2SD7.. A number of studies have explored the validity of the PARCA-R and
have produced slightly different cut-off scores with optimum diagnostic utility for
identifying moderate/severe developmental delay7-10. We opted to use a cut-off of
<44 as identified in Johnson et al 20088. This was derived from a large validation
study of very preterm infants born <32 weeks' gestation and most closely matches
the population studied in our paper (i.e., babies born <32 weeks gestation in the UK).
The PARCA-R has been shown to have concurrent validity with gold standard
developmental tests and diagnostic utility for identifying infants with developmental
delay in numerous cohorts9-11.
The time taken to complete the two parts of the questionnaire has been estimated at
around 15 minutes and generally, where attempted, is completed easily with few
items left blank12 13.
Table 1: Severe disability at 2 years was defined using definitions derived from the
Oxford Minimum Dataset which is shown below:
Domain
Aspects of development considered to
represent severe disability
Malformation
Any malformation which despite physical assistance
impairs the performance of daily activities.
Unable to sit;
Unable to use hands to feed self;
Unable to control head movement without support or no
head control.
Seizures more than 1/month despite treatment
Hearing impaired, uncorrected even with aids.
Unable to comprehend word/sign in cued situation;
Unable to produce >5 recognisable sounds or no
vocalisation.
Blind or sees light only.
More than 3 standard deviations below mean on
standardised assessment.
Neuromotor
function
Seizures
Auditory function
Communication
Visual function
Cognitive function
RESULTS
Twenty-three Primary Care Trusts (PCTs; the commissioning bodies when the study
was established) and 30 participating hospitals were situated within the study area. A
large number of babies received care in more than one hospital. For individual PCTs,
the highest number of potential cases in any one year was 85 and the lowest was 4.
Aggregated data are therefore presented.
There was little variation in the numbers of eligible babies in each of the five years
for which data are presented and >4000 over the 5 years (Table 2).
Prospective consent was sought during 2009, 2010 and 2011. In 2009, 102 babies
were excluded from the study primarily because the required hospital permission
processes and approvals were not in place at the time of discharge. The rate of
consent to participate in follow-up showed a steady increase over time to 83.1% in
2011 when the study was accepted onto the NIHR portfolio. However despite this
increase in early parental consent, the eventual response rate in terms of completion
and return of questionnaires as a proportion of those eligible at two years showed
little change overtime varying between 42% and 46%. Amongst parents that
provided prospective consent the rate of return of completed forms was initially
63.2% but fell to 51.8% by the end of the study.
Findings from assessment forms that were returned are summarised for the whole
population in Table 3. The rate of disability appears to be consistent over time, with
no suggestion of either improvement or change. Within any one domain there were
apparent differences between years (e.g. Neuromotor function 9.7% [7.0% to 13.0%]
in 2007 vs 4.9% [2.9% to 7.8%] in 2009) but confidence intervals are wide because
of small numbers consequently these changes were not significant.
Table 2: Pattern of consent and response amongst those eligible to join the study.
Status
Eligible
Prospective consent
obtained
2007
2008
2009
2010
2011
880
819
821
820
864
-
-
551
644
718
67.1
78.5
83.1
Consent obtained as
a percentage of
eligible
Eligible but not
approached
-
-
120
128
106
Excluded (reasons
included permissions
not in place or
language difficulties)
-
-
102
10
6
Died
8
8
13
7
2
Lost to follow-up
21
11
4
1
0
Responders returning
questionnaires
413
358
348
371
372
Response as a
percentage of eligible
46.9
43.7
42.4
45.2
43.1
Response as a
percentage of those
who provided
prospective consent
N/A
N/A
63.2
57.6
51.8
Table 3: Mean percentage rates of disability in each of the various domains (and
95% Binomial Confidence intervals) based on those children whose parents
responded to the questionnaire in each of the 5 years:
Year of birth
2007
2008
2009
2010
Responded to questionnaire 46.9%
43.7%
42.4%
45.2%
as a proportion of those
(43.6% to 50.3%) (40.3% to 47.2%) (39.0% to 45.9%) (41.8% to 48.7%)
eligible
2011
43.1%
(39.7%, 46.4%)
OF THOSE WHO RESPONDED:
Moderate/severe cognitive
delay: PRC < 44
22.3%
19.6%
21.8%
17.8%
25.8%
(18.4% to 26.6%) (15.6% to 24.1%) (17.6% to 26.6%) (14.0% to 22.1%) (21.4% to 30.6%)
Severe disability
Malformations
Neuromotor function
Seizures
Auditory function
Communication
5.3%
4.8%
5.8%
8.9%
8.1%
(3.4% to 8.0%)
(2.8% to 7.5%)
(3.6% to 8.7%)
(6.2% to 12.3%)
(5.5% to 11.3%)
9.7%
8.5%
4.9%
6.0%
7.9%
(7.0% to 13.0%)
(5.8% to 11.8%)
(2.9% to 7.8%)
(3.8% to 8.9%)
(5.3% to 11.1%)
2.9%
2.0%
2.6%
1.9%
2.4%
(1.5% to 5.1%)
(0.8% to 4.0%)
(1.2% to 4.9%)
(0.8% to 3.9%)
(1.1% to 4.6%)
1.7%
2.0%
2.9%
2.5%
1.4%
(0.7% to 3.5%)
(0.8% to 4.0%)
(1.4% to 5.3%)
(1.1% to 4.6%)
(0.4% to 3.2%)
13.5%
12.9%
12.9%
12.1%
18.8%
(10.4% to 17.2%) (9.6% to 16.8%)
Visual
Any severe disability
(9.6% to 16.9%)
(9.0% to 15.9%) (15.0% to 23.2%)
0.7%
0.3%
0.3%
0.3%
0.3%
(0.1% to 2.1%)
(0.0% to 1.6%)
(0.0% to 1.6%)
(0.0% to 1.5%)
(0.0% to 1.5%)
24.9%
22.9%
21.0%
22.4%
25.5%
(20.8% to 29.4%) (18.7% to 27.6%) (16.8% to 25.6%) (18.2% to 27.0%) (21.5% to 31.2%)
We wished to assess whether the Oxford Minimum Dataset and the PARCA-R
identified different children, to determine whether those with motor or sensory
disability also performed poorly on the PARCA-R. This comparison is shown in Table
4. Although there is overlap in the children identified with problems, a significant
number of children are detected with problems by one of the two screening tests but
not by both.
Table 4: Number (and percentage) of children that met the criteria for disability
based on the various categories of the Oxford minimum dataset (but not cognitive
deficit) and whether they did or did not also demonstrate cognitive impairment based
on a PARCA-R of <44
2007
2008
2009
2010
2011
Severe disability
Severe disability
Severe disability
Severe disability
Severe disability
PARCA-R score
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
PRC < 44
67
(16.1%)
25
(6.1%)
52
(14.5%)
18
(5.0%)
51
(14.7%)
25
(7.2%)
52
(14.0%)
14
(3.8%)
74
(19.9%)
22
(5.9%)
PRC ≥ 44
36
(8.7%)
285
(69.0%)
30
(8.4%)
258
(72.1%)
22
(6.3%)
250
(71.8%)
31
(8.4%)
274
(73.9%)
26
(7.0%)
250
(67.2%)
103
310
82
276
73
275
83
288
100
272
Total
DISCUSSION
Outcome data at 2 years regarding the survivors of neonatal intensive care has long
been considered an important quality measure. Given the financial constraints within
the NHS this study aimed to assess a cost-effective method of collecting such data
that could be done on a routine basis, and we believe represents a first in the UK.
However the overall response rate in this study was disappointing. Considering the
eligible population in each year as a whole, completed questionnaires were returned
for less than half the children in any of the five years. Surprisingly the percentage of
returned forms was highest in 2007 when information was requested without any
prior warning, suggesting parents’ willingness to collaborate with this type of follow
up may have declined over time, a finding duplicated in research studies of preterm
babies and population-based cohorts more generally within the UK13-15 indicating
that the poor response does not relate simply to this particular combination of
instruments. Amongst families who gave prospective consent response rates varied
from 63% (2009) to 52% (2011).
We chose to use a score of 44 or less for the PARCA-R to identify potential cognitive
deficits and we accept that alternative cut offs could have been more appropriate
particularly if the aim is to identify severe rather than moderate/severe disability.
However the results make clear that even when using a cut-off score of 44 the
PARCA-R is largely identifying a different group of children to those with other forms
of disability. This underscores the need to assess multiple developmental domains to
identify those with potential long term problems.
Strengths and limitations
A number of factors may have contributed to the disappointing response rate. One
possibility is poor communication between the research team and the families, but
the fact that the system worked successfully for many families makes this seem
unlikely. In addition, the population that was the focus of the study is very well known
to the researchers involved. A single part time administrator ensured that
questionnaires and reminders were sent to parents and hence, there was no
personalised “chasing” of non-responders. Providing this may have increased the
response rate but would have substantially increased the expense associated with
the study, and it had been agreed with commissioners that a low cost approach
would be assessed i.e. a method with potential to be used routinely by the NHS.
The main aim of our study was to obtain data relevant to the quality improvement
agenda, with feedback of the findings to both providers and Commissioners. As such
there was no feedback to parents and no incentive, as the system was run as simply
and cheaply as possible. The method we employed therefore provided no potential
benefit for the child or family. It may be that, for many families, the motivation simply
to provide information about their child’s health status was poor compared with other
inevitable calls on their time. It is also important to note that, within the context of our
study, questionnaire follow-up was not intended to replace clinical follow-up and ran
in parallel with systems already in place in participating hospitals.
We chose to use a combination of questionnaires that we believed could simply and
effectively provide a full picture of the child’s health status and the results of this
study show that a combination of measures is indeed needed to identify all those
with developmental impairment at two years. However, some aspects of the
questionnaires may have represented a challenge for parents who were less able
intellectually or from ethnic minorities and whose first language was not English. We
have considered previously whether the use of an online system of completion would
have enhanced the response16.
Implications for practice
The study described here was developed primarily to identify a cost effective means
of providing data that Commissioners could use to monitor care quality. Since data
collection commenced for this study, having a system in place to monitor health
status at two years has become a CQUIN (Commissioning for Quality and
Innovation) for neonatal care. However it may well be that it was this limited aim of
simply providing audit data that led to the lack of engagement with some families. In
the future, it would seem sensible that any process to collect routine later health
status data should, at the very least, have a means of providing feedback to parents
about the results and the potential for clinical consultation for those with emergent
parental concerns or screening positive for developmental problems. Having the
child and family at the centre of a process that provides standardised “care quality
information” as a natural bi-product seems much more likely to meet with parental
engagement.
When considering alternative approaches to collecting this type of information in the
future it may be important to recognise the potential for replication of developmental
assessments that some children and their families may face. For “high risk” births
there will normally be a period of routine clinical follow up in outpatient clinics
perhaps culminating in some formal developmental assessment process. In addition,
the child’s health visitor will also assess the child’s health status as he or she grows
as part of the Health Child Programme, which from September 2015 will include a
request for parents to complete a developmental screening questionnaire. These
systems currently run in parallel and efforts to integrate and standardise these
assessments may improve patient engagement with routine developmental
screening and follow-up for their child. Certainly there is evidence that the relevant
Government Departments are actively looking for ways to harmonise the various
approaches to assessment currently in place
(http://www.foundationyears.org.uk/files/2013/11/Information_Sharing_in_the_Found
ation_Years_Report_FINAL.pdf ; http://www.local.gov.uk/health//journal_content/56/10180/5886759/ARTICLE).
.
CONCLUSIONS
It appears that further work is needed to establish a low cost means of obtaining data
for routine reporting of health status in high-risk preterm babies. Simply requesting
data for this purpose alongside existing follow-up does not appear to be effective. A
process focused on the individual child’s needs that is part of an existing surveillance
programme and which, as a by-product, generates the required information may be
an appropriate way forward. However, it seems reasonable to consider testing the
effectiveness of questionnaire based follow-up in those groups where large numbers
preclude formal outpatient follow-up from birth, such as babies born late or moderate
preterm, despite evidence of significant rates of adverse outcomes at and beyond
two years.
ACKNOWLEDGEMENTS
We wish to acknowledge the collaboration of all of the teams delivering perinatal
care in the East Midlands and Yorkshire and adjacent regions. We also wish to thank
all of the parents who contributed to this work. We also wish to provide special
thanks to Mrs Helen Holden who managed the PANDA project.
FUNDING
Funding for the PANDA study was provided by the NHS Specialist Commissioners
for the East Midlands and Yorkshire Regions of the UK.
COMPETING INTERESTS
None
AUTHORSHIP
DF,ESD and BM planned the original study. BM, ES and TD carried out the analysis.
All of the authors contributed to interpretation of the findings and preparation of the
final paper.
WHAT IS ALREADY KNOWN ON THIS TOPIC
Monitoring of the later health status of babies born very preterm has long been an
important aim of neonatologists. However attempts to provide such data in a manner
which allows the monitoring of outcomes over time and comparisons between
services has not been possible.
WHAT THIS STUDY ADDS
The use of validated questionnaires had been felt to be a potential means of
providing cost effective later health status data. Results of this study showed that the
rate of parental engagement was insufficient to recommend implementation. The
effect of additional parental reminders and or its use as a screening tool in the
absence of clinical follow up remain to be tested.
REFERENCES
1. Specialist neonatal care quality standard: National Institute for Health Care
Excellence, 2010.
2. Children First: A study of hospital services. London: The Audit Commission, 1993.
3. Johnson A. Follow up studies: a case for a standard minimum data set. Archives
of disease in childhood. Fetal and neonatal edition 1997;76(1):F61-3.
4. Marlow N, Greenough A, Peacock JL, Marston L, Limb ES, Johnson AH, et al.
Randomised trial of high frequency oscillatory ventilation or conventional
ventilation in babies of gestational age 28 weeks or less: respiratory and
neurological outcomes at 2 years. Archives of disease in childhood. Fetal and
neonatal edition 2006;91(5):F320-6.
5. Group IC, Brocklehurst P, Farrell B, King A, Juszczak E, Darlow B, et al.
Treatment of neonatal sepsis with intravenous immune globulin. The New
England journal of medicine 2011;365(13):1201-11.
6. Seaton SE, King S, Manktelow BN, Draper ES, Field DJ. Babies born at the
threshold of viability: changes in survival and workload over 20 years.
Archives of disease in childhood. Fetal and neonatal edition 2013;98(1):F1520.
7. Johnson S, Marlow N, Wolke D, Davidson L, Marston L, O'Hare A, et al. Validation
of a parent report measure of cognitive development in very preterm infants.
Developmental medicine and child neurology 2004;46(6):389-97.
8. Johnson S, Wolke D, Marlow N, Preterm Infant Parenting Study G. Developmental
assessment of preterm infants at 2 years: validity of parent reports.
Developmental medicine and child neurology 2008;50(1):58-62.
9. Cuttini M, Ferrante P, Mirante N, Chiandotto V, Fertz M, Dall'Oglio AM, et al.
Cognitive assessment of very preterm infants at 2-year corrected age:
performance of the Italian version of the PARCA-R parent questionnaire.
Early human development 2012;88(3):159-63.
10. Martin AJ, Darlow BA, Salt A, Hague W, Sebastian L, McNeill N, et al.
Performance of the Parent Report of Children's Abilities-Revised (PARCA-R)
versus the Bayley Scales of Infant Development III. Archives of disease in
childhood 2013;98(12):955-8.
11. Martin AJ, Darlow BA, Salt A, Hague W, Sebastian L, Mann K, et al. Identification
of infants with major cognitive delay using parental report. Developmental
medicine and child neurology 2012;54(3):254-9.
12. Dawson C, Perkins M, Draper E, Johnson A, Field D. Are outcome data
regarding the survivors of neonatal care available from routine sources?
Archives of disease in childhood. Fetal and neonatal edition 1997;77(3):F20610.
13. Johnson S, Evans TA, Draper ES, Field DJ, Manktelow BN, Marlow N, et al.
Neurodevelopmental outcomes following late and moderate prematurity: a
population-based cohort study. Archives of disease in childhood. Fetal and
neonatal edition 2015;100(4):F301-8.
14. Moore T, Hennessy EM, Myles J, Johnson SJ, Draper ES, Costeloe KL, et al.
Neurological and developmental outcome in extremely preterm children born
in England in 1995 and 2006: the EPICure studies. Bmj 2012;345:e7961.
15. Lynn P BJ, Kaminska O, Knies G and Nandi A. An initial look at non-response in
Understanding Society: University of Essex: Institue for Social and Economic
Research, 2012.
16. Johnson S, Seaton SE, Manktelow BN, Smith LK, Field D, Draper ES, et al.
Telephone interviews and online questionnaires can be used to improve
neurodevelopmental follow-up rates. BMC research notes 2014;7:219.