Research Protocol
Building Social and Emotional Competence in Young
High-Risk School Children:
A Pragmatic Randomised Controlled Trial of the Incredible Years
Therapeutic (Small Group) Dinosaur Curriculum in Gwynedd Primary
Schools, Wales
Tracey Bywater, Judy Hutchings, Ceri Evans, & Laura Parry
Final version: 21.09.2010
Building social and emotional competence in young high-risk school children
Written by: Tracey Bywater1, Judy Hutchings2, Ceri Evans3, & Laura Parry4
1
University Principal Investigator, 3Research Project Support Officer, School of
Psychology, Bangor University, Brigantia Building, Bangor, Gwynedd, LL57 2AS
2
Director, Incredible Years Cymru, Nantlle Building, Normal Site, Bangor, Gwynedd,
LL57 2PZ
4
Ph.D Student, funders: School of Psychology, Bangor University, and Incredible
Years Cymru, addresses as above.
Author contributions:
TB and JH designed the study, TB wrote the initial draft of the protocol, JH, CE, and
LP were involved in subsequent drafts of the manuscript and revising it critically for
intellectual content.
Direct correspondence to first author:
t.bywater@bangor.ac.uk
01248 383845
Funding:
This project is funded by the BIG LOTTERY. Incredible Years Cymru is the grant
holder, with Bangor University and Gwynedd Education as partners.
Ethics approval:
The project “Building social and emotional competence in young high-risk school
children”, Ethics proposal 1506, was granted ethical approval on 1.3.2010 by the
School of Psychology Ethics Review Committee Bangor University.
North Wales Research Ethics Committee (West) 10/WNo01/55 gave a favourable
opinion of this study on 17th September, 2010.
Acknowledgements:
We wish to thank the funders for the wonderful opportunity to make a difference in
the community. Thank you also to partners and collaborators for being so
supportive in all steps taken so far in this research project. Special thanks to Chris
Whitaker from NWORTH, for his statistical support and advice. In anticipation;
thank you to all schools, parents, & children, without whom this project will not be
possible.
Trial Registration:
UK Clinical Research Network - UKCRN ID 8615
ISRCTN has been applied for.
Project website:
www.incredibleyearscymru.org
Conflict of interests:
The Chief Investigator, Professor Hutchings is the Director of the Charity that holds
the grant. Professor Hutchings is answerable to the Charity trustees.
2
Building social and emotional competence in young high-risk school children
Contents
Section:
Page:
1. Abstract
5
2. Background literature
6
3. Method and design
8
4. Ethical issues
19
5. Project management
21
6. Communication and dissemination
21
7. References
23
8. Appendices
Appendix 1: Gantt Chart
26
Appendix 2: Participant Flow Chart
27
Appendix 3: Details on measures
28
Appendix 4: Organisational Chart
43
Appendix 5: Funder reporting schedule
44
3
Building social and emotional competence in young high-risk school children
Purpose of the Protocol
This protocol provides information regarding the background literature, design, and
tools for the Lottery funded research project investigating whether The Incredible
Years (IY) Therapeutic Dinosaur School programme, when delivered as a schoolbased targeted intervention with ‘at-risk’ children, improves social and emotional
competencies. The Programme is an 18-week (2-hours/week) child-training
curriculum that strengthens children’s social, emotional and academic competencies,
such as understanding and communicating feelings, problem-solving strategies, anger
management and friendships.
Section 1 provides a summary of the protocol, Section 2 is an overview of the
literature on child conduct problems and a brief description of IY Therapeutic
programme, concluding with a summary of the background literature, aims, and
rationale for this study. In Section 3 the design and method are detailed. Section 4
presents ethical issues, Section 5 relates to project management, Section 6 discusses
the communication and dissemination plan, Sections 7 and 8 include references and
appendices.
In addition to this protocol there is an accompanying Home & School Visits Manual
(Bywater et al., in preparation) to aid researchers when in the field; it provides
details on participant recruitment criteria, home and school visit procedures, more
detailed description of the measures to be used, including the rationale for their
selection, administered and scoring information.
This three-year project begins in June 2010 and ends in May 2013 (see project Gantt
Chart, Appendix 1 for details) enabling assessment of 240 children over the three
years, undertaken as a two-phase project.
4
Building social and emotional competence in young high-risk school children
1. Abstract
Background
Parenting programmes are effective in developing children’s social skills and reducing
child behavioural problems, however provision is limited, some parents cannot be
recruited and although parenting programmes improve child behaviour at home this
does not always generalise to school settings. School-based classroom programmes
such as the Incredible Years Classroom Dinosaur Programme target whole
classrooms, yet for some children this is not enough - a more intense approach is
needed. The Incredible Years Therapeutic Dinosaur Programme was developed, and
proved useful, for clinically referred children by increasing social skills, problem
solving and peer relationships when delivered in a clinical setting.
The aim of this trial is to evaluate the effectiveness of the Therapeutic programme,
delivered with small groups of children at high-risk of developing conduct disorder,
within schools already implementing the Classroom Programme.
Methods
This is a pragmatic, parallel, randomised controlled trial.
Children aged 4-8 years will be recruited to the study with parental consent (N =
240). The inclusion criterion consists a teacher rating of a child as above the
‘borderline’ to ‘abnormal’ range on the Strengths and Difficulties Questionnaire.
Randomisation is by individual, within blocks (schools); 1:1 ratio, intervention to
waiting list control, stratified by gender. Twenty schools will participate in two
phases. Two teachers from each school will deliver the programme to six
intervention children for 2-hours/week for 18 weeks between baseline and first
follow-up. The control children will receive the intervention after first follow up.
Phase 1 participants will have data collected at three points – baseline and two
follow-up points, eight months post baseline and again eight months later. Phase 2
participants will not receive the second follow-up.
The IY Therapeutic Programme includes; how to do your best in school and learn
school rules, understanding, identifying, and articulating feelings, problem solving,
anger management, how to make and keep friends.
The primary outcomes are changes in child social, emotional, and behavioural
difficulties. Secondary outcomes are teacher and parental mental wellbeing, child
academic attainment, child and teacher school attendance/absenteeism. Intervention
delivery will be assessed for fidelity from videotaped recordings of sessions.
Intention to treat analyses will be conducted. ANCOVA, effect sizes, mediator and
moderator analyses will be applied to establish differences between conditions, and
for whom the intervention works best and why.
Discussion
This trial will provide information on the effectiveness of the IY Therapeutic
Programme, delivered to high-risk pupils in schools already implementing the
Classroom Programme.
5
Building social and emotional competence in young high-risk school children
2. Background literature
Children who are not socially/emotionally competent and display anti-social
behaviour and conduct problems are at high risk of developing early onset conduct
disorder (CD). CD is the biggest child mental health problem and is increasing
(Office for National Statistics, 1999). Diagnostic criteria for CD include clusters of
symptoms used to form broad descriptions of an individual’s functioning;
characterised by age inappropriate disruptive and antisocial behaviours that include
high rates of oppositionality, defiance and aggression (Webster-Stratton, 2003).
Children with early onset problems are at high risk of low school attendance,
educational underachievement, school dropout (Kupersmidt et al., 2000) and
subsequent mental health problems (Webster-Stratton, 1998).
Living in poverty, plus the disadvantages of having young and/or single parents, living
in poor quality housing, and with adults with unemployment, and mental health
problems increases the probability that children will arrive at school with social and
emotional skill deficits (Ward, 2009). There has been a significant increase in poverty
and disadvantage across Wales in recent years (Brewer, Muriel, Phillips & Sibieta,
2009) with one in six children (17%) now living below the nationally recognised
poverty level (Office for National Statistics, 2007). Additional risk factors associated
with higher prevalence for conduct disorders include, frequent changes of parental
figures, parental psychopathology, parental substance abuse, marital problems, and
poor parenting skills (Bloomquist & Schnell, 2002).
The IY programmes, for parents, children and teachers, have been developed over
the last 30 years and have strong evidence of effectiveness both as treatment
programmes for children with CD and as early preventive programmes. This
evidence comes from many different countries including America, Canada, Norway,
England, Wales and Jamaica (Webster-Stratton, 1998; Webster-Stratton & Taylor,
2001; Hutchings, Lane, Owen, & Gwyn, 2004; Baker-Henningham, Walker &
Gardner, 2009; Larsson et al., 2008).
Parenting programmes can be very effective in both developing children’s social skills
and reducing child behavioural problems (Webster-Stratton & Hammond, 1997;
Hutchings, Bywater, Daley, Gardner et al., 2007; Bywater et al., 2009). These
programmes also reduce later social care and education costs, and the costs of
lifetime unemployment and welfare dependence (Edwards, et al., 2007; Raver, 2002).
However, provision is limited to the few families that can access programmes due to
their cost and difficulties in recruiting high-risk families. Even when parents do access
programmes schools continue to deal with significant numbers of children with
behavioural and self-regulatory difficulties because improvements at home do not
always transfer to school settings (Taylor & Biglan, 1998; Mayhall, 1994).
Gwynedd Education Service have established the universal IY Classroom Dinosaur
curriculum in all of their 102 primary schools; however high-risk children need
additional input to meet their needs. The IY Therapeutic programme (WebsterStratton, 1990) fulfils the criterion of improving social and emotional competence in
high-risk children. It was developed for clinically referred children for whom it
increases social skills, problem solving and strengthens peer relationships (WebsterStratton, Reid, & Hammond, 2001).
6
Building social and emotional competence in young high-risk school children
In Wales initial steps to tackle the increasing numbers of children arriving in school
with social and emotional difficulties have been taken; there has been a change in the
curriculum of early years education (The Foundation Phase) and, a commitment to
improve home-school relationships. The IY Therapeutic programme (WebsterStratton, 1990) has been run in Welsh CAMHS services but the cost of transport
and clinician costs make this a costly intervention which can only be delivered to a
few, despite the need, with levels of CD in disadvantaged areas as high as 35%
(Attride-Stirling, Davies, Day & Sclare, 2000; NICE, 2006). It seems likely that the IY
Therapeutic programme would be ideally placed within schools, delivered by
teachers.
Summary of background literature
•Conduct problems are increasing in young children
•Social & emotional deficits among children entering school is increasing
•Parent programmes are beneficial but more support is needed for children
•Schools are an ideal arena for targeting difficult behaviour due to:
–The volume of accessible children and possible impact
–Learning behaviour ‘rules’ in one context may not generalise to another
–The opportunity they have to strengthen school-home links
•Gwynedd has IY programmes (Teacher and Classroom Dina) in all 102 primary
schools
•There is a need to evaluate IY Therapeutic Dina with high-risk children in these
schools
The rationale for the trial
The project will identify & support young school children with social, emotional and
behavioural deficits that limit their ability to thrive in school by giving them the
additional support to enhance their school readiness. It addresses one aspect of
early preventive intervention that can reduce behaviours associated with risk of
long-term antisocial behaviour, delinquency & substance misuse. Conducting this
study will establish whether the IY Therapeutic Dinosaur School Programme, when
delivered as a school-based targeted intervention, improves ‘at risk’ children’s social
and emotional competencies.
The primary research question is:
1. Does small group coaching in the Therapeutic Dinosaur curriculum to groups
of 6 high-risk 4-8 year old children attending schools where the classroom
version of the same curriculum is also being delivered, provide added benefits
to these children?
Secondary questions include:
2. For which children is the intervention most effective?
3. What are the environmental/contextual circumstances that improve the
likelihood of success?
4. Do parents and teachers perceive similar behaviour patterns in the children
at each time point, i.e. if positive behavioural changes are found in school
following participation in the intervention do they generalise to the home?
5. Does the duration of time participating in an intervention affect likelihood of
success, that is, is there a dosage affect?
7
Building social and emotional competence in young high-risk school children
6. Can the intervention be implemented efficiently and effectively with fidelity by
teachers in mainstream schools?
7. Does the intervention impact on teachers’ mental health?
3. Method and design
Timetable (see Gantt Chart, Appendix 1, for more details):
Twenty schools will participate. Half of the recruited schools will deliver the
intervention in Phase 1 (beginning January 2011), with the remaining schools
delivering the intervention in Phase 2 (beginning January 2012).
There will be three data collection points - baseline (at recruitment stage), and a
maximum of two follow-up points at 8-month intervals.
Phase 1:
Recruitment of participants will begin in October 2010, to give teachers time to
become acquainted with new children starting school or new to their class.
Phase 1 follow ups will be conducted June-August 2011 and March-May 2012.
The intervention children will receive the programme from January to end of May
2011, The waiting list children will receive the programme from October 2011.
Phase 2:
Phase 2 will be recruited in October 2011 and have a follow-up in June-August 2012.
Phase 2 participants will only receive one follow-up to allow time for data analysis to
be conducted, a final report to be completed and papers submitted for publication.
The intervention children will receive the programme from January to end of May
2012.The waiting list children will receive the programme from October 2012.
Data inputting, main analyses and interpretation will take place September 2012-May
2013, and will include production of final written reports, conference presentations,
and paper preparation for publication in peer reviewed journals.
Randomisation:
A pragmatic, parallel, randomised controlled trial (RCT) design will be applied, with
participants individually randomised on a 1:1 basis, intervention or waiting-list
control.
The North Wales Organisation for Randomised Trials (NWORTH), a registered
trials unit, will undertake the randomisation, maintaining true objectivity, and
ensuring that researchers (with the exception of the Principal Investigator) will be
blind to condition.
Individual (child) block randomisation within school will be applied with gender as
the stratification variable.
The Principal Investigator will forward information required by NWORTH in order
to conduct the randomisation process; this consists of school identifier, child
participant identifier plus initials, date of birth and gender, and confirmation of
parental consent. This information will be given on completion of baseline data
collection within each school (block), via a completed excel spreadsheet. NWORTH
will then perform randomisation using a specially designed computer programme to
ensure equivalent ratios of boys to girls across a 1:1 split intervention to control in
each school.
8
Building social and emotional competence in young high-risk school children
Each school will deliver the programme to six intervention children during one
school year and six waiting-list control children the following year.
Parents will be informed that their child will be chosen randomly as to whether they
receive the intervention in the first or second year of delivery (intervention or
control respectively).
Blinding:
Randomisation will be conducted by NWORTH, with results of allocation forwarded
to the Principal Investigator. The Principal Investigator will be the only member of
the research team un-blinded to allocation.
Randomisation will not be carried out until baseline data collection is complete so
will not an issue for researchers at this data collection time point.
Blinding of researchers will become difficult during follow-up visits as parents or
children may want to discuss their experiences. Participants will be respectfully
asked not to discuss the intervention with the researchers during interview or
collection of the research measures. This request will be made during initial
interview and at subsequent scheduling of appointments. In addition researchers will
re-iterate this request promptly on arrival at the home or school where data is to be
collected.
Parents, children and teachers will not be blinded and will know of child allocation to
the first (intervention) or second (waiting-list control) group, following completion
of baseline measures of all participants within a participating school (block).
Sample size and power calculation:
We seek to recruit 20 schools, 10 will deliver the programme in the first year and
10 in the second year. Forty teachers and/or classroom assistants (two from each
school) will deliver the programme. In total 240 children will participate (120
intervention, 120 control) across the twenty schools.
The programme is delivered to a maximum of six children within a group format; to
take any potential difficulties in recruitment or attrition in to account we have
conducted the power calculation based on 4 children per condition per school, i.e.
33% below our target.
Children will score within the borderline range or above in to the ’abnormal’ range
on the Teacher Strengths & Difficulties Questionnaire, Goodman, 1997 at baseline,
and we expect to find at least a 2-point difference between the intervention and
waiting list control children at follow-up based on similar studies of interventions
aimed to reduce social, emotional and behavioural difficulties (e.g. Hutchings et al.,
2007)
A factorial design with two factors of intervention group and school with 2 and 20
levels has 40 cells (treatment combinations). A total of 160 participants are required
to provide 4 participants per cell. The within-cell standard deviation is 4.0. This
design achieves 88% power when an F test is used to compare the intervention
groups at a 5% significance level.
Participants:
Participant summary:
 20 schools (10 in each phase)
 240 children, 12 in each school, 6 intervention and 6 waiting list control
9
Building social and emotional competence in young high-risk school children



240 parents (primary caretaker of child participants)
Teachers (of participant children, n = TBC following recruitment of families)
40 facilitators (teachers and classroom assistants)
A Consort diagram showing the flow of participants will be completed at the end of
the trial (see Appendix 2)
The 20 schools will be identified in conjunction with Gwynedd Education services.
The 240 high-risk children, aged 4-8 years1 will be identified as scoring above the
borderline (12-15) on the total difficulties score of the Teacher Strengths and
Difficulties Questionnaires (SDQ, Goodman, 1997). School staff will approach the
parents of highest scoring children and, if they are interested in hearing more about
the research parents will complete and sign a ‘note of interest’ form with contact
details, and forward to the research team (SAEs will be provided)
Recruitment:
The research team will recruit the families (see Home and School Visits Manual for
details of recruitment procedure). If a parent is not interested in the research, the
parents of the child with the next highest score will be approached until a full 12
children and parents are recruited in each of the 20 schools.
The parent, or person, with the primary care of the child will also participate in the
research and complete measures on child behaviour in the home (see Home and
School Visits Manual for details), N = 240. The research team will call the parent to
make an appointment for a home visit. If the parent does not show for three
subsequent pre-arranged visits contact will be terminated.
The Home and School Visits Manual contains scripts to guide the researcher in their
giving of information to parents and to children.
In addition to the participating families teachers with a participating child in their
class will complete measures on teacher stress and background; Teacher sample size
will be determined after recruitment of families. Note that teachers will possibly
change within school year or from one academic year to the next. Measures will be
collected on the child by their class teacher at each specific time point.
Randomisation will occur after eligibility has been assessed, informed consent has
been obtained, and baseline measures have been collected from parents and
children.
Forty facilitators (teachers, or teachers partnered with classroom assistants) will
deliver the intervention to the children.
Consent:
Parents will consent on behalf of their child. Consent will be sought at the initial
home visit. An information sheet will be given to the parent, or the researcher will
read the sheet if reading is an issue. The information sheet is available in Welsh and
English. The researcher will ensure that the purpose, nature, benefits, and burdens of
the research are presented and understood by the parent, allowing plenty of time
for questions. In addition the parent will have a week to decide whether to
1
The age range may need to be widened due to the small size of some prospective
schools and the possible difficulty of obtaining a large enough sample in these schools
10
Building social and emotional competence in young high-risk school children
participate if required, and will be encouraged to discuss participation with their
partner, family, and friends.
The information sheet also makes explicit that participants, having given their
consent, will be free to withdraw their child and themselves from the study at any
time without affecting future access to family or children’s services. Confidentiality
will be assured, unless the researcher has cause for concern regarding child
protection issues.
Data collectors will be fluent in Welsh and English to enable home/school visits to be
conducted in the participant’s preferred language.
Standardised measures will only be available in English if this is the language they
were validated in with no Welsh versions available.
Parents will be asked to consent to their child participating in the programme, to
participate in assessments, for school academic attainment and attendance records
to be accessed, for the child to be audio-recorded in the problem solving task, and
videotaped during programme delivery - the camera will be trained on the facilitators
but the children may be visible. The programme is recorded for supervisory
meetings and to establish level of fidelity in delivery.
Risks and benefits:
There are no obvious risks related to participation, except the small possibility that
parents may reflect on their mental state when completing the stress and depression
measures. The researchers have a leaflet with useful telephone numbers to give to
parents, and will immediately contact their line manager should an issue arise.
Benefits could be far reaching; children will attend a small group for 2 hours a week
to learn about feelings and how to articulate them, how to make and talk to friends,
and follow school rules. These factors are likely to be protective and have been
shown to increase the likelihood of doing well at school and being more sociable.
These benefits may generalise to the home environment. Home-school links will be
strengthened as schools liaise more closely with parents regarding 'homework'
during the 18 group sessions.
Children attending the sessions will have lots of fun with puppets, videos, and sharing
activities.
All participating children will receive the intervention. Children are allocated on a 1:1
ratio intervention to waiting list control, so each school will deliver the programme
twice - in the first year to intervention children and in the second year to control
children.
Inclusion Criteria:
•The index child will be rated by their teacher as within the ‘borderline’ range or
above on the screening measure – Teacher Strengths & Difficulties Questionnaire
(SDQ; Goodman, 1997)
•The child will be 4-8 years of age
•The child and parent speak Welsh and/or English
•The parent reads (or is read) the information sheet, understands the trial and
consents to:
o Their child attending the programme
o Their child being observed
o Their child completing a Wally problem solving task and being recorded
o Completing questionnaires about themselves and their child
o Their child being randomly allocated to an early or later group
11
Building social and emotional competence in young high-risk school children
o The group being filmed for supervision purposes - whilst the camera is
trained on the facilitators, their child may be in frame
o Their child to be audio-taped during the Wally problem-solving task
o Their child’s school academic and attendance records being accessed
Exclusion criteria:
•The child does not reach at least the ‘borderline’ range on the SDQ Total
Difficulties Score
•The child is the incorrect age
•The parents do not consent
•Parent and child do not speak either Welsh or English
Intervention:
The IY Therapeutic Dinosaur School Programme (Webster- Stratton, 1990) is an 18week (2 hours/week) child-training curriculum that strengthens children's
social/emotional & academic competencies. It is delivered to groups of 6 children.
It is part of the successful evidenced based IY Series, which includes parent, child and
teacher programmes. All IY programmes are manualised and require trained
facilitators.
The programme will be delivered during school time as part of the typical school
day. Teachers and/or classroom assistants will deliver the programme after
undergoing necessary training. They will receive supervision throughout intervention
delivery to ensure that the programme is delivered as it was developed.
The Therapeutic Dinosaur School Programme is a child-training curriculum that
strengthens children's social/emotional & academic competencies such as
understanding & communicating feelings, problem-solving strategies, managing anger,
& friendships.
Puppets and lots of role-play and video clips are used, to make it a very fun learning
experience.
The Dinosaur School programme sessions cover six separate programmes:
1. Making new friends and learning school rules;
2. Dina Dinosaur teaches how to do your best in school;
3. Understanding and detecting feelings;
4. Detective Wally teaches problem-solving steps (including anger management);
5. Molly Manners teaches how to be friendly;
6. Molly explains how to talk to friends.
Wally, Molly and Dina Dinosaur are puppets. Children relate better to puppets than
to therapists and are more likely to imitate their appropriate behaviour (WebsterStratton et al., 2001). Each session includes activities such as “feelings” and “let’s
suppose” games, cooperative art projects and guessing games to improve cooperation skills. Group leaders praise and reward appropriate behaviours by labelling
the behaviour and awarding “dinosaur chips” as tokens that children can spend on
small gifts such as pencils or stickers. Weekly homework activities involve the child
talking to their parent/s about what they have learned to encourage positive parentchild interaction.
For more information on this intervention, or any of the IY Programmes, see
www.incredibleyears.com
12
Building social and emotional competence in young high-risk school children
Facilitators:
The 40 facilitators will be teachers and/or classroom assistants working in the
participating schools. Many teachers and assistants will have some experience with
the IY programmes as the teachers will have received the IY classroom management
training and the Classroom Dina curriculum is delivered in all of the proposed
schools.
Facilitators’ breadth and depth of experience with the IY programmes will be
ascertained at the beginning of the trial, and all facilitators will undergo the specific
IY Therapeutic Dina training prior to delivery. Additional IY training will be given to
those facilitators who require it.
During delivery of the intervention facilitators will be required to attend supervision
by an IY trainer (the Chief Investigator) once per month during the 18 weeks, for
delivery support and to ensure implementation fidelity. At supervision facilitators will
bring video-recorded tapes (or DVDs) of their sessions for discussion and feedback.
Parental consent for trial participation will include consent to video record the
group purely for facilitator supervisory purposes.
Training in the delivery of the intervention will be organised and delivered by
Incredible Years Wales. Support to achieve implementation fidelity, to include
school-based support, will be provided by the teacher seconded to IY Cymru for the
duration of the intervention.
Measures (see Appendix 3 for detailed information):
Measures will be completed by teachers, parents, children and facilitators, and
collected by the research team. Measures include:




Measures completed by/with parent in the home (see Table 1):
o Demographics
o Child behaviour x 3
o Parent mental health x 2
Measures completed by teacher in the school (Table 2):
o Teacher background
o Teacher mental health
o Child behaviour
Measure involving the child in school (Table 3):
o Problem-solving task
o Child behaviour - by independent observation
o Friendship assessment – number of friends
Additional assessments conducted by research team in the school:
o Academic attainment tracking of child
o Attendance levels of child & their teacher at school
Additional measures (below) will assess elements relating to implementation and
participant responsiveness.

Measures relating to programme delivery completed/assessed by facilitators
in the school:
o Implementation fidelity checks – completed weekly as part of delivery
13
Building social and emotional competence in young high-risk school children
o Child Dina homework completion rates
o Attendance levels of child at Dina group
Table 1. Parent measures completed in the home, collected by the researcher
* Phase 1 only will have a second follow up.
14
Building social and emotional competence in young high-risk school children
Measure
1. Teacher Stress Inventory;
Modified version (TSI; Boyle,
Borg, Falzon, & Bagiloni, 1995)
Assesses
Teachers with a high total score on the
Teacher Stress Inventory self-rated as
experiencing stress than those with a
lower score.
Completed when
At baseline, follow
up 1 and follow up
2*
2. Teacher Demographics
Questionnaire
3. Teacher Strength and
Difficulties Questionnaire
(Goodman, 1997)
Provides information on teachers'
qualifications and experiences.
The occurrence of particular behaviours
that have been associated with conduct
problems, hyperactivity, emotional
symptoms, peer problems, and prosocial behaviour in children aged 4-16
At baseline
At baseline, follow
up 1 and follow up
2*
Table 2: Teacher measures completed in the school
* Phase 1 only will have a second follow up.
Table 3: Measures involving the child
Measure
1. Wally Problem Solving Task
(Webster-Stratton & Reid,
2001) & number of friends
2. Observations
Assesses
Children's problem solving skills or
solutions in response to hypothetical
problem situations. Researcher will ask
how many friends the child has during
introductions
Core elements of Dina Programme:
Making new friends
Detecting and understanding feelings
How to do your best in school
Problem solving steps
How to be friendly
How to talk to friends
Completed when
At baseline, follow
up 1 and follow up
2*
At baseline, follow
up 1 and follow up
2*
* Phase 1 only will have a second follow up.
Data Collection:
This is a two-phase project, with ten schools in each phase, to enable the staggering
15
Building social and emotional competence in young high-risk school children
of training, programme delivery and data collection (see Gantt Chart, Appendix 1).
Teacher, parent and children data will be collected at baseline, and at two follow-up
time points for Phase 1 children. Phase 2 children will only have one follow-up due
to time constraints.
The initial home visit will be undertaken to discuss the research, to obtain written
informed consent, and to collect parent report data at that point if the parent is able
and willing to do so.
Data will be collected by research staff blind to condition.
Parent measures will be completed during home visits by the research team. A £10
cash incentive will be paid to the parent at the end of each data collection visit on
completion of measures.
Teacher measures will be completed during school time.
Child data will be collected during school time, by prior arrangement with the
schools.
At least 20% of ‘blind’ observations will occur with two researchers to enable interrater reliability to be gauged. Research staff will be trained in observation measures
until 70% inter-rater reliability is obtained prior to data collection.
Facilitator implementation checklists and child programme attendance will be
collected weekly following programme delivery.
Teacher and child attendance levels will be collected termly.
Child academic attainment, by means of expected and achieved grades will be
collected at each time point from Gwynedd Education Central database, or
requested through individual schools.
Databases:
Various Excel and SPSS databases will be developed and maintained throughout the
trial.
Excel databases will hold personal, identifiable information and will be password
protected and stored on a secure server. Statistical databases will hold nonidentifiable data – participants will have a numerical identification number.
Excel databases include:
1.School contacts
1.School contact details
2.Facilitator names & experience
3.Dates of visits
2.Family contacts
1.Family contact details & ID
2.Child details, including SDQ score, & school
3.Visits – dates scheduled, dates occurred, cash receipt
4.Data inputted
3.Screening
1.Child name, school, SDQ score, ID
2.Parent note of interest received, consent obtained
Non-identifiable statistical data will be inputted into SPSS (a statistical software
programme) by a member of the research team. Twenty percent of inputted data
will be checked by another member of the team to assess input error rates.
Syntax files will be used to attain scores from questionnaires that include reverse
scoring, i.e. the SDQ (Goodman, 1997) and The Parenting Scale (Arnold et al., 1993).
16
Building social and emotional competence in young high-risk school children
All questionnaires will be inputted at the item level. Main summary databases will
include subscale and total scores from the measures and include demographic
information to allow for subgroup analyses.
There will be three main summary databases:
1. Parent and child data – family demographics, subscales and total scores on all
outcome measures (including teacher completed SDQs), attendance levels at
school and in the group, group homework completion rates, academic
attendance, observation ratings
2. Teacher data – background, stress levels, attendance, number of child
participants they teach
3. Facilitator data - implementation fidelity rates
Statistical analyses:
Statistical analyses, including subgroup analyses and adjusted analyses, indicating
those pre-specified and those exploratory will be conducted to include (amongst
others):
•Sample characteristics
•ANCOVA will establish differences between conditions & across time points
•Effect sizes will be calculated using Cohen’s d
•Mediator & moderator analyses will be conducted to establish whom the
intervention works best and why
•Regression analyses & correlations will assess scores on a measure
predicting, or having a relationship, with another
The primary outcome measure is the total difficulties score on the Teacher
SDQ at the 8 and 16-month post baseline follow-up visit.
Secondary outcome measures include:
For children:
 Frequency and intensity of child behavioural problems as assessed by the
Eyberg Child Behaviour Inventory across all time points
 Levels of increased social and emotional competence assessed by the Dina
Questionnaire
 Child 'Wally' Problem Solving task - number of positive solutions from
baseline to follow-up
 Frequency of observed positive interaction with peers across all time points
 Number of friends a child has - assessed across all time points by interview
 Academic attainment across all time points assessed by school records
 School attendance across all time points assessed by school records
For parents:
 Levels of parenting competencies/strategies across all time points assessed
using the Parenting Scale questionnaire
 Levels of depression across all time points assessed using the Beck
Depression Inventory
 Levels of well-being across all time points assessed using the WarwickEdinburgh Mental Well-being Scale
17
Building social and emotional competence in young high-risk school children
For teachers:
 Levels of teacher stress across all time points assessed using the Teacher
Stress Inventory
Intention To Treat (ITT):
ITT avoids the problems created by omitting dropouts and noncompliant parents,
which can negate randomisation, introduce bias, and overestimate clinical
effectiveness.
Analysis by ITT is a strategy that compares the study groups in terms of the
condition to which they were randomly allocated, irrespective of the treatment they
actually received or other trial outcomes. Regardless of protocol deviations and
participant compliance or withdrawal, analysis is performed according to the
assigned treatment group.
Two types of analyses are recommended for each research study; one on the ITT
data and one on data from participants who have remained in the study, attended at
least one intervention session and have completed measures at each time point.
A less stringent way would be to include only those participants classed as
‘completers’ (participants who attended a high percentage of sessions).
Missing data:
Missing baseline data is not an issue, as randomisation will occur after baseline
measures are completed.
Total baseline scores could be inserted into the follow-up 1 database, thereby
denoting ‘no change’. This approach is called the last observation carried forward
(LOCF. Hollis & Campbell, 1999). Another possible approach fits a straight line to
the available data to establish what might have happened had the trend continued. It
is good practice to try more than one way to ‘fill in the blanks’ to see whether
conclusions change with the different methods.
Other methods include mean scores of the outcome measures for all intervention
participants replacing missing follow-up data.
Different methods of dealing with missing data will be explored following data
collection and establishment of how many, and what type of, missing values are
achieved.
To include participants with just one data point, such as at baseline, the number of
missing data points per participant will decrease the reliability of the statistical model
used to analyse the data. The more ‘true’ data used, as opposed to estimated data,
will render the model more efficient.
Demographic data:
Characteristics of the sample (child and parent) will be analysed and differences (if
any) between the two conditions, intervention and control (and lost participants),
will be established.
Percentages of demographic variables (e.g., parental status, ethnicity, education,
income levels) will aid the analysis of for whom the intervention works best and why.
18
Building social and emotional competence in young high-risk school children
Analysis of behaviour outcomes:
A measurable improvement on the primary outcome measure (teacher & parent
SDQ) denoting improved social and emotional competence in school and home.
Improvements will be between baseline and follow-up data for intervention children
and as compared to control group children.
An initial analysis of the effects of baseline value, area, treatment and their
interaction will be conducted. The difference between the intervention and control
conditions on follow-up scores will be based on a mixed model analysis of
covariance (ANCOVA), run on SPSS, of the response taking account of the random
school effect, fixed treatment effect, gender, age/school year, and the baseline
response value. Any similarities within the intervention group, due to participating n
a group format intervention, will be controlled for in the model.
Effect sizes will be calculated using Cohen’s (1988) guidelines. The American
Psychological Association encourages the reporting of effect sizes in this type of
study to enhance result replicability and comparability, and to indicate practical
significance. Cohen’s effect size is a standardised mean difference statistic whereby
the difference between the two group means is divided by the pooled standard
deviation. Thus an effect size of .3 indicates that one-third standard deviation
separates the two means. If the two groups were equal at baseline this figure
represents a minimal clinically useful improvement by the intervention group at
follow-up. As a rule of thumb Cohen suggests that an effect size of .2 is a small
effect, .5 is medium and .8 is a large effect.
4. Ethical issues
The project proposal has received ethical approval form the School of Psychology
Research Ethics Committee, Bangor University (1.03.2010). Gwynedd Education
service does not have an ethics committee, but the project will be monitored by a
high level official (Orina Pritchard, Senior Manager, Gwynedd Education who has also
submitted a letter of support).
NHS ethics has been granted with a favourable opinion (17.09.2010) by the North
Wales Research Ethics Review Committee.
In order to ensure information remains confidential, participants will be assigned an
identification number to be used on all evaluation measures and schedules. The
database linking participants to their identification number will be stored on
protected computer files, only accessible to members of the evaluation team. The
evaluation team and the implementation team are entirely different people.
Data held on research computers will be password protected. All procedures will
abide by the Data Protection Act (1998). Data collectors will be CRB checked, will
abide by the University’s and Charity’s lone worker policy, and will be trained in
Child Protection issues and how to deal with any other sensitive issues arising in the
home environment.
Prior to requesting consent an information sheet will be given to each parent. The
leaflet explains the purpose of the study, why the family was chosen, and what will
happen should they decide to take part. The parent will be reassured that all
19
Building social and emotional competence in young high-risk school children
information provided by the family will remain confidential. The information sheet
also makes it explicit that participants, having given their consent, will be free to
withdraw their child or themselves from the study at any time without affecting
future access to family or children’s services.
If data collectors perceive a risk to the child whilst on a home visit they will follow
University and Charity child protection procedures and immediately contact their
line manager who will seek advice from the relevant bodies. This potential break of
confidentiality will be clearly noted on the information sheet.
Home and school visits will be carried out in the participant’s preferred language
(Welsh or English).
Standarised measures will only be available in English if this is the language they were
validated in, and no Welsh versions exist.
Confidentiality and anonymity:
Anonymised computer-based data will be uploaded to a secure server at regular
intervals.
Personal data such as addresses and names will be kept on the main office computer
and uploaded to a secure server at regular intervals, and will be password protected
at many levels. When data is transferred electronically it will be password protected
and only minimal data sent on a need to know basis, e.g. to inform facilitators of
children allocated to intervention just initials and DOB will be forwarded.
Hard copies of data will be kept in locked cabinets, within the locked office in the
University where the Principal Investigator resides.
Identifiable data will be kept separately from anonymised data. Data will be
anonymised by allocation of an identification number.
Identifiable data will be shredded within four years of trial completion. Video
recording devices will be used during the programme sessions. The videos will not
be focused on the children, although they may occasionally be in shot. This is
explained to parents and is included on the consent form. The videos will be used
purely to offer feedback to facilitators on programme delivery and to assess
implementation fidelity, and this will be made clear to parents at the initial
information/recruitment visit.
Audio recordings will be made of the children carrying out the Wally problemsolving task, an identifier will be used to identify the child on the tape. No names will
be used.
Any quotes used by participants relaying their experiences will be anonymised.
Safeguarding:
The Principal Investigator will train the researchers on the Charity’s safeguarding
procedures specific to this project, and on Bangor University’s general lone-worker
guidelines.
Any issues or incidents arising during data collection will be swiftly reported to the
Principal Investigator who shall assess the situation and act accordingly. The Director
of IY Cymru is a retired Consultant Clinical Psychologist with an honorary contract
with the Betsi Cadwaladr University NHS Trust and will be available for consultation
on any mental health issues arising.
20
Building social and emotional competence in young high-risk school children
During parent interview some questions may invoke strong/hidden feelings due to
their sensitive nature. Researchers will leave support telephone numbers with
parents if required, offering signposting to relevant services or helplines.
The Principal Investigator will also conduct in-depth training on all data collection
methods and ethical obligations, including data protection.
In addition researchers will attend external ‘Good Clinical Practice’ training and
Child Protection training from relevant bodies.
5. Project Management
A steering group has been established to include IY Cymru Trustees, Professor
Hutchings (the Director) and other main stakeholder representatives from partners
Gwynedd Education Service and Bangor University. The research team staff will
attend and the IY Cymru administrator will service the committee.
A WAG representative will have observer status, ensuring that they receive minutes
of meetings and are welcome to attend meetings. The steering group will meet on a
quarterly basis.
The Trustees will have the ultimate responsibility for ensuring that this project is run
in a satisfactory way; they have academic, professional, and service user backgrounds.
A statistician from Bangor University (Mr Chris Whitaker), an independent advisor
from Oxford University (Professor Frances Gardner) and a research nurse from
NISCHR are also members of the committee.
Appendix 4 presents all project partners, collaborators and staff in an Organisational
Chart.
6. Communication and Dissemination plan
The findings could contribute solutions to a major problem facing our society and
there is significant media interest in both the problem and potential solutions. We
will disseminate via published articles, conference presentations, responses to media
enquiries, press releases, contributions to the Incredible Years Centre Annual
Conference and Newsletter.
Our target audiences include policy makers, research partners, voluntary and
statutory organisations throughout Wales, teachers and parents.
Policy Makers:
The Welsh Assembly Government (WAG) will have access through attendance at
quarterly steering group meetings and at the Annual conference organised by the
Incredible Years Centre. Representatives of WAG always attend these conferences,
which have, for many years, been officially opened by the Assembly Government’s
Minister for Children.
Partners:
Communication will be through quarterly steering group meetings, and as and when
needed throughout the trial to ensure effective collaboration. Both partners will use
their own existing dissemination strategies via their public relations Departments.
21
Building social and emotional competence in young high-risk school children
Participants:
Participants are the teachers, parents and children. Communication with teachers
will be ongoing via school visits by the teacher seconded to the research team, and
the six Bangor based supervision days in each year that the intervention is being
implemented. A summary of outcomes for the sample as a whole will be sent to all
parents and teachers at the end of the project.
Teachers throughout Wales:
Attendance at teacher conferences, articles submitted to educational and other
relevant journals, circulation of summary of the project in first year to all Local
Authorities in Wales and of outcomes, on project completion.
Parents:
Information dissemination to parents will be via a summary sheet of the trial as a
whole, and through national and local media. The IY Wales Centre produces regular
press releases in conjunction with Bangor University. The Director of Incredible
Years Cymru, Professor Hutchings, regularly speaks on the radio and television and
is extensively quoted in national and local newspapers.
Funders:
Regular reports will be submitted to the Big Lottery. Please see Appendix 5 for a
reporting schedule.
22
Building social and emotional competence in young high-risk school children
7. References
Arnold, D., O'Leary, S. G., Wolff, L., & Acker, M. M. (1993). The Parenting Scale: A
measure of dysfunctional parenting in discipline situations.Psychological
Assessment, 5, 137–144.
Attride-Stirling, J., Davies, H., Day, C., & Sclare, L. (2000). Someone to talk who’ll
listen: Addressing the psychosocial needs of children and families.
Journal of Community Applied Social Psychology, 11, 179-191.
doi:10.1002/casp.613.
Baker-Hanningham, H., Walker, S., Powell, C., & Gardner, J, M. (2009). A Pilot Study
of the Incredible Years Teacher Training Programme and a Cirriculum
Unit on Social and Emotional Skills in Community Pre-Schools in
Jamaica. Child: Care, Health and Development, 35(5), 632-642.
Beck, A. T., Ward, C. M., Mendelson, M., Mock, J. E., & Erbaugh, J. K. (1961). An
inventory for measuring depression. Archives of General Psychiatry, 4,
561-571.
Bloomquistm M. L., Schnell, S. V. (2002). Helping children with Aggression and Conduct
Problems: Best Practices for Intervention. New York: Guilford Press.
Boyle, G. J., Borg, M. G., Falzon, J. M. & Baglioni, A. J. (1995). A structural model of
the dimensions of teacher stress. British Journal of Educational
Psychology, 65, 49–67.
Brewer, M., Muriel, A., Phillips, D., & Sibieta, L. (2009). Poverty and Inequality in the
UK: 2009. Institute for Fiscal Studies, May, 2009. Pattersons,
Tunbridge Wells.
Bywater, T., Hutchings, J., Daley, D., Whitaker, C., Yeo, S.T., Jones, K., Eames, C., &
Tudor Edwards, R. (2009). Longg Term Effectiveness of a Parenting
Intervention in Sure Start Services in Wales for Children At Risk of
Developing Conduct Disorder. British Journal of Psychiatry, 195.
Doi:10.1192/bjp.bp.108.056531
Cohen J. Statistical Power for the Behavioural Sciences. (1988). Hillsdale, NJ: Erlbaum.
Edwards, R. T., O Ceilleachair, A., Bywater, T., Hughes, D. A., & Hutchings, J. (2007).
Parenting Programme for Parents of Children at Risk of Developing
Conduct Disorder: Cost Effective Analysis. British Medical Journal, 345,
682-688.
Eyberg, S. M. (1980). Eyberg Child Behavior Inventory. Journal of Clinical Child
Psychology, 9, 27.
Eyberg, S.M. & Ross, A.W. (1978). Assessment of child behavior problems: The
validation of a new inventory. Journal of Clinical Child Psychology,
7(2), 113-116.
Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A research
note. Journal of Child Psychology, Psychiatry, and Allied Disciplines, 38,
581-586.
Hollis, S. & Campbell, F. (1999). What is meant by intention to treat analysis? Survey
of published randomised controlled trials. British Medical Journal, 319: 670-74.
Hutchings, J., Appleton, P., Smith, M., Lane, E., & Nash, S. (2002). Evaluation of two
treatments for children with severe behaviour problems: Child
behaviour and maternal mental health outcomes. Behavioural and
Cognitive Psychology, 30, 279–295.
23
Building social and emotional competence in young high-risk school children
Hutchings, J., Lane, E., Owen, R. E., Gwyn, R. (2004). The Introduction of the
Webster-Stratton Classroom Dinosaur School Programme in
Gwynedd, North Wales: A Pilot Study. Education and Child Psychology,
22(4), 4-15.
Hutchings, J., Bywater, T., Daley, D., Gardner, F., Whitaker, C., Jones, K., et al.
(2007). Parenting Intervention in Sure Start Services for Children at
Risk of Developing Conduct Disorder: Pragmatic Randomised
Controlled Trial. British Medical Journal, 334, 678-682.
Hutchings, J., Bywater, T., Daley, D., & Lane, E. (2007). A Pilot Study of the WebsterStratton Incredible Years Therapeutic Dinosaur School Programme.
Clinical Psychology Forum, 170, 21-24.
Kupersmidt, J. B., Bryant, D., Willoughby, M. (2000). Prevalence of Aggressive
Behaviours Among Preschoolers in Head Start and Community Child
Care Programs. Behavioural Disorders, 26, 42-52.
Larsson, B., Fossum, S., Clifford, G., Drugli, M., Handegard, B., & Morch, W. (2008).
Treatment of Oppositional Defiant and Conduct Problems in Young
Norwegian Children: Results of a Randomized Controlled Trial.
European Child Adolescent Psychiatry, 18, 42-52.
Mayall, B. (1994). Children’s Childhoods: Observed and Experienced. London,
Washington DC, Falmer Press. Chapter 7, 114-127.
Office for National Statistics (1999). The mental health of children and adolescents in
Great Britain summary report. London: Office for National Statistics.
Taylor, T. K., & Biglan, A. (1998). Behavioural Family Interventions for Improving
Child Rearing: A Review of the Literature for Clinicians and Policy
Makers. Clinical Child an Family Psychology Review, 1(1), 41-60.
Tennant, R., Joseph, S., & Stewart-Brown, S. (2007). The Affectometer 2: a measure
of positive mental health in UK populations. Qual Life Res, 16(4):687695
Ward, L. (2009). Some Children are so deprived it can be incredible hard to
counteract the impact [Media Release]. Retrieved from
http://www.atl.org.uk/media-office/media-archive/Some-children-sodeprived -it-can-be-incredibly-hard-for-schools-to-counteract-theimpact.asp.
Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) (2006). © NHS Health
Scotland, University of Warwick and University of Edinburgh.
Webster-Stratton, C. (2003). Aggression in Young Children Perspetive: Services
Proven to be Effective in Reducing Aggression. Retrieved from
http://ww.incredibleyears.com/research/article -aggression-in-youngchildren-perspective.pdf
Webster-Stratton, C. (1998). Preventing conduct problems in Head Start children:
Strengthening parenting competencies. Journal of Consulting and Clinical
Psychology, 66(5), 715–730.
Webster-Stratton, C., & Hammond, M. (1990). Predictors of treatment outcome in
parent training for families with conduct-problem children. Behavior
Therapy 21:319-337.
Webster-Stratton, C., & Hammond, M. (1997). Treating Children with Early-Onset
Conduct Problems: A Comparison of Child and Parent Training
Interventions. Journal of Consulting and Clinical Psychology, 65(1), 93-109.
Webster-Stratton, C., & Reid, M.J. (2004). Strengthening social and emotional
competence in young children – the foundation for early school
24
Building social and emotional competence in young high-risk school children
readiness and success: Incredible Years Classroom Social Skills and
Problem-Solving Curriculum. Journal of Infants and Young Children, 17,
196–113.
Webster-Stratton, C., Reid, M. J., & Hammond, M. (2001). Social Skills and Problems
Solving Training for Children with Early-Onset Conduct Problems:
Who Benefits? Journal of Child Psychology and Psychiatry, 42(7), 943-952.
Webster-Stratton, C. Taylor, T. (2001). Nipping Early Risk Factors in the Bud:
Preventing Substance Abuse, Delinquency, and Violence in
Adolescence Through Interventions Targeted at Young Children (0-8
Years). Prevention Science, 2(3), 165-192.
25
Building social and emotional competence in young high-risk school children
8. Appendices
Appendix 1: Gantt Chart
26
Building social and emotional competence in young high-risk school children
Appendix 2: Participant Flow Chart
27
Building social and emotional competence in young high-risk school children
Appendix 3: Details on measures
Measures: Teacher, Parent & Child
As part of the study we will be required to collect measures for the child, parent and
teachers at baseline, follow-up 1 and follow-up 2 (phase 1 only) at approximately 8
months apart.
1. Teacher measures
Modified version of the Teacher Stress Inventory (TSI; Boyle, Borg,
Falzon, & Baglioni, 1995).
The Teacher Stress Inventory (TSI) is a self-administered 20-item inventory
established from the work of Kyriacou and Sutcliffe (1978b), which is selfadministered by teachers to examine the effect of factors such as difficult classes,
impolite pupils and maintaining class discipline on the their stress levels. The teacher
is required to rate factors on a 5-point scale ranging in severity from 0 (No stress)
to 4 (Extreme stress).
For the purpose of the current evaluation 5 from the 10 factors found in the teacher
stress inventory (Fimian 1984,1985,1988) were used:
Time management
Work-related stressors,
Discipline and motivation
Professional stress and professional investment.
Rationale for use:
Pupil misbehaviour has been associated with teachers’ job stress (Borg, 1991) and
resignation (Macdonald 1999). The TSI is used in the current evaluation to examine
any differences in teachers’ stress levels between baseline and follow-ups.
Administration:
The inventory is a self-administered teacher report measure, which takes
approximately 10 - 15 minutes to complete.
Scoring:
Each item receives a score ranging from 0 to 4, where 4 represents the most
extreme response. Teachers’ perceived total stress is measured by summing the
scores for all items. Comparing scores across each item will also allow for defining
which events are most/least stressful.
Interpretation:
Teachers with a high total score on the TSI self-rated as experiencing stress than
those with a lower score.
Reliability & Validity
Fimian & Fastenau (1990) conducted a study on aggregate TSI data (N= 3401).
Cronbach alpha coefficients for the five factors of interests here were; time
28
Building social and emotional competence in young high-risk school children
management: 0.83, work-related stressors: 0.80, discipline and motivation: 0.86,
professional stress: 0.82, and professional investment: 0.75.
Factor analysis was conducted by Fimian & Fastenau (1990) on the TSI. TSI was
developed and revised over 10 years. Ninety-two experts in the field examined the
tool’s content validity.
Teacher Demographics Questionnaire
For the purpose of this study a demographic questionnaire for teachers was
developed to obtain information on teachers’ qualifications and teaching experiences.
The questionnaire covers background aspects such as teacher’s name; gender and
date of birth; qualification aspects such as subject of first degree and teaching
experience, for example; how many years spent in the teaching profession; how
many schools they have taught in; and if they have completed any other professional
training.
Rationale for use:
Research has been inconclusive in finding a link between number of teaching years
and positive child outcomes. Some studies have found teaching experience to be
predictive of positive outcome (Aydin & Hoy, 2005; Piantau et al, 2005) and others
report that more years of teaching experience can lead to less effective teaching
(Ghaith & Yaghi, 1997). Studies have also found teachers’ age to influence
acceptance of new interventions. (Kallestad & Olweus, 2003; Jones, 2006).
Therefore the teacher demographics questionnaire in this study will allow for
examining any teacher characteristics factor that may influence child behaviour
outcomes.
Administration
The Teacher’s Demographics Questionnaire is a self-administered report. The
questionnaire takes approximately 10 minutes to complete.
Teacher Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997)
This 25-item inventory (containing five subscales) was designed as a behavioural
screening measure to assess the occurrence of particular behaviours that have been
associated with conduct problems, hyperactivity, emotional symptoms, peer
problems, and pro-social behaviour in children aged 4-16. The respondent (parent or
teacher) is asked to rate how true of the index child a particular behaviour is, using a
3-point scale ranging from 0 (not true) to 2 (certainly true). A sample item would be:
Considerate of other people’s feelings.
In addition to the 25 items on psychological attributes, an impact supplement is also
available. This supplement asks whether the respondent thinks the index child has a
problem, and if so, asks further questions about chronicity, distress, social
impairment, and burden to others. Teachers are required to answer questions on
how the index child’s behaviour affects the classroom and interactions with peers.
29
Building social and emotional competence in young high-risk school children
The SDQ has been used as a screening measure (Goodman, Ford, Simmons,
Gatward, & Meltzer, 2000), as part of a clinical assessment (Goodman, Renfrew, &
Mullick, 2000), and as a measure of treatment outcome (Garralda, Yates, &
Higginson, 2000).
Rationale for use:
The SDQ is reported to perform at least as well as its longer-standing counterparts,
the CBCL (Achenbach & Edelbrock, 1986) and the Rutter questionnaires (Rutter,
Tizard, & Whitmore) correlating highly with these scales (Goodman, 1997;
Goodman & Scott, 1999). However, the SDQ has some advantage over these other
measures. First, the SDQ represents a brief measure taking only 5 minutes to
complete. Second it has been developed and standardised using a British sample.
Third, in comparison to the Rutter questionnaires, the SDQ focuses on strengths as
well as difficulties, and gives better coverage of inattention, peer relationships and
pro-social behaviour (Goodman, 1997). Fourth, as judged against a semi-structured
interview, the SDQ was significantly better than the CBCL at detecting inattentive
and hyperactivity, and at least as good at detecting internalising and externalising
problems (Goodman & Scott, 1999).
Administration:
Respondents are asked to provide answers based on the child’s behaviour over the
last six months. For young children the inventory is a self-administered parent (or
teacher) report measure and takes approximately 5 minutes to complete.
Scoring:
Scoring may be done by hand by summing the items within each of the five subscales
(5 items per subscale, minimum score = 0, maximum score = 10). A total Difficulties
score is calculated by summing the scores from all scales except the pro-social scale
(minimum score = 0, maximum score = 40). For more detail on scoring see the
SDQ website (www.sdqinfo.com) where a syntax file for SPSS is also available.
When using a version of the SDQ that includes an “impact supplement”, the items
on overall distress and social impairment can be summed to generate an impact
score that ranges from 0-10 for the parent-completed version and 0-6 for the
teacher-completed version. Responses to the questions on chronicity and burden to
others are not included in the impact score. If the answer is “no” to the first
question on the impact supplement, i.e., when the parent does not perceive the child
as having any emotional or behavioural difficulties, they do not proceed to the
subsequent questions and automatically receive a score of zero. Total impact scores
can be classified as ‘abnormal’ (a score of 2 or more), ‘borderline’ (a score of 1), or
‘normal’ (a score of zero).
Normative information:
Data from a sample of 4801 yielded the following means for total scores and subscales: Total score = 6.7 (SD = 5.9); Emotional Symptoms = 1.5 (SD = 1.9); Conduct
Problem = 0.9 (SD = 1.6); Hyperactivity = 3.0 (SD = 2.8); Peer Problem = 7.3 (SD =
2.4); Impact score = 0.4 (SD = 0.9).
30
Building social and emotional competence in young high-risk school children
Interpretation:
SDQ symptom scores can be used as continuous variables or classified as within
normal range, borderline and abnormal. As a rough guide to identifying “cases” with
mental health disorders, Table 1 gives details of a guide that may be used for
interpretation of scores, for more information see the SDQ website
(www.sdqinfo.com).
Reliability & Validity:
The scale has demonstrated good stability, whether judged by internal consistency
(mean Cronbach's alpha: 0.73), cross-informant correlation (mean: 0.34), or re-test
stability after 4-6 months (mean: 0.62) (Goodman, 2001). The SDQ has good
convergent validity, showing significant correlation with long-standing measures such
as the CBCL (r = .87). In terms of discriminative validity, high SDQ scores have been
associated with a strong increase in psychiatric risk (Goodman, 2001).
Table 1: Guide to Assist Interpretation of SDQ
Teacher completed
Total Difficulties score
Emotional Symptoms score
Conduct Problems score
Hyperactivity score
Peer Problems score
Pro-social score
Normal
Borderline
Abnormal
0-11
0-4
0-2
0-5
0-3
6-10
12-15
5
3
6
4
5
16-40
6-10
4-10
7-10
5-10
0-4
Reliability & Validity:
The scale has demonstrated good stability, whether judged by internal consistency
(mean Cronbach's alpha: 0.73), cross-informant correlation (mean: 0.34), or re-test
stability after 4-6 months (mean: 0.62) (Goodman, 2001). The SDQ has good
convergent validity, showing significant correlation with long-standing measures such
as the CBCL (r = .87). In terms of discriminative validity, high SDQ scores have been
associated with a strong increase in psychiatric risk (Goodman, 2001).
2. Parent measures
Parent Demographics Questionnaire
The Demographics Questionnaire attains basic socio-demographic and general health
data on family members. The interview is carried out with the primary care taker
and covers aspects of the child’s health and development. The PDHQ also includes
questions about other residents in the child’s home, quality of relationship between
parents where applicable, quality of housing, and primary carer's education.
For the purposes of the current evaluation one item pertaining to drug use and one
pertaining to criminal activity are also added. The items are worded such that they
refer to other members of the family; however, if the parent is perceived to be open
31
Building social and emotional competence in young high-risk school children
to this line of questioning then the researcher will probe further to ascertain the
situation for the parent also.
Socio-economic disadvantage (SED 6; Hutchings, 1996):
Data for the SED 6 is derived from answers provided on the Demographics
Questionnaire. The SED 6 is designed to attain basic data concerning family socioeconomic status. Six socio-economic risk factors are measured: employment status,
marital status, number of children, maternal education, housing, area of residence
(high/low crime); these were selected based on the findings of Dumas and Wahler
(1983) and Rutter and Quinton (1977).
Rationale for use:
The literature suggests that the risk of a child developing conduct problems is
increased by the prevalence of a variety of family factors (Farrington, 1995; WebsterStratton, 1999), for example parental substance abuse (Patterson et al, 1989). The
inclusion of this measure serves two important functions. First, to provide data for
attrition analysis and equivalency of intervention and control groups. Second, to
establish rapport with the parent, and, important in terms of assessment contiguity,
for the parent to have the opportunity to express matters concerning their child
prior to being asked to complete the subsequent questionnaires.
Socio-economic disadvantage has been identified as an associated risk factor for the
development of child behavioural problems (Farrington, 1995; Webster-Stratton,
1999).
This measure is included to provide data of equivalency of intervention and control
groups.
Administration:
The Demographics Questionnaire represents a semi-structured interview
administered by the researcher, completed by the mother/primary caregiver. It takes
5-10 minutes to complete.
Scoring SED 6:
The six SED 6 factors are coded as follows:






Employment status of primary provider: employed = 0, dependent on
benefits = 1
Marital status: married/cohabiting = 0, single parent = 1
Number of children: small family size = 0, large family size = 1 (based on the
findings from Brown and Harris (1978), three or more children represent
large family size).
Maternal education: education up to sixteen = 1, education beyond 16 = 0
Housing circumstances: poor quality/overcrowded/insecure = 1, this rating is
made on the basis of responses given in the interview and the interviewers
observations.
Area of residence: high crime = 1, low crime = 0
32
Building social and emotional competence in young high-risk school children
Each of the six items is given a 0 or 1 score and this is summed into an index of
socio-economic disadvantage for each participant. Items can also be scored across
the sample for a detailed summary.
Parent Follow-up Demographics Questionnaire
Similar to The Parent Demographics Questionnaire, the Parent Follow-up
Questionnaire attains basic socio-demographic and general health data on family
members. The interview is carried out with the primary care taker and questions
cover aspects of child development; employment status; housing circumstances and
relationships.
Administration:
The Demographics Questionnaire represents a semi-structured interview
administered by the researcher, completed by the mother/primary caregiver. It takes
2-5 minutes to complete.
Rationale for use:
The literature suggests that the risk of a child developing conduct problems is
increased by the prevalence of a variety of family factors (Farrington, 1995; WebsterStratton, 1999). The measure was developed to probe for any differences in child
development; income; housing; and relationships since baseline measurement.
Wally Problem Solving Task (Webster-Stratton & Reid, 2001)
The Wally Problem Solving test (Webster-Stratton, 1990) measures children’s
problem solving skills or solutions in response to hypothetical problem situations.
Summary scores include the number of different positive and negative strategies that
children generate in order to solve the problem. The Wally was derived from Spivak
ad Shure’s Preschool Problem Solving Test (Spivak & Shure, 1985) and Rubin and
Krasnor’s Child Social Problem-Solving Test (Rubin & Krasnor, 1986; WebsterStratton; Reid & Stoolmiller, 2008)
The Wally Problem Solving Task is researcher administered, where the researcher
presents 13 illustrated problem situations to the to assess the child’s ability to
problem-solve the scenario. Responses are rated according to their content using
pre-determined response codes. An example problem item includes: Suppose you ask
another friend to play with you and she refuses. What would you do? Responses to
situations are rated as positive (asking for a reason), negative (claiming for self),
neutral (ignore), pro-social (help to repair) or agonistic (aggressive) responses.
Rationale for use
The Wally Problem Solving Task is a useful tool when examining children with
conduct disorder, as it is sensitive to social skills and problem solving treatment
effects (Webster-Stratton & Hammond, 1997).
Administration
Once the child is comfortable in the researcher’s company the researcher informs
the child that he/she must take on the role of “detective” to see how good they are
at problem solving. The researcher reads the situation on the back of each problem
33
Building social and emotional competence in young high-risk school children
situation and asks the child what they would do. The researcher records the child’s
answer on a coding sheet.
Scoring
There are 59 possible child response codes, which are either pro social or agonistic
responses. The coding sheet allows for a maximum of 6 codes per question. The
responses are scored depending on the type of solutions given (positive or negative).
Two summary scores are produced from the questions; (1) the number of different
positive responses given and (2) the ratio of positive responses to negative
responses.
Interpretation
It has been shown that children are liked more by other children, are more able to
play constructively and are more cooperative at school and at home when they have
the ability to apply relevant problem solving approaches (Webster-Stratton & Reid,
2004).
Reliability & Validity
Construct validity was established by correlating the Wally total pro social score and
Rubin total positive strategies (r = .60) and also Wally negative scores and Rubin
negative strategies (r = .50).
Inter-rater reliability for number of different positive strategies was ICC .93 and for
different negative strategies was ICC .71
Parent Strength and Difficulties Questionnaire (SDQ; Goodman, 1997)
This 25-item inventory containing five subscales (see table 1) was designed as a
behavioural screening measure to assess the occurrence of particular behaviours
that have been associated with conduct problems, hyperactivity, emotional
symptoms, peer problems, and pro-social behaviour in children aged 4-16. For
children aged 3 (and 4) a slightly modified version has also been developed consisting
of 22 identical items to the original, but with the item on reflectiveness softened and
the 2 items on antisocial behaviour replaced with items on oppositionality. The
respondent (parent or teacher) is asked to rate how true of the index child a
particular behaviour is, using a 3-point scale ranging from 0 (not true) to 2 (certainly
true). A sample item would be: Considerate of other people’s feelings.
In addition to the 25 items on psychological attributes, an impact supplement is also
available. This supplement asks whether the respondent thinks the index child has a
problem, and if so, asks further questions about chronicity, distress, social
impairment, and burden to others. Parents are required to answer questions about
how the child’s behaviour interferes with home life, leisure activities, friendships and
classroom learning.
The SDQ has been used as a screening measure (Goodman, Ford, Simmons,
Gatward, & Meltzer, 2000), as part of a clinical assessment (Goodman, Renfrew, &
Mullick, 2000), and as a measure of treatment outcome (Garralda, Yates, &
Higginson, 2000).
34
Building social and emotional competence in young high-risk school children
Rationale for use:
The SDQ is reported to perform at least as well as its longer-standing counterparts,
the CBCL (Achenbach & Edelbrock, 1986) and the Rutter questionnaires (Rutter,
Tizard, & Whitmore) correlating highly with these scales (Goodman, 1997;
Goodman & Scott, 1999). However, the SDQ has some advantage over these other
measures. First, the SDQ represents a brief measure taking only 5 minutes to
complete. Second it has been developed and standardised using a British sample.
Third, in comparison to the Rutter questionnaires, the SDQ focuses on strengths as
well as difficulties, and gives better coverage of inattention, peer relationships and
pro-social behaviour (Goodman, 1997). Fourth, as judged against a semi-structured
interview, the SDQ was significantly better than the CBCL at detecting inattentivity
and hyperactivity, and at least as good at detecting internalising and externalising
problems (Goodman & Scott, 1999).
Administration:
Respondents are asked to provide answers based on the child’s behaviour over the
last six months. For young children the inventory is a self-administered parent (or
teacher) report measure and takes approximately 5 minutes to complete.
Scoring:
Scoring may be done by hand through summing the items within each of the five
subscales (5 items per subscale, minimum score = 0, maximum score = 10). A total
Difficulties score is calculated by summing the scores from all scales except the prosocial scale (minimum score = 0, maximum score = 40). For more detail on scoring
see the SDQ website (www.sdqinfo.com) where a syntax file for SPSS is also
available.
When using a version of the SDQ that includes an “impact supplement”, the items
on overall distress and social impairment can be summed to generate an impact
score that ranges from 0-10 for the parent-completed version and 0-6 for the
teacher-completed version. Responses to the questions on chronicity and burden to
others are not included in the impact score. If the answer is “no” to the first
question on the impact supplement, i.e., when the parent does not perceive the child
as having any emotional or behavioural difficulties, they do not proceed to the
subsequent questions and automatically receive a score of zero. Total impact scores
can be classified as ‘abnormal’ (a score of 2 or more), ‘borderline’ (a score of 1), or
‘normal’ (a score of zero).
Normative information
To date there is no normative information available for children aged 3/4. However,
data from a sample of 5855 children aged 5-10 years yielded the following means for
total scores and sub-scales: Total score = 8.6 (SD = 5.7); Emotional Symptoms = 1.9
(SD = 2.0); Conduct Problem = 1.6 (SD = 1.7); Hyperactivity = 3.6 (SD = 2.7); Peer
Problem = 1.4 (SD = 1.7); Pro-social = 8.6 (SD = 1.6); Impact score = 0.3 (SD = 1.1).
Interpretation:
SDQ symptom scores can be used as continuous variables or classified as within
normal range, borderline and abnormal. As a rough guide to identifying “cases” with
mental health disorders, Table 1 gives details of a guide that may be used for
interpretation of scores, for more information see the SDQ website
(www.sdqinfo.com).
35
Building social and emotional competence in young high-risk school children
Table 2. Guide to Assist Interpretation of SDQ
Parent completed
Total Difficulties score
Emotional Symptoms score
Conduct Problems score
Hyperactivity score
Peer Problems score
Pro-social score
Normal
Borderline
Abnormal
0-13
0-3
0-2
0-5
0-2
6-10
14-16
4
3
6
3
5
17-40
5-10
4-10
7-10
4-10
0-4
Reliability & Validity:
The scale has demonstrated good stability, whether judged by internal consistency
(mean Cronbach's alpha: 0.73), cross-informant correlation (mean: 0.34), or re-test
stability after 4-6 months (mean: 0.62) (Goodman, 2001). The SDQ has good
convergent validity, showing significant correlation with long-standing measures such
as the CBCL (r = .87). In terms of discriminative validity, high SDQ scores have been
associated with a strong increase in psychiatric risk (Goodman, 2001).
Eyberg Child Behavior Inventory (ECBI; Eyberg & Ross, 1978; Eyberg,
1980)
This is a 36-item inventory designed to be completed by the parent for the
assessment of problem behaviours occurring in children from age 2-16 years. An
example item of problem behaviour would be: Has temper tantrums. Each behaviour
is rated on two scales: a 7-point Intensity scale that measures how often the
behaviour is perceived to occur, ranging in response intensity from 1 (Never) to 7
(Always); and a Yes-No Problem scale that identifies whether the behaviour is
currently seen as a problem for the parent.
The ECBI can be used:
 As a screening measure in the clinical identification of children for the
diagnosis and treatment of externalising behaviour problems.
 As a selection measure for the identification of “high risk” children for
delinquency prevention programmes.
 As a measure of treatment outcome.
Rationale for use:
The ECBI has been used extensively within the field of parent training intervention.
Following Webster-Stratton (1998b) the ECBI is used in the current evaluation as
both a selection measure to identify participants for entry into the study, and as an
outcome measure to evaluate the intervention.
Administration:
The inventory is a self-administered parent report measure and takes approximately
10 minutes to complete.
36
Building social and emotional competence in young high-risk school children
Scoring:
A total score for each scale is used. For the Intensity scale, circled responses are
totalled to give the raw score (minimum score = 36, maximum = 252). Where
missed responses occur count as 1 (Never) and sum as before. If four or more items
are missed the scale becomes invalid and cannot be scored. For the Problem scale
circled Yes responses are totalled to give the raw score (minimum score = 0,
maximum score = 36). Where missed responses occur count as a No response and
sum as before. When there are four or more items missed the scale becomes invalid
and cannot be scored.
Normative information:
Original standardisation of the ECBI (Robinson, Eyberg, & Ross, 1980), with a sample
of 512 children (aged 2 –12 years), yielded norms of 103.8 (S.D. = 34.6) for Intensity
and 6.9 (S.D. = 7.8) for Problem scales. Sub-sample analysis of 57 children identified
as having conduct problems yielded mean scores of 137.2 (S.D. = 38.8) for Intensity
and 15.0 (S.D. = 9.6) for Problem scales.
Interpretation:
Both scales of the ECBI are continuous such that higher scores on the scale indicate
a greater level of conduct-disordered behaviour and greater impact on the parent.
Based on the 1980 normative data clinical cut-off scores of 127 or more for Intensity
and 11 or more for Problem scales are suggested.
Reliability & Validity:
The scale demonstrates good stability and homogeneity, with reliability coefficients
from .86 (test-retest) to .98 (internal consistency) (Robinson et al., 1980). The ECBI
has shown good convergent validity, with ECBI scores being significantly correlated
with scores on the Child Behaviour Check List (CBLC; Achenbach & Edelbrock,
1986) and the Parenting Stress Index (PSI; Abidin, 1990). The ECBI has been shown
to discriminate well between children with and without conduct problems (Eyberg &
Ross, 1978; Baden & Howe, 1992). In addition the ECBI has proven a sensitive
measure of treatment change in both clinically referred children (e.g. WebsterStratton & Hammond, 1997) and children expressing behaviour problems within
normal limits (e.g. Brestan, Eyberg, Boggs, & Algina, 1997).
The Bangor Dinosaur School Questionnaire (Hutchings, 2004)
The Bangor Dinosaur School Questionnaire has been developed by Hutchings
(2004), to determine any improvement in children’s problematic behaviour after the
intervention. Parents were asked about their child’s use of 10 specific behaviours
taught in Dinosaur School.
Scoring:
Never – 0; Rarely = 1; Sometimes = 2; Often = 1; Always = 4, scores are summed to
give each participant a total. A higher total score suggests a bigger social and
emotional problem.
Interpretation:
A reduction in the score indicates an improvement in problematic behaviour.
37
Building social and emotional competence in young high-risk school children
Beck Depression Inventory (BDI; Beck, Ward, Mendelson, Mock, &
Erbaugh, 1961)
This is a 21-item inventory measuring the severity of characteristic attitudes and
symptoms associated with depression. Such attitudes and symptoms are reflected by
the items, which are: sadness, pessimism/discouragement, sense of failure,
dissatisfaction, guilt, expectation of punishment, self-dislike, self-accusation, suicidal
ideation, episodes of crying, irritability, social withdrawal, indecisiveness, body-image
distortion, work retardation, insomnia, fatigability, loss of appetite, loss of weight,
somatic preoccupation, and loss of libido. Each item contains four possible responses
that range in severity from 0 (e.g. I do not feel sad) to 3 (e.g. I am so sad or unhappy
that I can’t stand it). For each item the respondent is required to select the one
statement that best describes the way that he/she has been feeling over the previous
week.
Rationale for use:
The co-occurrence of maternal depression and child conduct problems is well
documented (e.g. Murray & Cooper, 1997). Levels of depression in mothers of
children with conduct disorders have been shown to decrease following parent
training interventions (Webster-Stratton & Spitzer, 1996; Hutchings, et al 2007,
Bywater et al 2009). The monitoring of maternal depression in this study is
important considering the potential impact on child behaviour after attending the
Dina programme. Reduced child problem behaviour may affect parental depression
levels.
The BDI was chosen over other well-standardised measures of depression, such as
the Hamilton Rating Scale for Depression (Hamilton, 1967), for two reasons. First,
the BDI displays less over-reactivity to changes in depression (Edwards et al., 1984).
Second, the BDI has been used extensively in studies of mothers with young children
(e.g. Forehand et al., 1984; Webster-Stratton & Hammond, 1990; Webster-Stratton
& Spitzer, 1991; Hutchings et al., 2002, 2007).
Administration:
Respondents are asked to provide answers based on the way they have been feeling
over the previous week. The inventory is self-administered and takes approximately
10 minutes to complete.
Scoring
The scores from each of the 21-items are summed to achieve a total score
(minimum score = 0, maximum = 63). If more than one statement on an item has
been circled the highest scored statement is chosen.
Interpretation:
The total score provides an index of overall severity of depression. By convention,
total score levels of depression are interpreted in the following way:
Score 0 - 13 = minimal
Score 14 -19 = mild
Score 20 - 28 = moderate
Score 29 - 63 = severe
38
Building social and emotional competence in young high-risk school children
Cut off scores for the BDI-II can be adapted based on the rationale of using the BDIII and characteristics of the sample.
Reliability & Validity:
The BDI demonstrates high internal consistency with a mean coefficient alpha of .92
reported for psychiatric groups and .93 for the college students (Beck, Steer, &
Brown, 1996). The test-retest reliability correlation is 0.93. Various types of analysis
were used to estimate the BDI-II is convergent validity. The correlation between the
BDI-II and BDI-IA was 0.93 (p < .001).
The BDI has shown significant correlation with both clinicians’ ratings of depression
(Metcalf & Goldman, 1965) and objective behavioural measures of depression
(Williams, Barlow, & Agras, 1972). In addition, evidence indicates that the BDI
discriminates between subtypes of depression and differentiates depression from
anxiety (Beck et al., 1988).
Warwick-Edinburgh Mental Well-being Scale (WEMWBS; NHS Health
Scotland, University of Warwick and University of Edinburgh, 2006)
The WEMWBS is a 14 item positively worded item scale with five response
categories from ‘none of the time’ to ‘all of the time’. It has a time frame for
assessment of the previous two weeks that is consistent with DSM-IV criteria. The
instrument covers most aspects of mental health currently in the literature, including
both hedonic (subjective experience of happiness and life satisfaction) and
eudaimonic (psychological functioning and self realisation) perspectives: positive
affect (feelings of optimism, cheerfulness, relaxation), satisfying interpersonal
relationships and positive functioning (energy, clear thinking, self acceptance,
personal development, mastery, and autonomy). It does not include items specifically
on life satisfaction, but hedonic well-being is well represented. Items are summed to
give an overall score that can be presented as a mean score or graphically.
Rationale for use:
The co-occurrence of maternal depression and child disruptive behaviour is well
documented. Although pre-treatment levels of maternal depression have been found
to be significantly related to poor outcomes in parent training intervention
(Forehand, Furey, & McMahon, 1984), there is also evidence to suggest that levels of
depression in mothers of children with conduct problem may decrease following
parent training intervention (Webster-Stratton & Spitzer, 1996; Hutchings, Appleton,
Smith, Lane & Nash, 2002). The WEMWBS will therefore in this study examine any
differences in maternal well-being following the Incredible Years small group Dina
programme.
Administration:
The scale is self-administered and takes approximately 10 minutes to complete.
Individuals are required to tick the box that best describes their experience of each
statement.
Scoring
The Likert scale represents a score for each item 1 to 5 respectively, giving a
minimum score of 14 and a maximum score of 70. All items are score positively. The
39
Building social and emotional competence in young high-risk school children
overall score for the WEMWBS is calculated by totalling the scores for each item,
with equal weights.
Interpretation:
A higher WEMWBS score therefore indicates a higher level of mental well-being.
Reliability and Validity:
In a validation study (Tennant et al., 2007), WEMWBS demonstrated good content
validity. A Cronbach’s alpha score of 0.89 (student sample) and 0.91 (population
sample) suggests some item redundancy in the scale. WEMWBS showed high
correlations with other mental health and well-being scales and lower correlations
with scale measuring overall health. Its distribution was near normal and the scale
did not show ceiling effects in a population sample. Test-retest reliability at one
week was high (0.83). Social desirability bias was lower or similar to that of other
comparable scales.
The Parenting Scale (Arnold, O’Leary, Wolff & Acker, 1993)
This 30-item inventory is designed to measure dysfunctional discipline practices in
parents of children aged 18-48 months but has been used successfully in other
children up to age sixteen (Hutchings et al 2007, Lindsay et al 2009). The scale
targets specific aspects of parental discipline practice rather than providing a global
measure of such attitudes and beliefs. The measure contains three sub-scales:
Laxness, Over-reactivity and Verbosity. Responses are made using a 7-point scale
anchored between two alternative responses to a situation, where a score of 7
represents the highest score in terms of ineffectiveness. An example from the
Laxness sub-scale would be: When I say my child can’t do something, (situation) I let my
child do it anyway (most ineffective response, score 7), or I stick to what I said (most
effective response, score 1).
Rationale for use
The Parenting Scale was chosen over other short scales, such as the Parent
Behaviour Inventory - Part 11 (Budd, Riner, & Brockman, 1983), because, due to its
format, it asks specifically about parenting practices and is not tied to the frequency
of the child’s misbehaviour.
Administration:
This questionnaire is completed by the parent and takes approximately 10 minutes.
Scoring:
Each item receives a 1-7 score, where 7 represents the most ineffective response.
The following items have 7 on the left-hand side: 2, 3, 6, 9, 10, 13, 14, 17, 19, 20, 23,
26, 27, 30. The following items have 7 on the right-hand side: 1, 4, 5, 7, 8, 11, 12, 15,
16, 18, 21, 22, 24, 25, 28, 29. To calculate the total score add the responses on all
items and take the mean score. To calculate a factor score, take the mean for the
sum of responses on that factor. Laxness contains 11 items: 7, 8, 12, 15, 16, 19, 20,
21, 24, 26, 30. Over reactivity contains 10 items: 3, 6, 9, 10, 14, 17, 18, 22, 25, 28.
Verbosity contains 7 items: 2, 4, 7, 9, 11, 23, 29. Four items not on a factor are items
1, 5, 13, 27.
40
Building social and emotional competence in young high-risk school children
Normative information:
Arnold et al. (1993) examined data from 168 mothers of children aged 18-48 months
collected by recruitment from clinically referred cases (self-referred due to extreme
difficulties in coping with the index child) and from volunteers to participate in
parenting studies. From these two groups of participants 77 pairs were derived
(matched for demographic characteristics), their data was used to compare mean
scores on the Parenting Scale for clinic and non-clinic groups (see Table 2).
Table 3.
Parenting Scale and CBCL Scores for Clinic and Non-clinic Groups
Clinic group
(n = 26)
M
29.9
29.6
Child’s age (months)
Mother’s age (years)
Parenting Scale
Laxness
2.8
Over reactivity
3.0
Verbosity
3.4
Total score
3.1
CBCL Externalising T
58.7
score
Note. CBCL = Child Behaviour Checklist.
*p < .05, **p< .01
SD
4.5
6.7
Non-clinic group
(n = 51)
M
SD
28.6
3.3
31.7
3.9
1.0
1.0
1.0
0.7
10.3
2.4
2.4
3.1
2.6
47.7
0.8*
0.7**
1.0
0.6**
8.4***
Interpretation:
Although the preliminary data demonstrates the ability of the Parenting Scale to
distinguish between clinical and non-clinical groups, the results are based on small
samples. Arnold et al. (1993) note that normative data from a larger sample is
required before conclusions can be drawn about the parenting style of individuals.
Reliability & Validity:
The scale demonstrates adequate internal consistency, with coefficient alpha for the
factor and total scores ranging from .63 to .84. Test-re-test reliability over a 2-week
period yielded correlations of .79 to .84. In terms of convergent validity, the
Parenting Scale has shown significant correlational overlap with measures considered
to assess analogous constructs, such as the Child Behaviour Check list (r = .22 to
.54), Marital Adjustment test (Locke & Wallace, 1959; r = -.35 to -.53), and Beck
Depression Inventory (r = .30 for Over-reactivity). Further, Parenting Scale scores
showed significant correlation with observational assessments of parenting (r = .53
to .65). The factors Over-reactivity and Laxness have also proven stable across
different samples, i.e. school aged children with ADHD (Harvey, Danforth, Ulaszek,
& Eberhardt., 2001).
4. Child measures
41
Building social and emotional competence in young high-risk school children
The Wally Problem-solving Task (Webster-Stratton & Reid, 2001)
The task takes approximately 15 minutes to administer. A researcher presents the
child with a series of situations to ascertain how the child would react in certain
situations.
The Wally's Test presents thirteen (13) situations to the child, which cover eleven
categories or themes. The themes presented in the situations are as follows:
Rejection (#1), Making a mistake (#2, 6), Unjust treatment (#3), Victimization (#4,
7), Prohibition (#5), Loneliness (#8), Being cheated (#9), Disappointment (#10),
Dilemma (#11), Adult disapproval (#12), Attack (#13).
The codes: There are fifty-nine (59) coded child responses that can be identified.
The scoring sheet is designed for a maximum of 6 codes per situation (question)
presented to the subject child.
42
Building social and emotional competence in young high-risk school children
Appendix 4: Organisational Chart
43
Building social and emotional competence in young high-risk school children
Appendix 5: Funder reporting schedule
44