Doctorate in Educational and Child Psychology
Dannika Osei
Case Study 1 – An Evidence-Based Practice Review Report
Theme: Interventions Involving Parents
Does the ‘Tuning into Kids’ program enable parents to reduce emotional and behavioural
difficulties in their children?
Summary
This systematic literature review aims to discover how effective the Tuning into Kids
programme is in enabling parents to reduce emotional and behavioural difficulties in their
children. Tuning into Kids and its variants (Tuning into Toddlers and Tuning into Teens) is
an emotion-focused parenting programme aimed at developing emotionally responsive
parenting with the aim of increasing emotion knowledge in children as well as reductions in
child behaviour problems. Six studies met the inclusion criteria and were reviewed using
the Weight of Evidence Framework (Gough, 2007) and the APA Task Force Coding
Protocol by Kratochwill (2003). The programme was effective in reducing emotional and
behavioural difficulties in children as indicated by small-medium effect sizes across the
majority of studies however was most effective for a clinical population as shown by large
effect sizes. Potential use and implications for future directions are discussed.
Introduction
What is Tuning into Kids?
The Tuning into Kids (TIK) program and its age-specific programs (Tuning into Toddlers;
Tuning into Teens) is a parenting intervention developed by Havighurst, Wilson, Harley and
Prior (2009) and is aimed at improving parent’s emotion coaching skills. According to
Gottman, Katz and Hooven (1996), emotion coaching parents are aware of low-intensity
emotions in themselves and their children, see negative emotions in their child as an
1
Doctorate in Educational and Child Psychology
Dannika Osei
opportunity for intimacy or teaching, validate their child’s emotions and help the child to
verbally label their emotions. Emotion coaching also involves problem solving with the
child, identifying goals for dealing with the problem situation and setting behavioural limits.
This contrasts with emotion dismissing parents who aim to quickly change a child’s negative
emotions, deny their feelings and convey to their children that negative emotions are
unimportant. The latter parenting style is believed to be related to poor child outcomes
(Havighurst et al, 2009).
The components of the program are outlined in Havighurst et al (2009) and have been
summarised in Table 1. TIK is a six-session, two-hour-per-week parenting program. TIK
hinges on teaching parents the five steps of emotion coaching outlined by Gottman (1997)
cited in Havighurst et al (2009). These five steps are similar to the above however also
emphasises communicating understanding and acceptance of the child’s emotion. The
steps are broken down into a series of exercises to be carried out throughout each session.
As the first four steps are thought to be most difficult to learn, they are prioritised, therefore
first three weeks of the program emphasise attending to the lower intensity emotions
exhibited by children, being able to reflect, label and empathise with the child’s negative
emotion. The fourth session focuses on anxiety and problem solving skills. The last two
weeks examine emotions such as anger and teach emotion regulation techniques. As the
importance of parents being aware of their own emotions was noted by Gottman et al
(1996), parents are taught how to understand and regulate their own emotions and reflect
on their experiences with emotion beliefs and responses deriving from their family of origin.
Sizes of parenting groups ranges from 7-15 and are carried out by two trained facilitators.
As the review focuses on Tuning into Kids, Toddlers and Teens, the program will be
referred to as “Tuning into” (TI).
2
Doctorate in Educational and Child Psychology
Dannika Osei
Table 1 – Summary of steps taught in ‘Tuning into Kids’
Steps taught in ‘Tuning into Kids’
1
Emphasis
To become aware of the child’s Becoming aware of emotions and
emotion, especially if this emotion is at how
a low intensity.
2
may
present
at
a
physiological level.
To view the child’s emotion as an
opportunity for intimacy and teaching.
3
they
Attending
to
the
child’s
To communicate to the child an intensity behaviours.
lower
Reflecting,
understanding and acceptance of their labelling and empathising with the
emotion.
4
emotion.
To help the child to be able to use Focuses on anxiety and problem
words to describe how they feel.
solving.
Also focuses on more
intense emotions such as anger.
5
To assist the child with problem solving Teaching
while setting limits.
emotion
regulation
strategies involving slow, breathing,
relaxation, expressing anger in a
safe way and the turtle technique
which teaches self-control (PATHS;
Greenberg et al, 1995).
Throughout
Parents are taught to regulate and understand their own emotions
program
3
Doctorate in Educational and Child Psychology
Dannika Osei
Psychological Basis
The TI program draw on a number of psychological theories. A parent’s ability to respond
to their child’s emotions hinges on the parent’s emotional well-being. Therefore emotion
expression and regulation (Havighurst et al., 2009), meta-emotion (Gottman et al., 1996)
and Mindfulness is integrated into the program.
This teaches parents to reflect on their
experiences of emotion socialisation as well as being able to “sit with” their emotions when
they need to respond to their child’s emotions.
However, Social learning Theory (Bandura, 1977) strongly underpins this program.
According to Bandura, humans learn through observing the attitudes, behaviour and the
consequences of behaviour that others in their environment display. Thus, most human
behaviour is learned through others such as parents, teachers, peers who model the
behaviour. Through this, people are able to form ideas on how to execute new actions
which later serves as a guide for how they should perform the behaviour. According to
Denham (1998), the emotional expression and regulation demonstrated by parents is an
important model for the child on how to go about managing and showing their emotions.
The TI program suggests children experience emotion socialisation through interacting with
caregivers, siblings and teachers and the emotional experiences they are exposed to
through these people.
This plays a crucial role in how a child develops emotional
competence (Havighurst et al., 2009).
Attachment theory is another major concept in the development of the program. Bowlby
(1958) states attachment is an innate primary drive in an infant and results in the infant
maintaining proximity to their caregiver and looks at how infant’s emotions and behaviours
such as crying, smiling and clinging were received and responded to by parents. Ainsworth,
Blehar, Waters and Wall (1978) identified three categories of attachment: securely
attached, insecure-avoidant attachment and insecure resistant. Secure attachment is
4
Doctorate in Educational and Child Psychology
Dannika Osei
associated with positive interactions between parent and child whereas the others were
associated with negative interactions.
Relating attachment to emotion socialisation,
Ainsworth, Bell and Stayton (1971, 1974) in Meins (2003) found mothers who responded
sensitively to their infants’ cues had securely attached infants while mothers who rejected
their infants’ cues had insecure-avoidant children and inconsistency in parenting was linked
to insecure-resistant attachment styles.
According to Green and Goldwyn (2002),
ambivalent attachment styles are linked to anxiety and social withdrawal and avoidant
styles are linked to antisocial development. They also note attachment disorganisation
could lead to vulnerability in a child’s self-concept and emotion-regulation. Given the
research on and importance of early attachments, TI aims to develop supportive and
emotionally responsive parenting.
Rationale
In the new Special Educational Needs and Disabilities Code of Practice (SEND, 2014)
emotional and behavioural difficulties is referred to as Social, Emotional and Mental Health
(SEMH) Difficulties. This area of need is characterised by displaying withdrawn or isolated
behaviour, challenging, disruptive behaviour as well as disturbing behaviour. According to
the Department for Education (2014), in state-funded primary schools, 18.4% of pupils with
statements and those at school action plus had a primary need of Behaviour, Emotional
and Social difficulties (BESD) and in state-funded secondary schools this figure was 26.7%.
As well as supporting the inclusion of pupils with academic learning difficulties, Educational
Psychologists (EPs) must also work to support the inclusion of pupils whose SEMH pose
barriers to their learning.
These pupils are overrepresented in UK exclusion figures
5
Doctorate in Educational and Child Psychology
Dannika Osei
(Sellman, 2009). Farrell et al. (2006) highlight the role EPs have in providing support and
intervention for children and young people who are experiencing BESD. However, early
intervention and prevention is another valuable aspect of EP practice (Farrell et al., 2006)
and resonates with frameworks such as Positive Educational Practices (PEPs; Noble and
McGrath, 2008) which states EPs must focus on promoting wellbeing such as through
teaching social and emotional competency.
As emotion socialisation begins with the
primary caregivers of a child, promoting positive emotional learning opportunities from early
childhood could be particularly advantageous to pupils at-risk of SEMH, as well as to
Educational Psychology Services (EPSs) in the long run. Therefore the TI intervention
could potentially be important to EPs in empowering parents in being able to prevent the
development of behavioural difficulties in their children. Therefore this review aims to
answer the following question: Does the ‘Tuning into’ program enable parents to reduce
emotional and behavioural difficulties in their children?
Critical Review of the Evidence Base
Literature Search
Initial searches were carried out during December 2014 using electronic databases
PsychINFO, Medline and ERIC. Using a multi-field search, the following search terms were
entered into ‘all fields’ or ‘title’ to retrieve studies (see Table 2). As the research base for
this intervention is relatively new, the Principal researcher for the TI programme was
emailed to retrieve unpublished or ‘in press’ studies on the intervention.
6
Doctorate in Educational and Child Psychology
Dannika Osei
Table 2: Search terms used in PsychINFO, Medline and ERIC
Search
number
1
1
2
3
Tuning into
(T)
Tuning into
(T)
Emotion
social*
Emotion
Behavio*
Emotion
social*
Behavio*
4
Emotion
focused
parenting
Tuning
Behavio*
5
Parenting
Emotion
coaching
Emotion
coaching
2
3
Behavio*
‘conduct’ or
‘behavio’
(T)
4
child OR toddler
OR Teen OR
adolescent
child OR toddler
OR Teen OR
adolescent
kids
children
T = title only
* = wildcard search term
Inclusion and exclusion criteria
Studies retrieved were included in the review if they met the criteria detailed in Table 3. As
shown in Figure I, 86 papers were found through the electronic databases: 65 from
PsychINFO, 14 from ERIC and 7 from Medline. Furthermore, 15 studies were received
from the principal researcher of the intervention, bringing the total amount of retrieved
papers to 101. 39 papers were excluded as duplicates and the remaining studies were
screened through their titles and abstracts and excluded based on the below criteria. 9
papers were eligible for a full review and a further 2 were excluded using the criteria in
Table 3 (see Appendix A). As shown in Appendix B, a total of 7 papers were suitable for
inclusion in the review.
7
Doctorate in Educational and Child Psychology
Dannika Osei
Table 3: Inclusion and exclusion criteria
Inclusion criteria
Exclusion criteria
1. Type
of
publication
a) Must be in a peer
reviewed-journal
or
have been submitted
for peer-review
a) Is not or has
not
been
submitted for
peer review
i.e.
Books
chapters.
2. Language
and setting
a) Must be written in
English. No restrictions
on country in which
research has taken
place.
a) The study must solely
implement the Tuning
into
Kids/Teens/Toddlers
intervention.
a) Study is not
written
in
English
3. Intervention
b) Must include core
components
of
teaching 5 steps of
emotion
coaching
skills (Gottman, 1997)
and
must
be
implemented
for
standard duration of 6
weeks.
4. Type
design
of
5. Outcomes
6. Participants
a) Must be a group
design that reports
between
group
outcomes (e.g. an
RCT) or one that
reports pre and post
measures.
a) The study reports on
outcomes of child
behaviour
postintervention
a) Must involve parents of
children and young
people.
b) Sample is unique to
this paper
Rationale
Peer
reviewers
assess the quality of a
study and therefore
the study in these
journals is likely to
meet the required
standards.
Reviewer does not
have the resources to
access
other
languages.
a) Study
does
not implement
the
‘Tuning
into’
intervention
or,
study
implements
‘Tuning
into…’
intervention
alongside
another
intervention.
b)
Study
implements
TIK program
for
longer
than
6
sessions.
a) The study is a
single-case
design.
The review is based
on the ‘Tuning into’
intervention.
Reviewer
will
be
unable to discern what
effects are due to the
‘Tuning
into’
intervention.
If
interventions
last
longer
in
some
studies, effect sizes
may differ according
to this variable.
a) Child
behaviour
outcomes are
not reported
The review aims to
find out whether the
parenting intervention
improves
child
behaviour.
The review aims to
find
out
how
a
parenting programme
can
improve
behavioural outcome
for children.
If sample has been
used for more than
one study, findings
may not be entirely
representative
therefore the study
that most closely fits
inclusion criteria will
be used.
a) The
study
does
not
involve
parents
b) Sample has
been used for
multiple
articles
8
Doctorate in Educational and Child Psychology
Dannika Osei
Figure 1: Flowchart: Application of inclusion and exclusion criteria
Papers for review
of title and abstract
Articles identified from electronic
databases
Psychinfo
n= 65
ERIC
n= 14
Medline
n= 7
Total
n= 86
Articles sent by author
N= 15
N=101
Electronic database:
Papers excluded on basis of inclusion criteria 3a
N= 54 A
Papers excluded as duplicates
N=24
Total – 78
Sent by author:
Papers excluded on basis of inclusion criteria 1a,
3a, 3b, 4a, 3b, 5a
N=6
Papers excluded as duplicates
N= 9
Total - 15
Papers for review of
full text
N =9
Papers excluded
Inclusion criteria 6b – n=2
Papers included
N=7
9
Doctorate in Educational and Child Psychology
Dannika Osei
Critical appraisal for quality and relevance
The seven papers were summarised to capture the main aspects as well as the information
relevant to the review question (See Appendix C). The quality and relevance of each study
was appraised using the Weight of Evidence (WoE) framework (Gough, 2007). WoE
comprises four judgements. WoE A examines methodological quality in terms of the quality
of how well the study was executed. WoE A was judged using the APA Task Force Coding
Protocol by Kratochwill (2003) which was adapted according to the purpose of the review
question and to determine the methodological quality of the studies (See Appendix D).
Each study was examined using this protocol in order to systematically analyse each study
based on the same criteria. WoE B addresses the methodological relevance of the study the appropriateness of the study in relation to the review question. Finally WoE C appraises
the relevance of the focus of the study to the review question.
Outcomes from the
aforementioned WoEs are averaged to calculate an overall WoE – termed WoE D (See
Table 4). For further information on how each study was appraised, see Appendix E.
Table 4- Overall Weight of Evidence (WoE D)
Studies
WoE A
Quality of
methodology
WoE B
Relevance of
Methodology
WoE D
Overall
weight of
evidence
Medium
2
WoE C
Relevance of
evidence to
the review
question
Medium
2
Havighurst
et al (2009)
and
Havighurst
et al (2010)
Havighurst
et al (2011)
Kehoe et al
(2014)
Lauw et al
(2014)
Wilson et al
(2012)
Havighurst
et al (2004)
High
2.5
High
3
High
2.5
Medium
1.25
High
2.75
Medium
2
High
3
Medium
2
Low
1
Medium
2
Low
1
High
3
Medium
2
Low
1
Low
1
Medium
2
High
3
Medium
2.17
Low
1.11
Medium
1.92
Medium
1.27
Medium
2.17
19
Doctorate in Educational and Child Psychology
Dannika Osei
Participants
The number of participants in the studies ranged from 34-225 and were recruited from cities
such as Melbourne and Knox, Australia. The socioeconomic status (SES) of parents
across the sample varied as indicated by household incomes, however three studies
(Kehoe et al.; 2014; Lauw et al.; 2014; Wilson et al.; 2012) had a middle-upper class
majority. All included studies included only the primary caregiving parent of which the
majority were female (93.8%). Inclusion criteria for involvement in the study involved
parents being able to speak enough English to understand the intervention, having a child
within the target age for the study and not being committed to other research projects or
other parenting programmes. However, Havighurst et al. (2010) and Havighurst et al.
(2013) excluded parents if they had a child with a diagnosis of any communication disorders
or pervasive developmental disorders. Finally, Havighurst et al. (2013) only included
parents who had children who were above the clinical cut-off on the Eyberg Child Behaviour
Inventory (ECBI) to ascertain whether TIK could be used as an intervention for children with
behavioural difficulties therefore was rated highly on WoE C.
Samples were acquired using convenience sampling methods whereby parents in schools,
preschools, a kindergarten, a Maternal and Child Health centre, or behaviour clinics in
hospitals were asked to participate through distribution of information flyers and letters. In
Psychology research, this method of sampling is common however, non-random sampling
methods like convenience sampling are biased as not every member of the target
population has an equal chance of being selected (Barker, Pistrang and Elliot, 2002). As
the majority of participants opted into the study, particular characteristics in these parents
such as a motivation to improve parenting, an interest in socioemotional functioning or the
time available, may have influenced the decision to participate which may not be present
20
Doctorate in Educational and Child Psychology
Dannika Osei
in parents that did not opt-in. Therefore this may affect generalizability of results to the rest
of their target population.
Design
Randomised Controlled trials (RCTs) and prettest-posttest designs were included in this
review. Four of the studies were RCTs: Havighurst et al. (2009; 1a) and (2010; 1b),
Havighurst et al. (2013), Kehoe et al. (2014) and Wilson et al. (2012). Randomisation
methods varied from randomising schools into intervention/control groups to randomising
participants into groups (Havighurst et al., 2011) and was done using a random-number
generator. Havighurst et al. (2013) was given a high rating because it used an ‘active’
comparison group over a waitlist control group who received treatment as usual (Paediatric
treatment). This was rated highly as using an active comparison may show the intervention
is more effective than currently applied interventions for children presenting with
behavioural difficulties.
Furthermore active comparisons are more ethical than a no
intervention group which withholds a potentially beneficial intervention from a group,
however this poses less risk if participants are not clinically distressed (Barker, Pistrang
and Elliot, 2002). The remaining studies using a waitlist intervention are deemed more
ethical than a no intervention group however, on WoE B, received a medium rating. This
is because one cannot conclude from these studies that the TI intervention is better than
alternative interventions in reducing emotional and behavioural difficulties in children. All
RCTs used objective methods to randomise participants into intervention and control
groups thus increasing the chances of equivalence between groups and reducing the risk
of error of bias in results (Evans, 2003).
Prettest-posttest designs used were in Lauw et al. (2014) and Havighurst et al. (2004) and
were both pilot studies. These studies were rated low on WOE B as it is problematic
21
Doctorate in Educational and Child Psychology
Dannika Osei
attributing changes in behaviour to the intervention alone without the presence of a control
group and poses threats to validity and reliability. Cook and Campbell (1979) cited in
Barker, Pistrang and Elliot (2002) note there are possible threats to internal validity when
using a prettest-posttest design including maturational trends whereby participants grow
out of their problem, a case especially relevant to children.
Studies varied on when measurements were taken. Some measured behaviour at preintervention and post-intervention only (Lauw et al., 2014) pre-intervention, postintervention and follow-up (Havighurst et al., 2009;1a; 2010;1b; Havighurst et al., 2013; and
Havighurst et al., 2004) whereas others did pre-intervention and follow-up only (Kehoe et
al., 2014) and Wilson et al., 2012). Taking follow-up measures as well as pre-intervention
and post-intervention measures provides information on not only how effective an
intervention is but how long-lasting effects are. However if measures are only taken at two
time points pre-intervention and post-intervention then the study only provides information
on the immediate effect of the intervention. Conversely, it can be argued that, only taking
measures of behaviour at follow-up provides some evidence about the sustainability of
change after the intervention. However without immediate measures of change to compare
it to, it is possible that other factors may have contributed to improvement in behaviour over
the 3/6 month follow up period. Therefore studies measuring behaviour change preintervention, post-intervention and follow-up were rated high on WoE B whereas those with
control groups that only took measures at two time-points were rated ‘medium’ and low
ratings were given to those without a control group that only took measures at two-time
points.
22
Doctorate in Educational and Child Psychology
Dannika Osei
Measures
The Eyberg Child Behaviour Inventory 6 (ECBI; Eyberg and Pincus, 1999) was used across
all studies but Kehoe et al. (2014) and Lauw et al. (2014). The ECBI has Cronbach’s alpha
reliability scores ranging from .90 to .94 across studies therefore has a high reliability.
These studies were also multi-source and multi-method thus receiving high ratings on WoE
A and C. The Sutter-Eyberg Student Behaviour Inventory – a teacher’s version of the ECBI
was used by Havighurst et al. (2009;1a and 2010;1b) and Havighurst et al. (2013) which
had Cronbach’s alpha’s of .97. As this was an additional measure of behaviour, in addition
to the ECBI as completed by parents, this moved data toward triangulation which led to
high weightings on WoE A and C. Other measures include the Social Competence and
Behaviour Evaluation (SCBE-30; LaFreniere and Dumas, 1996), a teacher report. In this
study (Wilson et al., 2012), they examined social competence and anger and aggression
scales thus was deemed a suitable measure of behaviour. Reliability coefficients ranged
from .88 - .92. Kehoe et al. (2014) sought to measure internalising behaviour difficulties
(anxiety and depression) in youth. Anxiety was measured using the Spence Children’s
anxiety scale (SCAS; Spence, 1998) and the parent-report version (SCAS-P, Nauta et al.,
2004) both of which had Cronbach’s alphas of .90 - .93. Depressive symptoms were
measured using the Child Depression Inventory (CDI; Beck, 1977) and the parent version,
CDI:P (Garber, 1984) and had reliability coefficients ranging from .84-.87. Consequently,
this study was rated highly on WOE A and C.
In Havighurst et al. (2004), the Strengths and Difficulties Questionnaire (SDQ: Goodman,
1997) was used which had reliability coefficients ranging from .57 to .77. Cronbach’s
alphas were not reported for the ECBI or the ERC, however studies reporting reliability
were cited, therefore on WOE A this received a medium rating. Lauw et al. (2014) also
received a medium rating for measures but only included the Brief Infant-Toddler Social
23
Doctorate in Educational and Child Psychology
Dannika Osei
and Emotional Assessment (Briggs-Gowan and Carter,2007) with no reliability coefficients
reported, this was given a medium weighting on WoE A and low on WoE C.
All child behavioural outcomes were measured using questionnaire data however using
self-report may be subjective which poses validity issues. Respondents may be less
truthful on questionnaires due to social desirability bias in that they may want to impress
researchers or may report outcomes suggesting improvements due to expectancy effects
(Barker, Pistrang and Elliot, 2002). Therefore validity could be improved if observational
measures of child behaviour were taken pre and post intervention in addition to self-report
measures. However, where parent self-report was supplemented by self-reports from other
perspectives (as specified on WoE A and C), this increases validity.
Findings
Parent-reported behaviour
Effect size data is briefly summarised in Table 6. A more in depth summary can be found
in Appendix F. The majority of effect size outcomes in studies using the ECBI were smallmedium however some large effect sizes were found. This suggests the TI programme
has moderate efficacy in reducing emotional and behavioural difficulties in children.
Havighurst et al. (2009; 2010) had a small-medium effect size for behaviour intensity at
posttest which was maintained at follow-up as well as a medium WoE D suggesting the
effect size is an accurate depiction of the study’s effectiveness. Wilson et al. (2012)
however had a small effect size as well as non-significant results for their time-condition
interaction effect. This may be due to the fact that post-intervention measures were only
taken 6 months after the intervention ceased.
Therefore, immediate effects of the
intervention, which are valuable indicators of effectiveness, were not captured. Another
potential reason is that the programme facilitators were professionals from the local
24
Doctorate in Educational and Child Psychology
Dannika Osei
community, many of whom did not have Psychology degrees, experience in group
facilitation or parenting education. Although professionals were trained for two days and
provided with a manual, arguably as it is a specialised psychological intervention with a
range of components, the facilitator may need to have a background in Psychology to be
able to deliver this to full effect. This is reflected in a low WOE C rating.
Havighurst et al. (2013), was the only study with a large effect size on the ECBI and rated
high on WoE D making this a particularly impressive study. However effects for follow-up
measures were small for behaviour intensity and small-medium on behaviour problems.
This may suggest using the TI programme as an intervention with children at the clinical
level on the ECBI is where the intervention has the strongest effect, however long-term
effects are not as strong as immediate effects. The only drawback is that this study did not
have a sufficient sample size therefore may have been underpowered. If a sample does
not have enough power, the chances of the study detecting a significant effect is reduced,
therefore although a statistically significant effect was found for reduction in behaviour
intensity, a larger sample size may have found a smaller p-value. Despite having a medium
WoE D, Havighurst et al. (2004) also used the ECBI along with the ERC which both had
small effect sizes. This could be because in addition to having a small sample size, this
study was the pilot of TI programme and although the essential ‘emotion coaching’
components were present, the added benefit of helping parents to help children to regulate
anger and worry, and enhancing parental emotion awareness and regulation was only
added when the intervention formally became TIK.
Kehoe et al. (2014) had medium effect sizes when anxiety and depression was rated by
parents (SCAS-P, CDI-P) however small effect sizes when this was rated by youth (SCAS,
CDI).
Potential reasons for this could be similar to the above in that there was an
expectancy bias present in parents. A small effect size was found on the Brief-ITSEA in
25
Doctorate in Educational and Child Psychology
Dannika Osei
Lauw et al. (2014). As this too was a pilot study without a control group, it was also rated
low on WOE D. Additionally, the study had approximately half the required sample size
and so may have lacked enough power to detect a more significant improvement in toddler
behaviour problems.
Teacher-reported behaviour
The SESBI was used in Havighurst et al. (2009; 2010) and Havighurst et al. (2013) as a
follow-up measure and showed small and medium effects respectively.
However,
Havighurst et al. (2004) used an alternative teacher-report at posttest rather than follow up.
These outcomes showed a medium effect size. This may be as teacher-reported measures
were only taken at follow up thus immediate effects observed by parents may have
diminished with time as also shown by smaller parent-reported effect sizes at follow-up.
Similarly parent-reported behaviour change may reflect an expectancy bias as they were
the main change agents for the children and were aware of the purposes of the intervention,
therefore they may have anticipated a positive change in their child’s behaviour which may
be reflected in their ECBI scores. Conversely, teachers may be more objective as they
were not directly involved in the intervention. However as some effect was shown, it
supports the idea that moderate improvements were observed.
26
Doctorate in Educational and Child Psychology
Outcome
Effect size
Study
Behaviour Intensity
(ECBI)
PPC SMD = -0.44
(Small-medium)
Havighurst et al
(2009)
PPC SMD= -0.76
(Large)
Behaviour Problem
(ECBI/SESBI/BITSEA)
Lability/Negativity
(ERC)
Dannika Osei
Number of
Participants
218
Overall Weight
of Evidence
Medium
Havighurst et al
(2013)
54
High
PP SMD = -0.26
(Small)
Havighurst et al
(2004)
47
Medium
PPC SMD= -0.83
(Large)
PP SMD = -0.26
(Small)
PP SMD = -0.31
(Small)
Havighurst et al
(2013)
Lauw et al (2014)
54
High
34
Low
47
Medium
PP SMD= 0.49
(Medium)
PP SMD = -0.31
(Small)
Havighurst et al
(2004)
Havighurst et al
(2004)
47
Medium
Havighurst et al
(2004)
Table 6 – Summary of effect sizes and overall quality ratings
a) Pre-post
33
Doctorate in Educational and Child Psychology
Outcome
Effect size
Study
Behaviour Intensity
(ECBI/SESBI)
PPC SMD
= - 0.41
(Small-medium)
PPC SMD = -0.2
(Small)
Havighurst et al
(2010)
Number of
Participants
216
Overall Weight
of Evidence
Medium
Havighurst et al
(2010)
216
Medium
PPC SMD = -0.13
(Small)
Havighurst et al
(2013)
54
High
PPC SMD= -0.58
(Medium)
Havighurst et al
(2013)
54
High
PPC SMD= -0.22
(Small)
Wilson et al (2012)
128
Medium
Havighurst et al
(2004)
47
Medium
Havighurst et al
(2013)
54
High
PPC SMD= -0.5
(Medium)
Havighurst et al
(2013)
54
High
PPC SMD= -0.26
(Small)
Wilson et al (2012)
128
Medium
PP SMD = -0.42
(Small-medium)
Havighurst et al
(2004)
47
Medium
PPC SMD = -0.19
(Small)
Kehoe et al (2014)
225
Medium
PPC SMD = -0.48
(Small-medium)
Kehoe et al (2014)
225
Medium
PPC SMD= -0.13
(Small)
Kehoe et al (2014)
225
Medium
PPC SMD =-0.46
(Small-medium)
Kehoe et al (2014)
225
Medium
PPC SMD= 0.02
(Small)
PP SMD = -0.41
(Small-medium)
Wilson et al (2012)
128
Medium
Havighurst et al
(2004)
47
Medium
PP SMD = -0.4
(Small-medium)
Behaviour Problem
(ECBI/SESBI)
Anxiety
(SCAS/SCAS-P)
Depression
CDI:S/CDI:P)
Anger/Aggression
Lability/Negativity
Dannika Osei
PPC SMD = -0.42
(Small)
b) Pre-follow-up
Note: Effect sizes were calculated from data given in the studies. For RCTs, Morris (2007) PrettestPosttest Control (PPC) design Standardised Mean Difference (SMD) was used. This SMD was
34
Doctorate in Educational and Child Psychology
Dannika Osei
used as opposed to Cohen’s d as it allows a computation of the difference in prettest-posttest scores
between the intervention and control groups whereas a Cohen’s d calculation would not factor in
pre-intervention scores. This gives a more accurate measure of the effects of the intervention and
takes advantage of the strengths of an RCT design (Morris, 2007). Effect sizes using this
calculation will be referred to as PPC SMD. For studies without a control group, Becker (1988)
SMD was used as this calculation also examines prettest and posttest scores (referred to as PP
SMD). Due to the absence of a control group, these effect sizes must be interpreted with caution
and cannot be directly compared to PPC SMDs. Cohen’s d (1988) effect size descriptors (high,
medium, low) have been used to describe effect sizes at .02, .05 and .08 respectively.
Conclusions
This systematic literature review aimed to discover to what extent the TI intervention
enabled parents to reduce emotional and behavioural difficulties in their children. This
programme is efficacious in achieving this with small-medium effect both at posttest and
prettest. However, largest effects are clearly observed when implementing the
programme with a clinical population. This may be because children were displaying
above average behavioural difficulties thus had greater scope for improvement than
mainstream populations. According to Forgatch and DeGarno (1999), interventions have
been shown to have greatest effect on participants when they are in the clinical range.
This is because the clinical sample are usually close to homogeneity in terms of
diagnosed problems whereas prevention studies use samples that are heterogeneous in
type and intensity of their difficulties thus need a greater amount of power than clinical
trials.
35
Doctorate in Educational and Child Psychology
Dannika Osei
Limitations
All of the studies were carried out by the same research group and consistently involved
Havighurst, Wilson and/or Harley. This could potentially be a source of bias across
studies as researchers may have a particular interest in and investment in promoting how
effective the programme is. This may lead to results being analysed and reported in
similar ways, as well as a tendency to only report the most significant results and
downplay the significance of any non-significant results. For example only commenting
on the “main effect” significance instead of the interaction effect i.e. in Wilson et al.
(2012). However, the use of standardised measures of child outcomes could be deemed
to control for some of the bias.
Recommendations
In light of the above, although TI shows promising outcomes as a prevention programme
in terms of WoE, effect sizes and strong outcomes as an intervention programme for a
clinical sample, a more extensive, varied body of research is needed before this is
adopted as an intervention programme within EPSs. More research on its effect on
children at the clinical level of behavioural difficulties is needed to build on the idea that
this programme is best as an intervention. To reduce potential bias in the studies, the
programme may benefit from different research groups researching its effectiveness.
Subsequent studies may benefit from including an observational measure of behavioural
into the measures to triangulate questionnaire data obtained. Most studies incorporated a
follow-up measure into the design which shows how long the effects of the intervention
last therefore to further develop this, a longitudinal study could be conducted whereby
child behavioural outcomes are measured over the course of a few years.
36
Doctorate in Educational and Child Psychology
Dannika Osei
The outcomes of this review show that the TI programme has potential benefits on the
emotional wellbeing of children and young people as both an intervention and prevention
programme. As studies where the programme was delivered by psychologists produced
better outcomes than the one that was delivered by other professionals who had received
training, it is recommended that this programme be delivered by EPs or other
psychologists of a similar background. After training, the programme could be delivered
by EPs to groups of parents identified as having children/young people with/at risk of
developing SEMH. The effectiveness of the programme should be evaluated by
measuring behaviour pre and post intervention and at follow up using the EBCI, SESBI,
SCAS, or CDI.
37
References
Ainsworth, M. D. S., Blehar, M. C., Waters, E., & Wall, S. (1978). Patterns of attachment: A
psychological study of the strange situation. Hillsdale, NJ: Erlbaum.
Bandura, A. (1977). Social learning theory. New Jersey: Prentice Hall.
Barker, C., Pistrang, N., & Elliott, R. (2002). Research methods in clinical psychology: An
introduction for students and practitioners. (2nd ed) Chichester: John Wiley &Sons LTD.
Bowlby, J. (1958). The nature of the child’s tie to his mother. International Journal of
Psychoanalysis, 39, 350-371.
Briggs-Gowan, M. J., & Carter, A. S. (2007). Applying the Infant-Toddler Social and Emotional
Assessment (ITSEA) and Brief-ITSEA in early intervention. Infant Mental Health Journal, 28, 564–
583.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Statistical Power Analysis
for the Behavioral Sciences.
Cohen, J. (1992). Quantitative Methods in Psychology. A Power Primer. Psychological Bulletin,
112(1), 155–159.
Dawson, J., & Singh-Desi, D. (2010). Educational psychology working to improve psychological
well-being: an example. Emotional and Behavioural Difficulties, 15, 295–310.
Denham, S.A. (1998). Emotional development in young children. New York: Guilford Press.
Department for Education. (2013). Statistical First Release: Special Educational Needs in England,
1–13.
Evans, D. (2003). Hierarchy of evidence: A framework for ranking evidence evaluating healthcare
interventions. Journal of Clinical Nursing, 12, 77–84.
34
Eyberg, S., & Robinson, E. A. (1983). Conduct problem behavior: Standardization of a behavioural
rating scale with adolescents. Journal of Clinical Child Psychology, 12, 347-354.
Eyberg, S., & Pincus, D. (1999). Eyberg Child Behavior Inventory and Sutter-Eyberg Student
Behavior Inventory_Revised: Professional manual. Odessa, FL: Psychological Assessment
Resources.
Farrell, P., Woods, K., Lewis, S., Rooney, S., Squires, G., & Connor, M. O. (2006). A Review of the
Functions and Contribution of Educational Psychologists in England and Wales in light of “ Every
Child Matters : Change for Children .”
Forgatch M. & S, DeGarmo D. S. (1999). Parenting through change: An effective prevention
program for single mothers. Journal of Consulting and Clinical Psychology. 67, 711–724.
Garber, J. (1984). The developmental progression of depression in female children. New
Directions for Child Development, 26, 29–58.
Gottman, J. M., Katz, L. F., & Hooven, C. (1996). Parental meta-emotion philosophy and the
emotional life of families: Theoretical models and preliminary data. Journal of Family Psychology,
10(3), 243–268.
Gough, D. (2007). Weight of evidence: a framework for the appraisal of the quality and relevance
of evidence, Research Papers in Education, 22(2), 213-228.
Green, J., & Goldwyn, R. (2002). Annotation: Attachment disorganization and psychopathology:
New findings in attachment research and their potential implications for developmental
psychopathology in childhood. Journal of Child Psychology and Psychiatry, 43, 835–846.
Greenberg, M.T., Kusche, C.A., Cook, E.T., & Quamma, J.P. (1995). Promoting emotional
competence in school-aged children: The effects of the PATHS curriculum. Development and
Psychopathology, 7, 117–136.
35
Havighurst, S. S., Harley, A., & Prior, M. (2004). Building Preschool Children’s Emotional
Competence: A Parenting Program. Early Education & Development, 15(4), 423–448.
Havighurst, S. S., Wilson, K. R., Harley, A. E., Kehoe, C., Efron, D., & Prior, M. R. (2013). “Tuning
into Kids”: reducing young children’s behavior problems using an emotion coaching parenting
program. Child Psychiatry and Human Development, 44(2), 247–64.
Havighurst, S. S., Wilson, K. R., Harley, A. E., & Prior, M. R. (2009). Tuning in to kids: an emotionfocused parenting program-initial findings from a community trial. Journal of Community
Psychology, 37(8), 1008–1023.
Havighurst, S. S., Wilson, K. R., Harley, A. E., Prior, M. R., & Kehoe, C. (2010). Tuning in to Kids:
improving emotion socialization practices in parents of preschool children--findings from a
community trial. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 51(12), 1342–
50.
Kehoe, C. E., Havighurst, S. S., & Harley, A. E. (2014). Tuning in to Teens: Improving Parent
Emotion Socialization to Reduce Youth Internalizing Difficulties. Social Development, 23(2), 413–
431.
Kovács, M. (1981). Rating scales to assess depression in school-aged children. Acta
Paedopsychiatrica, 46 (5–6), 305–315.
Kovács, M., & Beck, A. (1977). An empirical clinical approach toward a definition of childhood
depression. In J. G. Schulterbrandt & A. Raskin (Eds.), Depression in children: Diagnosis,
treatment and conceptual models (pp. 1–25). New York: Raven Press.
Kratochwill, T. R. (2003). Evidence-Based Practice: Promoting Evidence-Based Interventions in
School Psychology. School Psychology Quarterly, 18(4), 389-408.
36
LaFreniere, P. J., & Dumas, J. E. (1995). Social competence and behaviour evaluation (preschool
edition). Los Angeles, CA: Western Psychological Services.
LaFreniere, P. J., & Dumas, J. E. (1996). Social Competence and Behavior Evaluation in children
ages 3 to 6 years: The Short Form (SCBE-30). Psychological Assessment, 8, 369–377.
Laible, D. J., & Thompson, R. A. (1998). Attachment and emotional understanding in preschool
children. Developmental Psychology, 34, 1038–1045.
Lauw, M. S. M., Havighurst, S. S., Wilson, K. R., Harley, A. E., & Northam, E. A. (2014). Improving
Parenting of Toddlers’ Emotions Using an Emotion Coaching Parenting Program: a Pilot Study of
Tuning in To Toddlers. Journal of Community Psychology, 42(2), 169–175.
Meins, E. (2003). Emotional Development and Early attachment relationships. In Slater,A., and
Bremner, G. Developmental psychology: An introduction. Oxford: Blackwell.
Morris, S. B. (2003). Estimating Effect Sizes From Pretest-Posttest-Control Group Designs.
Organizational Research Methods, 11, 364–386.
Nauta, M. H., Scholing, A., Rapee, R. M., Abbott, M., Spence, S. H., & Waters, A. (2004). A parent
report measure of children’s anxiety: Psychometric properties and comparison with child-report in a
clinic and normal sample. Behaviour Research and Therapy, 42, 813–839.
Noble, T., & McGrath, H. (2008). The positive educational practices framework: A tool for
facilitating the work of educational psychologists in promoting pupil wellbeing. Educational and
Child Psychology, 25, 119–134.
Sellman, E. (2009). Lessons learned: student voice at a school for pupils experiencing social,
emotional and behavioural difficulties. Emotional and Behavioural Difficulties, 14(January 2015),
33–48.
37
Shields, A., & Cicchetti, D. (1999). Emotion Regulation Checklist. Unpublished questionnaire, Mt.
Hope Family Center, University of Rochester, NY.
Spence, S. H. (1998). A measure of anxiety symptoms among children. Behaviour Research and
Therapy, 36, 545–566.
Wilson, K. R., Havighurst, S. S., & Harley, A. E. (2012). Tuning in to Kids: an effectiveness trial of a
parenting program targeting emotion socialization of preschoolers. Journal of Family Psychology :
JFP : Journal of the Division of Family Psychology of the American Psychological Association
(Division 43), 26(1), 56–65.
38
Appendices
Appendix A – Excluded studies
Excluded Studies – excluded at full text
Studies
Havighurst, S. S., Kehoe, C. E., & Harley,
A. E. (Under review). Tuning in to Teens:
Improving Parental Responses to Anger
and Reducing Youth Externalizing
Behavior Problems. Development and
Psychopathology.
Kehoe, C. E., Havighurst, S. S., & Harley,
A. E. (Early view). Somatic complaints in
early adolescence: The role of parents’
emotion socialisation. Journal of Early
Adolescence.
Reason for exclusion
6b – This study used the same sample as
the Tuning into Teens study by Kehoe et
al (2014) which looked at internalizing
behaviour.
6b – This study also used the same
sample as the Tuning into teems study by
Kehoe et al (2014).
39
Appendix B – Included studies
Included studies
Havighurst, S. S., Harley, A., & Prior, M. (2004). Building Preschool Children’s Emotional Competence:
A Parenting Program. Early Education & Development, 15(4), 423–448.
doi:10.1207/s15566935eed1504_5
Havighurst, S. S., Wilson, K. R., Harley, A. E., Kehoe, C., Efron, D., & Prior, M. R. (2013). “Tuning into
Kids”: reducing young children’s behavior problems using an emotion coaching parenting program.
Child Psychiatry and Human Development, 44(2), 247–64.
Havighurst, S. S., Wilson, K. R., Harley, A. E., & Prior, M. R. (2009). Tuning in to kids: an emotionfocused parenting program-initial findings from a community trial. Journal of Community
Psychology, 37(8), 1008–1023. doi:10.1002/jcop.20345
Havighurst, S. S., Wilson, K. R., Harley, A. E., Prior, M. R., & Kehoe, C. (2010). Tuning in to Kids:
improving emotion socialization practices in parents of preschool children--findings from a
community trial. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 51(12),
1342–50.
Kehoe, C. E., Havighurst, S. S., & Harley, A. E. (2014). Tuning in to Teens: Improving Parent Emotion
Socialization to Reduce Youth Internalizing Difficulties. Social Development, 23(2), 413–431.
Lauw, M. S. M., Havighurst, S. S., Wilson, K. R., Harley, A. E., & Northam, E. a. (2014). Improving
Parenting of Toddlers’ Emotions Using an Emotion Coaching Parenting Program: a Pilot Study of
Tuning in To Toddlers. Journal of Community Psychology, 42(2), 169–175.
Wilson, K. R., Havighurst, S. S., & Harley, A. E. (2012). Tuning in to Kids: an effectiveness trial of a
parenting program targeting emotion socialization of preschoolers. Journal of Family Psychology :
JFP : Journal of the Division of Family Psychology of the American Psychological Association
(Division 43), 26(1), 56–65.
40
Appendix C – Mapping the field
A - Information
Author and Aim (relevant to review)
1) A – Havighurst, Wilson, Harley and Prior (2009) –
To report on an initial evaluation of a communitybased parenting program teaching skills to parents
that impact on children’s emotional competence and
behaviour
Sample size
a) 218
Intervention
106
Control - 111
–
Sample Characteristics (age, gender, any
presenting difficulties)
a) Primary caregivers - Mothers (n=209)
and Fathers (n=9) (Mean age=36.52,
SD=4.98) and their children (age = 4-5.11
years)
Country
Intervention
A+B)
Australia
A + B) Tuning into Kids intervention. Six
two-hours a week session delivered to
parents (average – 10 parents per group)
by trained facilitators.
Control group – waitlist control who were
offered a 10-month delayed start for the
intervention
B – Havighurst, Wilson, Harley, Prior and Kehoe
(2010) (Follow up study of Havighurst et al, 2009)
To consider whether the Tuning into Kids
(TIK) intervention resulted in children’s
emotion competence and behaviour
b) 216
Intervention
106
Control - 110
–
b) Primary caregivers - Mothers (n=207)
and Fathers (n=9) (Mean age=36.57,
SD=4.97) and their children (mean age =
56.28 months, SD= 4.59 months)
Community sample obtained from preschools.
2) Havighurst, Wilson, Harley, Kehoe, Efron and
Prior (2013)
Would emotion socialisation factors related
to children’s emotion competence and
behaviour improve after parents participated
in the TIK program?
Would the TIK program then improve
children’s emotion knowledge and reduce
their behaviour problems?
54
Intervention – 31
Control - 23
Primary caregivers (all mothers, Mean age=
35.66, SD =6.73) of children (mean age = 59.31
months, SD= 7.38) attending a behaviour clinic
as they presented with externalising behaviour
difficulties.
Australia
Tuning into Kids intervention. Six twohours a week session delivered to
parents (8-14 parents per group) by
trained facilitators.
Intervention groups consisted of part of
the clinical sample alongside another
community sample.
Control group – treatment as usual group
who received paediatric treatment
involving guidance on behavioural
strategies, speech and language,
psychology and occupational therapy
where needed.
41
3)
Kehoe, Havighurst and Harley (2014)
Does the Tuning into Teens program reduce
youth internalizing difficulties?
225
121
(intervention)
Primary caregivers (200 mothers, 25 fathers,
mean age=44.1, SD=5.13) of adolescents
(mean age= 12.01, SD= .42)
Australia
Community sample obtained from schools
4)
5)
6)
Lauw, Havighurst, Wilson and Harley (2014)
To determine whether an emotion-coaching
intervention, with adaptations to address the
toddler developmental stage, would be worth
investigating further
Wilson, Havighurst and Harley (2012)
To evaluate the effectiveness of the Tuning
into Kids program under real-world
conditions.
Havighurst, Harley and Prior (2004)
A program teaching parenting skills in
emotional awareness, acceptance and
coaching would lead to an improvement in
emotion competence and reduction in child
behaviour problems
104 (control)
34
Pre and post
measures design
128
62 (intervention)
66 (control)
47
Pre and post
measures design
Primary caregivers (mothers, mean age=35.91,
SD=3.36) and their toddlers (mean age= 25.37
months, SD=6.15)
Tuning into Teens (TINT) – an adapted
version of TIK aimed at parents of
adolescents.
Control group – no intervention
Australia
Tuning into Toddlers (TOTS) – an
adapted version of TIK aimed at parents
of toddlers
Sample obtained from Maternal and Child
Health centres
Primary caregivers (118 mothers, 10 fathers,
mean age= 36.3, SD=4.3) and their children
(mean age=4.19, SD=.41)
Australia
Tuning into Kids program
Community sample obtained from preschools.
Primary caregivers (43 mothers and 4 fathers)
and their children (aged between 4 and 5 years).
Australia
Control group – waitlist control group
A parenting program delivered in six, twohour sessions per week.
Sample obtained from kindergarten classes.
42
B - Methods
Author and Aim (relevant to review)
1) A – Havighurst, Wilson, Harley and
Prior (2009) – To report on an initial
evaluation of a community-based
parenting program teaching skills to
parents that impact on children’s
emotional competence and behaviour
B - Havighurst, Wilson, Harley, Prior and
Kehoe (2010) (Follow up study of Havighurst
et al, 2009)
To consider whether the Tuning
into Kids (TIK) intervention
resulted in children’s emotion
competence and behaviour
2) Havighurst, Wilson, Harley, Kehoe,
Efron and Prior (2013)
Would emotion socialisation
factors related to children’s
emotion
competence
and
behaviour improve after parents
participated in the TIK program?
Would the TIK program then
improve children’s emotion
knowledge and reduce their
behaviour problems?
Measures
(Child measures only)
A) The Eyberg Child Behaviour
Inventory 6 (ECBI; Eyberg and
Pincus, 1999). This is a 36-item
parent report scale that measures
perceptions of children’s problem
behaviours.
b) ECBI and
Sutter-Eyberg Student Behvaiour
Inventory – a teacher’s version of
the ECBI.
The Eyberg Child
Inventory 6 (ECBI)
Behaviour
Sutter-Eyberg Student Behaviour
Inventory
n.b. -Post-intervention measures
taken immediately after program
ended. Follow-up measures taken
6-month post-intervention
Analyses
a) A one way ANOVA
was used to compare
post-intervention
differences between the
control
group
and
intervention group.
b) General
Linear
Modelling
(GLM)
repeated measures was
used to analyse condition
i.e.
intervention
vs.
control across pre, post
and follow-up.
Data were analysed using
Growth Curve Modelling
(GCM) and an ANCOVA.
Outcomes
(child outcomes only)
a) Behaviour problems (intensity) –
researchers
found a significant
improvement in the intervention group
F(1, 181) = 18.39, p<.001
Follow up
Reported in 1b
b) Behaviour problems (intensity) –
Researchers found a significant interaction
between condition and time on the EBCI,
F(1, 169) = 11.14. p<.001
On the SESBI, there was also a significant
interaction between condition and time
F(1,150)=6.87, p=.02
Behaviour Outcomes - No significant
difference found between intervention and
control group for rate of improvement in
behaviour intensity, F (1,34) = -2.031,
p=.208 and behaviour problems, F(1,34) = 1.002, p=.098
Statistical
data
provided for 6-month
follow-up is in the
form of means and
standard deviations
and effect size.
ANCOVA at time 2 showed that parents in
the
intervention
condition
reported
significantly lower child behaviour intensity
than the control group, F(1,34) = 6.32, p
=.009
For the intervention
group, ECBI
Behaviour intensity
Cohen’s d = .74 and
for
behaviour
problem Cohen’s d=
1.00.
For the control group
Cohen’s d=.58 and
.55 respectively.
ANCOVAs of teacher reports on child
behaviour indicated that at follow-up,
children in the intervention condition were
perceived to have lower behaviour intensity,
F(1,26) = 4.87, p=.036, partial eta squared
=.16, and fewer behaviour problems F(1,26)
= 4.87, p=.036, partial eta squared = .16
For the intervention
group,
SESBI
behaviour intensity,
Cohen’s d = .56 and
for
behaviour
problem Cohen’s d=
.46
43
3) Kehoe, Havighurst and Harley
(2014)
Does the Tuning into Teens
program
reduce
youth
internalizing difficulties?
Spence children’s anxiety scale
(SCAS; Spence, 1998). This is a
self-report measure of youth anxiety
symptoms.
Multi-level
mixed
effect
models were used to analyse
data.
Youth with parents in the intervention
condition reported significantly lower
anxiety, F(1,217.36) = -2.17, p=0.31.
n.b.
–
post
intervention
measures taken at 6month follow up only
Parents in the intervention condition
reported significantly lower youth anxiety,
F(1,215) = -4.92, p<.001.
Spence child anxiety scale for
parents (SCAS-P; Nauta et al,
2004). This a parent-report of youth
anxiety symptoms.
No significant difference was found for
youth reported depressive symptoms,
F(1,206.97) = -1.17, p=.244
Child Depression Inventory shortform child self-report (CDI:S;
Kovacs, 1981; Kovacs and Beck,
1977). This measured self-reported
youth depressive symptoms.
Parent reported youth depression was also
significantly lower for the intervention
group, F(1,215.46) = -4.06, p<.001
Child depression Inventory parent
report (CDI:P; Garber,1984). A
parent report of youth depressive
symptoms.
4) Lauw, Havighurst, Wilson and
Harley (2014)
To determine whether an
emotion-coaching intervention,
with adaptations to address the
toddler developmental stage,
would be worth investigating
further
5) Wilson, Havighurst and Harley
(2012)
To evaluate the effectiveness of
the Tuning into Kids program
under real-world conditions.
Brief Infant-Toddler Social and
Emotional Assessment (BriggsGowan and Carter,2007) – an 11item subscale measuring behaviour
problems.
n.b. pre-post intervention measures
only
The Eyberg Child Behaviour
Inventory 6 (ECBI)
The short form of the Social
Competence
and
Behaviour
Evaluation (SCBE-30; LaFreniere
and Dumas, 1995,1996) is a
teacher-report measure of social
competence, affective expression
and adjustment on 2.5-6 year old
children.
Paired samples t-tests were
used to analyse data.
Behaviour problems
Parents reported significantly lower toddler
externalising behaviour difficulties
t(1,33)=-2.14, p<.05
None
Mixed
effects
multilevel
modelling used to analyse
data.
For interaction between time and condition,
parent reported behaviour intensity were
non-significant, F(1,123.46) = 2.80, p = .97,
as were parent reports of behaviour
problem frequency F(1,123.78) = 2.68,
p=.104
n.b
–
postintervention
measures taken at 6month follow-up only
No significant effects were found for teacher
reports of anger and aggression,
F(1,117.87) = 0.04, p=.843
44
6)
Havighurst, Harley and Prior (2004)
A program teaching parenting
skills in emotional awareness,
acceptance and coaching would
lead to an improvement in
emotion
competence
and
reduction in child behaviour
problems
The Emotion Regulation Checklist
(ERC; Shields and Cicchetti, 1999).
This is a parent-report measure of
Lability-Negativity and Emotion
regulation.
The Eyberg Child Behaviour
Inventory (ECBI: Eyberg and
Robinson,1983). This is a 36-item
parent
repoprt
measure
of
perceptions of conduct problem
behaviours in children.
A MANOVA was used to
analyse data.
ECBI
A significant effect was found for Behaviour
intensity, F(2,44) = 10.62, p <.001 and
behaviour problems, F(2,44) = 9.94, p<.001
ERC
A significant effect was found for emotion
lability/negativity F(2,45)=8.06, p<.001.
Statistical
data
provided for 6-month
follow-up is in the
form of means and
standard deviations
only
No significant effects found on emotional
difficulties and emotion regulation.
Univariate Repeated measures ANOVA
showed that for the group with lower scores
on the ECBI, there were no significant
changes in ECBI intensity post intervention,
F(2,22) = 2.79, n.s.
For the group with higher scores on the
ECBI, significant improvements were found
post-intervention,
F(2,22)
=
2.79
=9.66,p<.001
45
Appendix D
Coding protocols and items removed from APA Task Force Coding Protocol by Kratochwill (2003)
Coding Protocol: Group-Based Design
Domain:
School- and community-based intervention programs for social and behavioral problems
Academic intervention programs
Family and parent intervention programs
School-wide and classroom-based programs
Comprehensive and coordinated school health services
Name of Coder(s):
Date: 27/12/14
M /D/Y
Full Study Reference in APA format: Full Study Reference in proper format:__ Havighurst, S. S., Wilson, K. R., Harley, A. E.,
Kehoe, C., Efron, D., & Prior, M. R. (2013). “Tuning into Kids”: reducing young children’s behavior problems using an emotion
coaching parenting program. Child Psychiatry and Human Development, 44(2), 247–64
Intervention Name (description from study): Tuning into Kids
Study ID Number (Unique Identifier): 03
Type of Publication: (Check one)
Book/Monograph
Journal article
Book chapter
Other (specify)
46
I. General Characteristics
A. General Design Characteristics
A1. Random assignment designs (if random assignment design, select one of the following)
A1.1
A1.2
A1.3
A1.4
Completely randomized design
Randomized block design (between-subjects variation)
Randomized block design (within-subjects variation)
Randomized hierarchical design
A2. Nonrandomized designs (if nonrandom assignment design, select one of the following)
A2.1
A2.2
A2.3
A2.4
A2.5
Nonrandomized design
Nonrandomized block design (between-participants variation)
Nonrandomized block design (within-participants variation)
Nonrandomized hierarchical design
Optional coding of Quasi-experimental designs (see Appendix C)
A3. Overall confidence of judgment on how participants were assigned (select one of the following)
A3.1
A3.2
A3.3
A3.4
A3.5
A3.6
A3.7
Very low (little basis)
Low (guess)
Moderate (weak inference)
High (strong inference)
Very high (explicitly stated)
N/A
Unknown/unable to code
B. Statistical Treatment/Data Analysis (answer B1 through B6)
B1. Appropriate unit of analysis
B2. Familywise error rate controlled
B3. Sufficiently large N
Statistical
Test:Growth
Curve
Modelling
and
ANCOVA
_ level:
ES: .5
N required: 64
yes
no
yes
no
yes
no
N/A
B4. Total size of sample (start of the study): 54
N
B5. Intervention group sample size: 31
N
B6. Control group sample size: 23
N
47
C. Type of Program (select one)
C1.
C2.
C3.
C4.
C5.
Universal prevention program
Selective prevention program
Targeted prevention program
Intervention/Treatment
Unknown
D. Stage of the Program (select one)
D1.
D2.
D3.
D4.
Model/demonstration programs
Early stage programs
Established/institutionalized programs
Unknown
E. Concurrent or Historical Intervention Exposure (select one)
E1.
E2.
E3.
II.
Current exposure
Prior exposure
Unknown
Key Features for Coding Studies and Rating Level of Evidence/ Support
(3=Strong Evidence 2=Promising Evidence
1=Weak Evidence
0=No Evidence)
A. Measurement (answer A1 through A4)
A1. Use of outcome measures that produce reliable scores for the majority of primary outcomes. The table for
Primary/Secondary Outcomes Statistically Significant allows for listing separate outcomes and will facilitate decision making
regarding measurement (select one of the following)
1.1 Yes
1.2
A1.2 No
A1.3 Unknown/unable to code
A2. Multi-method (select one of the following)
A2.1
A2.2
A2.3
A2.4
Yes
No
N/A
Unknown/unable to code
A3. Multi-source (select one of the following)
A3.1
A3.2
A3.3
A3.4
Yes
No
N/A
Unknown/unable to code
48
A4. Validity of measures reported (select one of the following)
A5.1
A5.2
A5.3
A5.4
Yes validated with specific target group
In part, validated for general population only
No
Unknown/unable to code
Rating for Measurement (select 0, 1, 2, or 3):
3
2
1
0
B. Comparison Group
B1. Type of Comparison Group (select one of the following)
B1.1
B1.2
B1.3
B1.4
B1.5
B1.6
B1.7
B1.8
B1.9
B1.10
Typical contact
Typical contact (other) specify:
Attention placebo
Intervention elements placebo
Alternative intervention
PharmacotherapyB1.1
No intervention
Wait list/delayed intervention
Minimal contact
Unable to identify comparison group
B2. Overall confidence rating in judgment of type of comparison group (select one of the following)
B2.1
B2.2
B2.3
B2.4
B2.5
B2.6
Very low (little basis)
Low (guess)
Moderate (weak inference)
High (strong inference)
Very high (explicitly stated)
Unknown/Unable to code
B3. Counterbalancing of Change Agents (answer B3.1 to B3.3)
B3.1 By change agent
B3.2 Statistical
B3.3. Other
B4. Group Equivalence Established (select one of the following)
B4.1
B4.2
B4.3
B4.4
Random assignment
Posthoc matched set
Statistical matching
Post hoc test for group equivalence
B5. Equivalent Mortality (answer B5.1 through B5.3)
B5.1 Low Attrition (less than 20% for Post)
B5.2 Low Attrition (less than 30% for follow-up) 24% attrition
B5.3 Intent to intervene analysis carried out
Findings
Rating for Comparison Group (select 0, 1, 2, or 3):
3
2
1
0
49
B2. Overall confidence rating in judgment of type of comparison group (select one of the following)
B2.1
B2.2
B2.3
B2.4
B2.5
B2.6
Very low (little basis)
Low (guess)
Moderate (weak inference)
High (strong inference)
Very high (explicitly stated)
Unknown/Unable to code
B3. Counterbalancing of Change Agents (answer B3.1 to B3.3)
B3.1 By change agent
B3.2 Statistical
B3.3. Other
B4. Group Equivalence Established (select one of the following)
B4.1
B4.2
B4.3
B4.4
Random assignment
Posthoc matched set
Statistical matching
Post hoc test for group equivalence
B5. Equivalent Mortality (answer B5.1 through B5.3)
B5.1 Low Attrition (less than 20% for Post)
B5.2 Low Attrition (less than 30% for follow-up) – 24% attrition
B5.3 Intent to intervene analysis carried out
Findings
50
D. Educational/Clinical Significance
Pretest
Outcome Variables:
D1. Categorical Diagnosis
Data
Diagnostic information regarding
inclusion into the study presented:
Yes
No
Posttest
Follow Up
Positive change in
diagnostic criteria from pre
to posttest:
Positive change in
diagnostic criteria from posttest to
follow up:
Unknown
Yes
D2. Outcome Assessed via
continuous Variables
No
Unknown
Yes
Positive change in percentage of
participants showing clinical
improvement from pre to posttest:
Yes No
Unknown
Importance of behavior change is
evaluated:
Yes
No
Unknown
Importance of behavior change
from pre to posttest is evaluated
positively by individuals in direct
contact with the participant:
D4. Social Comparison:
Behavior of participant at
pre, post, and follow up is
compared to normative data
(e.g., a typical peer).
Participant’s behavior is compared
to normative data
Yes
No
No
No
Unknown
Importance of behavior change
from posttest to follow up is
evaluated positively by individuals
in direct contact with the participant:
Yes
Yes
Unknown
Positive change in percentage of
participants showing clinical
improvement from posttest to follow
up:
Yes
D3. Subjective Evaluation:
The importance of behavior
change is evaluated by
individuals in direct contact
with the participant.
No
No
Unknown
Unknown
Participant’s behavior has
improved from pre to posttest
when compared to normative data:
Participant’s behavior has improved
from posttest to follow up when
compared to normative data:
Unknown
Yes
Rating for Educational/Clinical Significance (select 0, 1, 2, or 3):
No
3
Unknown
2
Yes
1
No
Unknown
0
F. Implementation Fidelity
F1. Evidence of Acceptable Adherence (answer F1.1 throughF1.3)
F1.1 Ongoing supervision/consultation
F1.2 Coding intervention sessions/lessons or procedures
F1.3 Audio/video tape implementation (select F1.3.1 or F1.3.2):
F1.3.1 Entire intervention
F1.3.2 Part of intervention
F2. Manualization (select all that apply)
F2.1
Written material involving a detailed account of the exact procedures and
the sequence in which they are to be used
F2.2
Formal training session that includes a detailed account of the
exact procedures and the sequence in which they are to be used
F2.3
Written material involving an overview of broad principles and a description
of the intervention phases
51
F2.4
Formal or informal training session involving an overview of broad
principles and a description of the intervention phases
F3. Adaptation procedures are specified (select one)
Rating for Implementation Fidelity (select 0, 1, 2, or 3):
yes
no
3
2
unknown
1
0
H. Site of Implementation
H1. School (if school is the site, select one of the followingoptions)
H1.1
H1.2
H1.3
H1.4
H1.5
H1.6
Public
Private
Charter
University Affiliated
Alternative
Not specified/unknown
H2. Non School Site (if it is a non school site, select one of the following options)
H2.1
H2.2
H2.3
H2.4
H2.5
H2.6
H2.7
H2.8
H2.9
H2.10
H2.11
Home
University Clinic
Summer Program
Outpatient Hospital
Partial inpatient/day Intervention Program
Inpatient Hospital
Private Practice
Mental Health Center
Residential Treatment Facility
Other (specify): community settings
Unknown/insufficient information provided
I. Follow Up Assessment
Timing of follow up assessment: specify 6 months
Number of participants included in the follow up assessment: specify
41____
Consistency of assessment method used: specify Same method used
Rating for Follow Up Assessment (select 0, 1, 2, or 3):
3
2
1
0
III. Other Descriptive or Supplemental Criteria to Consider
A. External Validity Indicators
A1. Sampling procedures described in detail
yes
no
Specify rationale for selection: Participants chosen if children had elevated scores on ECBI
Specify rationale for sample size:
A1.1Inclusion/exclusion criteria specified
yes
no
52
A1.2 Inclusion/exclusion criteria similar to school practice
A1.3 Specified criteria related to concern
yes
yes
no
no
A2. Participant Characteristics Specified for Treatment and Control Group
53
Participants from
Treatment Group
Child/Student
Parent/caregiver
Teacher
School
Other
Child/Student
Parent/caregiver
Teacher
School
Other
Grade/age
35.66
Gender
Ethnicity
or Multiethnic
Ethnic
Identity
Female Australia
n
Race(s)
Acculturation
White
Pri mary
Language
SES
Family
Structure
Locale
Disability
Functional
Descriptors
Disability
Functional
Descriptors
English Worki unknow Australi
ng/mid n
a
dle
class
Child/Student
Parent/caregiver
Teacher
School
Other
Child/Student
Parent/caregiver
Teacher
School
Other
Participants from
Control Group
Child/Student
Parent/caregiver
Teacher
School
Other
Child/Student
Parent/caregiver
Teacher
School
Other
Grade/age
35.66
Gender
Ethnicity
or Multiethnic
Ethnic
Identity
Female Australia
n
Race(s)
Acculturation
White
Pri mary
Language
SES
Family
Structure
Locale
English Worki unknow Australi 35.66
ng/mid n
a
dle
class
Child/Student
Parent/caregiver
Teacher
School
Other
Child/Student
Parent/caregiver
Teacher
School
Other
A3. Details are provided regarding variables that:
A3.1 Have differential relevance for intended outcomes yes
no
Specify:
54
A3.2 Have relevance to inclusion criteria
yes
no
Specify: Participants excluded if they did not have sufficient English language skills
A5. Generalization of Effects:
A5.1 Generalization over time
A5.1.1 Evidence is provided regarding the sustainability of outcomes after intervention is
terminated yes
no
Specify: 6 month follow up measures reported
A5.1.2 Procedures for maintaining outcomes are specified
yes
no
Specify:
A5.2 Generalization across settings
A5.2.1 Evidence is provided regarding the extent to which outcomes are manifested in contexts
that are different from the intervention context
yes
no
Specify:
A5.2.2 Documentation of efforts to ensure application of intervention to other settings
yes no
55
Specify:
A5.2.3 Impact on implementers or context is sustained yes
no
Specify:
A5.3 Generalization across persons
Evidence is provided regarding the degree to which outcomes are manifested with participants who
are different than the original group of participants for with the intervention was evaluated
yes
no
Specify:
B. Length of Intervention (select B1 or B2)
B1.
Unknown/insufficient information provided
B2.
Information provided (if information is provided, specify one of the
following:)
B2.1 weeks
6
N
B2.2 months
N
B2.3 years
N
B2.4 other
N
C. Intensity/dosage of Intervention (select C1 or
C2) C1.
Unknown/insufficient information
provided
C2.
Information provided (if information is provided, specify both of the following:)
C2.1 length of intervention session 2 hours
N
C2.2 frequency of intervention session weekly
N
D. Dosage Response (select D1 or D2)
D1.
D2.
Unknown/insufficient information provided
Information provided (if information is provided, answer D2.1)
56
D2.1 Describe positive outcomes associated with higher dosage:
57
E. Program Implementer (select all that apply)
E1.
E2.
E3.
E4.
E5.
E6.
E7.
E8.
E9.
Research Staff
School Specialty Staff
Teachers
Educational Assistants
Parents
College Students
Peers
Other
Unknown/insufficient information provided
F. Characteristics of the Intervener
F1.
F2.
F3.
Highly similar to target participants on key variables (e.g., race, gender, SES)
Somewhat similar to target participants on key variables
Different from target participants on key variables
G. Intervention Style or Orientation (select all that apply)
G1.
G2.
G3.
G4.
G5.
G6.
G7.
Behavioral
Cognitive-behavioral
Experiential
Humanistic/interpersonal
Psychodynamic/insight oriented
other (specify):
Unknown/insufficient information provided
H. Cost Analysis Data (select G1 or G2)
H1.
H2.
Unknown/insufficient information provided
Information provided (if information is provided, answer H2.1)
H2.1 Estimated Cost of Implementation:
I. Training and Support Resources (select all that apply)
I1.
I2.
Simple orientation given to change agents
Training workshops conducted
# of Workshops provided
Average length of training
Who conducted training (select all that apply)
I2.1
I2.2
Project Director
Graduate/project assistants
58
I2.3
I2.3
I3.
I4.
I5.
I6.
Other (please specify):
Unknown
Ongoing technical support
Program materials obtained
Special Facilities
Other (specify):
J. Feasibility
J1. Level of difficulty in training intervention agents (select one of the following)
J1.1
J1.2
J1.3
J1.4
High
Moderate
Low
Unknown
J2. Cost to train intervention agents (specify if known):
J3. Rating of cost to train intervention agents (select one of the following)
J3.1
J3.2
J3.3
J3.4
High
Moderate
Low
Unknown
59
Summary of Evidence for Group-Based Design Studies
Overall
Evidence Rating
Indicator
Description of Evidence
Strong
Promising
Weak
No/limited evidence
NNR = No
numerical rating
or
or
0-3
Descriptive ratings
General Characteristics
General Design Characteristics
NNR
Statistical Treatment
NNR
Type of Program
NNR
Stage of Program
NNR
Concurrent/Historical Intervention Exposure
NNR
Key Features
Measurement
3
Strong
Comparison Group
3
Strong
Primary/Secondary Outcomes are
Statistically Significant
Educational/clinical significance
N/A
Identifiable Components
N/A
Implementation Fidelity
3
Replication
N/A
Site of Implementation
N/A
Follow Up Assessment Conducted
3
3
Strong
Strong
Strong
59
Appendix E
Weighting of studies
The APA Task Force Coding Protocol by Kratochwill (2003) coding protocol was used to code
each of the studies in order to generate a ‘Weight of Evidence A’ rating for included studies. The
table below shows the adaptations made to the protocol along with a rationale for these
amendments.
Items removed
Rationale
Sections I.B.7 - B.8
Studies did not use qualitative research methods.
Section II.C
The protocol was used to rate the methodological quality
of the included studies. Outcomes are examined
separately in the review
Section II.D removed
(with the exception of
Havighurst et al
2013)
All studies being reviewed (apart from one) examined the
programme as a prevention/universal program therefore
this section was not relevant for the type of studies in the
review.
Section II.E removed
The intervention components are not separated.
Section II.G removed
There was no within study replication.
Rating scale for
section II.H removed
As the review question was concerned with whether the
program would enable parents to improve behavioural
outcomes for their children, rating the site of
implementation was irrelevant in ascertaining the
methodological quality of each study. However for
information purposes, details on site of implementation
was left in.
Table in section III.A4 Receptivity of intervention by target group was not
removed
deemed necessary in determining methodological quality
nor was it reported in the studies.
60
Weight of Evidence A: Methodological Quality
This was assessed based on the guidance for methodological quality detailed in the Kratochwill
(2003) coding protocol. Studies were weighted on ‘measures’, ‘comparison group’, ‘fidelity’, and
‘follow-up’.
1. Measures
Weighting
High
Medium
Low
Descriptors
- Studies used measures that produce reliable
scores of at least .70, for the majority of
primary outcomes
- Data was collected using multiple methods,
and collected from multiple sources
- Validity is reported
- Studies used measures that produce reliable
scores of at least .70
- Data was collected using multiple methods,
and/or collected from multiple sources
- A case for validity does not need to be
presented.
- Studies used measures that produce reliable
scores of at least .50
- Data may have been collected either using
multiple methods and/or from multiple sources
however, this is not required.
- A case for validity does not need to be
presented.
2. Comparison group
Weighting
High
Medium
Descriptors
- Uses at least one type of "active"
comparison group
- Initial group equivalency must be
established
- Evidence that change agents
were counterbalanced
- Equivalent mortality and low
attrition at post, and if applicable,
at follow-up
- Uses at least a "no intervention
group" type of comparison
- There is evidence for at least two
of the following: counterbalancing
of change agents, group
equivalence established, or
61
-
Low
-
-
equivalent mortality with low
attrition.
If equivalent mortality is not
demonstrated, an intent-to
intervene analysis is conducted.
The study uses a comparison
group
At least one of the following is
present: counterbalancing of
change agents, group
equivalence established, or
equivalent mortality with low
attrition.
If equivalent mortality is not
demonstrated, an intent-tointervene analysis is conducted.
3. Fidelity
Weighting
High
Medium
Measures
- The study demonstrates strong
evidence of acceptable
adherence.
- Evidence of fidelity is measured
through at least two of the
following: ongoing
supervision/consultation, coding
sessions, or audio/video tapes,
and use of a manual.
- The “manual” is either written
materials involving a detailed
account of the exact procedures
and the sequence in which they
are to be used or formal training
session detailing exact
procedures and sequence.
- The study demonstrates
evidence of acceptable
adherence.
- Evidence of fidelity is measured
through at least one of the
following: ongoing
supervision/consultation, coding
sessions, or audio/video tapes,
and use of a manual.
- The “manual” is either written
materials involving an overview
of broad principles and a
62
Low
-
description of the intervention
phases, or formal/informal
training session involving an
overview of broad principles and
a description of the intervention
phases.
Demonstrates evidence of
acceptable adherence measured
through at least one of the above
criteria or use of a manual
4. Follow-up
Weighting
High
Medium
Low
Descriptors
- The study conducted follow up
assessments over multiple
intervals with all participants that
were included in the original
sample
- Uses similar measures used to
analyse data from primary or
secondary outcomes
- The study conducted follow up
assessments at least once with
the majority of participants that
were included in the original
sample
- Similar measures used to
analyse data from primary or
secondary outcomes
- The study conducted follow up
assessments at least once with
some participants from the
original sample.
Overall methodological quality
The following ratings were assigned to each weighting to calculate the overall methodological
quality of the studies:
High weightings = 3
Medium weightings = 2
Low weightings = 1
No rating = 0
63
The scores were averaged:
Overall Methodological Quality
High
Medium
Low
Average scores
>2.5
1.5 - 2.4
<1.4
Study
Measures
Comparison Fidelity
Group
Follow-up
Havighurst
et al (2009)
and
Havighurst
et al (2010)
Havighurst
et al (2011)
Kehoe et al
(2014)
Lauw et al
(2014)
Wilson et al
(2012)
Havighurst
et al (2004)
3
2
3
2
Overall
Quality of
Methodology
2.5
3
3
3
3
3
3
2
3
2
2.5
2
0
3
0
1.25
3
2
3
3
2.75
2
0
2
3
1.75
n.b. Havighurst et al (2009) and Havighurst et al (2010) have been weighted as one study (1a and 1b) as the
former reports preliminary post-intervention findings and the latter reports full post-intervention findings as well as
follow-up outcomes of the same sample.
64
Weight of Evidence B: Methodological relevance
This weighting is a review-specific judgement about the suitability of the evidence for answering
the review question.
Weighting
High
Medium
Low
Descriptors
An ‘active’ comparison group is used
Participants are randomly assigned to
groups
Pre, post and follow up measures are
taken for both groups.
A waitlist/no intervention comparison
group is used
Participants are randomly assigned to
groups
Pre and post or pre and follow-up
measures are taken for both groups
Pre and post measures are taken
As the TI programme is a relatively new intervention, the review question sought to review the
efficacy of the studies. Therefore in WoE B, a classic evidence hierarchy (Evans, 2003) was
used to influence descriptors for the weighting of studies. Therefore, randomised controlled
trials (RCTs) are deemed as high quality evidence as results are at lower risk of error or bias
that a study that has no control group or a sample that was not randomised to conditions. The
addition of an ‘active comparison group’ to the ‘high’ category is due to the fact that the use of
an active comparison group can show that the intervention being researched is better than the
usual/alternative form of intervention suggested for a presenting problem whereas a “no
intervention/waitlist control” group just shows that the intervention is better than not intervening
at all. Finally, pre, post and follow up measures are desirable as it shows not only the baseline
measures or participants before intervention versus immediately post-intervention it also shows
the durability of the intervention effects which adds to the evaluation of the intervention’s
efficacy.
65
Weight of Evidence C: Topic Relevance
This weighting is a review specific judgement about the relevance of the focus of the evidence
for review question
Weighting
High
Descriptors
The Tuning into Kids/Teens/Toddlers
programme is delivered by trained
psychologists.
Includes only children that are at the
clinical cut-off on the EBCI/other
behavioural measures
Medium
Behavioural measures are gathered
from parents and teachers or parents
and child/young person
The Tuning into Kids/Teens/Toddlers
programme is delivered by trained
psychologists.
Includes all children regardless of
baseline scores on ECBI/other
behavioural measures
Low
Behavioural measures are gathered
from parents and teachers or parents
and child/young person.
The Tuning into Kids/Teens/Toddlers
programme is delivered by trained
professionals
Includes all children regardless of
baseline scores on ECBI/other
behavioural measures
Behavioural measures are gathered
from parents
It is argued that TI is a specialised intervention that involves teaching parents how to emotion
coach their children using highly specialised psychological techniques as well as the ability to
teach parents about their own emotions. Therefore it is argued that the person delivering the
parent training would need to be a psychologist to be able to deliver this to the maximum effect,
whereas, although a manual is used, training professionals outside of the field to deliver this
intervention may not be as effective. Studies including children that are at the clinical cut off on
the ECBI were weighted as highest as this demonstrates that the intervention’s effectiveness
as treatment for children at the clinical level of behavioural difficulties. Finally, the if measures
were multi-source e.g. derived from parents and teachers or parents and the child/young
66
person, then this shows evidence of triangulating data which provides a richer, more valid
report of the outcomes of the intervention.
Weight of Evidence D: Overall Weight of Evidence
This is an overall assessment of the extent to which the evidence contributes to answering the
review question which is assessed by giving studies the following scores:
High weightings = 3
Medium weightings = 2
Low weightings = 1
The scores obtained were averaged to give an overall weight of evidence score
Overall Weight of Evidence
High
Medium
Low
Average scores
>2.5
1.5 - 2.4
<1.4
Studies
WoE A
WoE B
Quality of
Relevance
methodology of
Methodology
WoE D
Overall
weight of
evidence
Havighurst
et al
(2009) and
Havighurst
et al
(2010)
Havighurst
et al
(2011)
Kehoe et
al (2014)
Lauw et al
(2014)
Wilson et
al (2012)
Havighurst
et al
(2004)
High
2.5
Medium
2
WoE C
Relevance
of evidence
to the
review
question
Medium
2
High
3
High
3
High
3
High
3
High
2.5
Medium
1.25
High
2.75
Medium
1.75
Medium
2
Low
1
Medium
2
Low
1
Medium
2
Low
1
Low
1
Medium
2
Medium
2.17
Low
1.08
Medium
1.92
Medium
1.58
Medium
2.17
67
Appendix F
Full effect size table
Study
Measures
Number of
Participants
Outcomes
Effect Size
Descriptors
Pre
and
post
measures
1a.
Havighurst
et al (2009)
– pre-post
The Eyberg Child
Behaviour
Inventory 6 (ECBI;
Eyberg and Pincus,
1999
1b.
Havighurst
et al (2010)
– pre-follow
up
data
(Same study
as above)
218
1a. Behaviour problems
(intensity) – researchers
found
a
significant
improvement
in the
intervention group F(1,
181) = 18.39, p<.001
Smallmedium
Effect sizes
Pre and
measures
Effect size
descriptors
post
Pre
and
follow
up
measures
Smallmedium
ECBI
Intensity
PPC SMD = -0.44
Effect sizes
Pre and follow
up measures
ECBI
Intensity
Overall
Weight of
Evidence
Medium
PPC SMD
= - 0.41
Small
SESBI
PPC SMD = -0.2
216
1b. Behaviour problems
– Researchers found a
significant
interaction
between condition and
time on the EBCI, F(1,
169) = 11.14. p<.001
On the SESBI, there was
also
a
significant
interaction
between
condition
and
time
F(1,150)=6.87, p=.02
Havighurst
et al (2013)
The Eyberg Child
Behaviour
Inventory 6 (ECBI)
Sutter-Eyberg
Student Behaviour
Inventory
–
a
54
ANCOVA at time 2
showed that parents in
the
intervention
condition
reported
significantly lower child
behaviour intensity than
the
control
group,
F(1,34) = 6.32, p =.009,
partial eta squared =.16
ANCOVAs of teacher
reports
on
child
behaviour indicated that
ECBI
Intensity
High
ECBI
Small
Large
Large
Intensity
PPC SMD= -0.76
PPC SMD = -0.13
Problem
Problem
PPC SMD= -0.83
PPC SMD = -0.42
Smallmedium
SESBI
68
teacher’s version of
the ECBI.
Kehoe et al
(2014)
Spence
anxiety
(SCAS;
1998).
children’s
scale
Spence,
at follow-up, children in
the
intervention
condition
were
perceived to have lower
behaviour
intensity,
F(1,26) = 4.87, p=.036,
partial eta squared =.16,
and fewer behaviour
problems F(1,26) = 4.87,
p=.036,
partial
eta
squared = .16
225
Spence
child
anxiety scale for
parents (SCAS-P;
Nauta et al, 2004).
Medium
Small
PPC SMD = -0.19
SCAS-P
Medium
PPC SMD = -0.48
CDI:S
Small
However no significant
difference was found for
youth
reported
depressive symptoms,
F(1,206.97) = -1.17,
p=.244
Child
depression
Inventory
parent
report
(CDI:P;
Garber,1984)
Brief Infant-Toddler
Social
and
Emotional
Assessment
Medium
Behaviour
problem
PPC SMD= -0.5
SCAS
Parents
in
the
intervention
condition
reported
significantly
lower youth anxiety,
F(1,215)
=
-4.92,
p<.001.
Child Depression
Inventory
shortform child selfreport
(CDI:S;
Kovacs,
1981;
Kovacs and Beck,
1977).
Lauw et al
(2014)
Youth with parents in the
intervention
condition
reported
significantly
lower
anxiety,
F(1,217.36) = -2.17,
p<0.31.
Medium
Behaviour
intensity
PPC SMD= -0.58
PPC SMD= -0.13
CDI:P
PPC SMD =-0.46
Medium
Parent reported youth
depression was also
significantly lower for the
intervention
group,
F(1,215.46) = -4.06,
p<.001
34
Behaviour problems
Parents
reported
significantly
lower
Low
Toddler
behaviour
problems
69
Wilson et al
(2012)
(BITSEA; BriggsGowan
and
Carter,2007)
The Eyberg Child
Behaviour
Inventory 6 (ECBI)
The short form of
the
Social
Competence and
Behaviour
Evaluation (SCBE30; LaFreniere and
Dumas,
1995,1996)
128
toddler
externalising
behaviour difficulties
t(1,33)=-2.14, p<.05
Compared to the waitlist
control, parents in the
intervention
group
reported
significantly
lower
behaviour
problems shown by a
significant main effect for
condition, F(1,123.8) =
4.99, p=.027
However for interaction
between
time
and
condition,
parent
reported
behaviour
intensity
were
nonsignificant, F(1,123.46) =
2.80, p = .97, as were
parent
reports
of
behaviour
problem
frequency F(1,123.78) =
2.68, p=.104
Small
PP SMD = -0.26
Medium
ECBI
Small
Behaviour
intensity
Small
PPC SMD= -0.22
Behaviour
problem
PPC SMD= -0.26
SCBE-30
Anger/aggression
Small
PPC SMD= 0.02
No significant effects
were found for teacher
reports of anger and
aggression, F(1,117.87)
= 0.04, p=.843
70
Havighurst
et al (2004)
The
Emotion
Regulation
Checklist
(ERC;
Shields
and
Cicchetti, 1999)
The Eyberg Child
Behaviour
Inventory
(ECBI:
Eyberg
and
Robinson,1983).
47
A significant effect was
found for Behaviour
intensity,
F(2,44)
=
10.62, p <.001 and
behaviour
problems,
F(2,44) = 9.94, p<.001
Another significant effect
was
found
for
Oppositional
Defiant
symptoms,
Attention
Deficit
Hyperactive
Symptoms, and Conduct
Symptoms
combined,
F(2,44) = 4.23, p<.01
Univariate
Repeated
measures
ANOVA
showed that for the
group with lower scores
on the ECBI, there were
no significant changes in
ECBI
intensity
post
intervention, F(2,22) =
2.79, n.s. However for
the group with higher
scores on the ECBI,
significant
improvements
were
found post-intervention,
F(2,22)
=
2.79
=9.66,p<.001
ECBI
Medium
ECBI
Small
Small
Behaviour
intensity
PP SMD = -0.26
Behaviour
problem
PP SMD = -0.31
Smallmedium
Behaviour
Intensity
PP SMD = -0.4
Smallmedium
Behaviour
Problem
PP SMD = -0.42
ERC
Lability/negativity
ERC
Lability/negativity
Small
PP SMD = -0.31
Smallmedium
PP SMD = -0.41
Teacher
rated
behaviour
Medium
PP SMD= 0.49
71
0