Doctorate in Educational and Child Psychology
Polly Grant
Case Study 1: An Evidence-Based Practice Review Report
Theme: Interventions for children with Special Educational Needs
Are class-based oral language interventions effective in improving language
skills in children with Speech, Language and Communication Needs?
Summary
A systematic literature review was carried out to evaluate class-based interventions
for children with Speech Language and Communication Needs (SLCN). This is a
model that is increasingly being used in the UK to provide speech and language
therapy to children with SLCN. Class-based interventions refer to those delivered by
school staff (usually a teacher or teaching assistant), with support from a Speech
and Language Therapist. This model can also be referred to as ‘indirect therapy’ or
the ‘consultation model’ of speech and language therapy. The review evaluates five
studies that investigated class based interventions for children with SLCN. Studies
were evaluated using Gough’s (2007) weight of evidence framework; three were
assigned a low rating, and two achieved medium ratings. The studies assessed the
impact of the language intervention on various measures including receptive
language, expressive language, vocabulary, grammar and narrative skills. Four out
of the five studies showed that the oral language intervention had a significant
positive impact on at least one area of language. These results together with effect
sizes similar to the traditional model of speech and language therapy provide support
for the implementation of class-based oral language programmes for children with
SLCN. Recommendations on implementing the intervention and further research
needs are outlined.
Introduction
Oral language interventions
1
Doctorate in Educational and Child Psychology
Polly Grant
The oral language interventions addressed in this review focuses on the delivery of
oral language teaching within a class based setting by school staff.
While the
traditional model of therapy by Speech and Language Therapists (SLTs) “is clinic
based and provided by a single speech and language therapist to a single child”
(Law et al., 2002, p.146), the class based approach follows a consultative model for
speech and language therapy provision. In the consultative approach, the SLT acts
as consultant to school staff who implements a particular intervention. In the UK,
there is a movement from a direct intervention by SLT’s to this consultative model
(Law et al., 2002). The level of support provided by the SLT varies considerably
between interventions, some interventions offering substantial support and
professional development for school staff to deliver interventions, while others offer
none.
The content of oral language intervention programmes differ significantly between
programmes. Some programmes focus solely on vocabulary development (Steele &
Mills, 2011), while others involve a comprehensive curriculum covering vocabulary,
semantics and syntax as well as providing strategies for general support in language
development (Wilcox, Gray, Guimond & Lafferty, 2011).
Such strategies may
include: slowing the rate of speech, interacting at child’s eye level, rephrasing
questions
and
comments.
Psychological basis
The case for class-base approaches to oral language development is rooted in a
social constructivist theory of learning (Vygotsky, 1978). The adult provides the
dialogue, modelling and recasting of language in its correct form, to the child, who
learns through the process of scaffolding.
2
This social constructivist theory of
Doctorate in Educational and Child Psychology
Polly Grant
language development, purporting that language is a social process is supported by
empirical research indicating that a child’s level of exposure to language can predict
the level of language (Hoff & Naigles, 2002; Huttenlocher et al.,1991; Alston & St.
James-Roberts, 2005).
While the psychological underpinnings of the class based model do not necessarily
differ substantially from the direct therapy, or the ‘traditional model’, the former may
arguably be a more authentic reflection of the theory, with language instruction being
embedded in the natural setting.
It is conducive to Vygotsky’s (1978) ‘whole-
language’ perspective, with the language learner being immersed in language.
In addition to the social constructivist view of language learning, the theoretical basis
for class based interventions reflects a shift from a within child view of special
educational needs (SEN) towards the environmental end of the continuum.
While
SLTs tend to have a within-child view of language (Dunsmuir, Clifford & Took, 2007)
the implementation of class based models for language intervention require a more
systemic approach to the delivery of speech and language therapy services.
Rationale
The rationale for an Educational Psychologist role in class based interventions for
oral language development is two-fold. Firstly SLCN is a pervasive need, which is
correlated with various academic (Muter, Hulme, Snowling & Stevenson, 2004) and
behavioural difficulties (Botting & Conti-Ramsden, 2000; Yew & O’Kearney, 2012)
and may require a more holistic model of intervention.
Secondly, Educational
Psychology is rooted in the consultative approach, and is well placed to address the
3
Doctorate in Educational and Child Psychology
Polly Grant
systemic, class, group and individual level interventions that are required to
adequately address SLCN. Each of these reasons are discussed below.
SLCN is the most common childhood SEN. In the UK in 2013, SLCN accounted for
30.6% of the school action plus and statemented cases in primary schools
(Department of Education, 2013). There are correlations between SCLN and social
and emotional difficulties (Botting & Conti-Ramsden, 2000; Bradshaw &Tipping,
2010), school exclusion (Ripley & Yuill, 2005), literacy (Muter et al., 2004) and wider
academic achievement (Aram & Nation, 1980).
The pervasiveness and potentially detrimental effects of untreated SCLN calls for a
more integrated and systemic approach to the disorder, with Educational
Psychologists working together with SLTs to address the issue.
This view is
supported by the Bercow report (2008) which highlighted unacceptable variation and
a lack of equity in the provision of speech and language services for children.
The
report called for the exploration of the consultative model for speech and language
services and recommended collaborative working between education and health
services, and the development of more integrated service provision (Bercow, 2008).
Review Question
“Are class-based oral language interventions effective in improving language skills
of children with speech language and communication needs”.
Critical Review of the Evidence Base
A literature search was conducted on December 17th 2013 using the databases
PsycInfo, ERIC and Medline. The search was conducted using the terms listed in
4
Doctorate in Educational and Child Psychology
Polly Grant
Table 1. Database Search Terms
Intervention
Setting
Participant Group
Intervention
Early years
Language delay
Efficacy
Nursery
SLCN
Effectiveness
Pre-school
SLI
Preschool
Language Impairment
school
Speech language and
kindergarten
Communication need.
Pre-kindergarten
Each category of terms was searched individually using the “abstract” field. For
example, the first category searched the terms ‘intervention’ or ‘efficacy’ or
‘effectiveness’. The categories were subsequently combined, from the search
history, using the option ‘and’.
The search resulted in 345 articles from Psychinfo, 305 in ERIC, and 237 in Medline.
Some of the studies overlapped between databases. The title and, if necessary
abstract of all of these articles were reviewed and those not meeting the inclusion
criteria specified in Table 2, were excluded.
This resulted in 24 studies which
required an inspection of the full article. Of these studies, 19 did not meet the
inclusion criteria, resulting in a total of five for inclusion in this review. Figure I
provides a flow chart to illustrate this process. A list of the excluded articles and the
reason for their exclusion can be found in Appendix 1.
5
Doctorate in Educational and Child Psychology
Polly Grant
Figure I. Literature screening process
Studies identified from Psych INFO,
ERIC and Medline
PSYCHINFO
Article titles and if necessary,
abstract screened using
inclusion and exclusion
criteria.
N=345
ERIC
Article titles and if necessary,
abstract screened using
inclusion and exclusion
criteria
N=305
Full Articles screened using inclusion
and exclusion Criteria.
N= 24
Studies included in the final review.
N=5
6
Medline
Article titles and, if necessary
abstract screened using
inclusion and exclusion
criteria
N=237
Doctorate in Educational and Child Psychology
Polly Grant
Table 2. Inclusion and exclusion criteria for studies
Inclusion
criteria
1. Publication
type
Inclusion Criteria
Exclusion criteria
Rationale
In a peer reviewed journal
Not in a peer reviewed journal (i.e.
Dissertations)
These papers have undergone a level of scrutiny
2. Language
Available in the English
language
Published in a language other than
English
There is no funding for translations
3. Type of
study
Collected primary quantitative
data
Secondary data
The use of primary data ensures that any change in
language ability is due to the intervention under
investigation
4. Design
Group Design
Single case study
The use of group design allows for external variables to be
controlled (ie. effects of maturation can be limited)
5. Measures.
Uses one or more language
assessment tools
Does not use a language assessment
tool
The study addresses the impact on language; a language
assessment tool is the only appropriate measure
6. Sample
Children aged 2-11 years
People not aged 2-11 years
The review aims to address children that are in primary
and nursery school settings
7. Setting
In a mainstream classroom
Interventions that are delivered at
home, in a special school
Allows for some level of homogeneity so that studies can
be compared
8. Intervention
Oral language programme
delivered by staff without a
specialism in speech and
language therapy
SLT led programme, SLT assistant led
programmes, parent-led, researcherled, Computer based programmes
The class-based model is the model currently being
employed by many schools and nurseries in the UK
9. Participants
Speech delayed, specific
language impairment is primary
disorder in primary language
Speech and Language impairment is a
secondary disorder. Children with
difficulty in acquiring second language
Allows for some level of homogeneity among participants.
Difficulties in acquiring a second language follow a
different trajectory
7
Doctorate in Educational and Child Psychology
Polly Grant
Table 3. Selected studies for inclusion in review
Bickford‐Smith, A., Wijayatilake, L., & Woods, G. (2005). Evaluating the
Effectiveness of an Early Years Language Intervention. Educational Psychology
in Practice, 21(3), 161–173.
Gallagher, A. L. & Chiat, S. (2009). Evaluation of speech and language therapy
interventions for pre-school children with specific language impairment: a
comparison of outcomes following specialist intensive, nursery-based and no
intervention. International Journal of Language & Communication Disorders,
44(5), 616–38.
McCartney, E., Boyle, J., Ellis, S., Bannatyne, S., & Turnbull, M. (2011). Indirect
language therapy for children with persistent language impairment in
mainstream primary schools: outcomes from a cohort intervention. International
Journal of Language & Communication Disorders, 46(1), 74–82.
Mecrow, C., Beckwith, J., & Klee, T. (2010). An exploratory trial of the effectiveness
of an enhanced consultative approach to delivering speech and language
intervention in schools. International Journal of Language & Communication
Disorders, 45(3), 354–67.
Wilcox, M. J., Gray, S. I., Guimond, A. B., & Lafferty, A. E. (2011). Efficacy of the
TELL language and literacy curriculum for pre-schoolers with developmental
speech and / or language impairment. Early Child Research Quarterly, 26, 278–
294.
A summary of all of the included studies can be found in Appendix 2.
Critical review of Studies
The studies were coded for quality using the UCL Educational Psychology Literature
Review Coding Protocol, adapted from APA Task Force on Evidence Based
Interventions in School Psychology (Kratochwill, 2003). The studies were evaluated
using the Gough (2007) Weight of evidence framework which uses three categories
with which to assess the studies: Quality of Methodology (Weight of Evidence A),
Relevance of Methodology (Weight of Evidence B) and Relevance of Evidence to the
8
Doctorate in Educational and Child Psychology
Polly Grant
Review Question (Weight of Evidence C). The average score in each of these three
categories is taken to provide an Overall Weight of Evidence (Weight of Evidence D).
Appendix 3 contains information on the criteria for the weight of evidence in each
category in addition to information on how WOE A was calculated for each study.
The weight of evidence of each of the studies in this review is found in Table 4.
Table 4. Overall weight of evidence
Reviewed
Study
WoE A
Methodological
Quality
Low
WoE B
Methodological
Relevance
Low
WoE C
Relevance to the
Review Question
Low
WoE D
Gallagher
and Chiat
(2009)
Low
Medium
Medium
Medium
McCartney
et al.(2011)
Low
Medium
Low
Low
Mecrow et
al. (2010)
Low
Low
Medium
Low
Wilcox et
al. (2011)
Medium
Medium
High
Medium
BickfordSmith et al.
(2005)
Low
Participants
One of the studies (Wilcox et al., 2011) was conducted in Arizona, in the United
States, one in Scotland (McCartney, Boyle, Ellis, Bannatyne & Turnbull, 2011), two
in London (Gallagher & Chiat, 2009, and Bickford-Smith Wijayatilake, & Woods,
2005), and one in Newcastle, UK (Mecrow, Beckwith, & Klee, 2010).
The specific nature of the SLCN varied considerably between the studies as did the
method of selecting participants. Three of the studies identified participants as those
9
Doctorate in Educational and Child Psychology
Polly Grant
who already had an SLCN diagnosis: Mecrow et al. (2010) selected children who
were on School Action Plus, for an SLCN prior to the study. Gallagher and Chiat
(2009) selected children who were on the waiting list for Speech and Language
services, and Wilcox et al. (2011) selected children who had a Delayed Speech and
Language Impairment (DSLI) as identified by the school district.
One study
(Bickford- Smith et al., 2005) identified the school (which was the result of a Head
Teacher request for Educational Psychology services) and subsequently tested all
children in the nursery, classifying those with ‘delayed language’ as those who
scored one standard deviation below the mean.
While the Gallaher and Chiat
(2009) and Mecrow et al., (2010) studies used the need for SLT involvement as their
criteria, the McCartney et al. (2011) study included those with an SLI diagnosis and
scored 1.5 standard deviations below the mean yet did not require the specialist
skills and knowledge of an SLT.
Three studies (Bickford-Smith et al., 2005;
Gallagher and Chiat, 2009; Wilcox et al., 2011) identified preschool age participants
(3-5 years)
whereas the remaining two (McCartney et al., 2011; Mecrow et al.,
2010) used children aged 4 to 6.
All studies reported SLCN as the only SEN of participants. In addition, three of the
studies (McCartney et al., 2011; Wilcox et al., 2011; Bickford-Smith et al., 2005)
specifically screened for non-verbal skills, and excluded children below a floor level
in this area, thereby ensuring the verbal domain as the sole area of need. Gallagher
and Chiat (2009) did not use nonverbal test scores to exclude participants; however,
it was clear that none of the children had an ASD diagnosis, no EBD, neural damage
or hearing damage. Mecrow et al. (2010) did not provide information on the nonverbal abilities of their participants.
10
Doctorate in Educational and Child Psychology
Polly Grant
Design
All five papers used a quantitative approach, and each took pre and post measures
of the groups before and after the intervention. With the exception of Mecrow et al.
(2010) all studies used a between participants design, with Mecrow et al. (2010)
using pre and post measures within one group of participants (with each participant
acting as their own control). Of the four studies using between participants design,
two studies used a no intervention group (Bickford-Smith et al., 2005 and Wilcox et
al., 2011) which contributed to their low to medium ratings in WOE B. One study
(McCartney et al., 2011) used the ‘traditional’ Speech & Language therapy model as
the control. And one study (Gallagher & Chiat, 2009), applied a two level approach,
using one control group, as well as a nursery based intervention and a speech and
language therapy group which contributed to a higher rating on WOE B. None of the
studies counterbalanced the change agents; this was due to the fact that they were
class based interventions.
The Gallagher and Chiat (2009) study was the only one that randomly assigned
participants to a control and intervention groups. The Wilcox et al. (2011) study
randomly assigned classes of children to control or intervention conditions, which
resulted in a lower weight of evidence as effects may have been compounded by
other factors within the conditions. McCartney et al. (2011) used a historical cohort
control, and Bickford-Smith et al. (2005) used the morning session of the nursery as
the intervention, and the afternoon session as the control.
A power analysis was conducted to determine if the studies had sufficient number of
participants to detect a significant effect. This criteria is included in WOE A. To
determine if the study had sufficient participants, the researcher first reviewed
11
Doctorate in Educational and Child Psychology
Polly Grant
previous meta-analyses of language interventions to determine the expected effect
size of such an intervention. As there were no prior meta analyses on class based
interventions for oral language, various other studies were reviewed. The metaanalyses by Law, Garrett and Nye (2004) provided average effect sizes across
language outcomes (phonology, vocabulary, syntax) as d= 0.27.
interventions provided by speech and language therapists.
This was for
Roberts and Kaiser
(2010) conducted a meta-analyses of parent–led interventions for children with
SLCN and found an average effect size of d= 0.52 across the outcomes (expressive
language, receptive language, vocabulary). Finally, while no meta-analyses have
yet been conducted on class based interventions, a study by Fricke, Bower-Crane,
Hayley, Hulme and Snowling (2013) reported effect sizes of d= 0.49 across the
outcome measures (CELF scores, grammar, comprehension, narrative). Based on
this research, it was concluded that one would expect a small to medium effect for
oral language based interventions.
Intervention
The intervention quality was assessed using the following factors: time allocated to
the intervention, the degree to which the approach is manualised, breadth of content,
and treatment fidelity. These are some of the factors have been outlined by Law et
al. (2010) as important to consider in evaluating language interventions. The quality
of the intervention is reflected in WOE C.
Time: The least intensive programmes with regard to time allocation were Gallagher
and Chiat (2009) which implemented the programme once per week, and McCartney
et al. (2011) which implemented the programme one-two times per week. This
contributed to low to medium ratings on WOE C for these studies respectively. The
12
Doctorate in Educational and Child Psychology
Polly Grant
remaining studies implemented the programme four to five times per week varying
from 20 minutes Bickford -Smith (2005) to full day sessions (Wilcox et al., 2011). It
is acknowledged that in the Wilcox et al. (2011) intervention, parts of the full day
session were dedicated to literacy skills, and it was not solely an oral language
intervention.
Programme Manual: Two of these studies had a manual for the intervention: Wilcox
et al. (2011) provide a comprehensive curriculum package which was complete with
books, props, lesson plans and professional development plans.
McCartney et
al.(2011) also contained a less prescriptive manual with activities. While it was not
manualised the intervention in the Gallagher and Chiat (2009)
study provided
detailed plans of the lessons, and highlighted the difference between the class based
intervention and the intensive (speech and language therapist led) groups.
Breadth of Content: Four of the interventions reviewed had a relatively wide breadth
of content which focused on more than two elements of language, this was reflected
in the medium to high ratings in WOE C in all four studies. McCartney et al. (2011):
supported comprehension monitoring, and the development of vocabulary, grammar
and narrative. Gallagher and Chiat (2009) included: modelling, sentence recasting,
imitation. Wilcox et al. (2011) targeted vocabulary and complex language (sentence
length and complexity). Bickford-Smith et al. (2005) had the least breadth of content
with a focus only on vocabulary acquisition. The intervention aimed to teach ten
words per week. This was reflected in the low rating in WOE C.
Fidelity: Wilcox et al. (2011) and Gallagher and Chiat (2009) were the only studies
to report fidelity in implementing the programme which contributed to their medium to
high ratings in WOE C. The remaining three did not report the level to which the
13
Doctorate in Educational and Child Psychology
Polly Grant
programme was implemented thereby lowering the rating in WOE C for these
studies.
Outcome Measures
Outcome measures differed for each study, although the CELF was the most
common measurement, with four of the studies using a version of this (CELFP2UK,
CELF 3UK). None of the studies reported the validity of the CELF as a language
measure. The CELF is however a well-known and widely used assessment tool and
has been standardised for the UK population.
Outcomes were categorised
according to the area of language that was being measured: receptive vocabulary,
receptive grammar, receptive language (overall), expressive vocabulary, expressive
grammar, expressive, narrative, expressive (overall).
Effect sizes for each study were calculated using the formula provided by Becker
(1988), measuring within person change. This method was selected in order that
effect sizes were consistent across studies. While some studies provided effect
sizes, all effect sizes were recalculated by the researcher to ensure accurate
comparisons. Table 5 shows the outcomes for each study and indicates the effect
size for each outcome measure. Effect size ratings of small, medium or large was
based on Cohen’s criteria (Cohen, 1988).
14
Doctorate in Educational and Child Psychology
Polly Grant
Table 5. Effect sizes for each language outcome measure
Authors
Outcome measures
Effect size
Significantly
Significantly
Weight of
higher than
lower than
Evidence
no
SLT
intervention?
Bickford –Smith et
al. (2005)
Gallagher and
Chiat (2009)
Receptive Language
 Vocabulary
 Grammar
Expressive Language
 Vocabulary
Receptive Language
 Vocabulary
 Grammar
Expressive Language
 Vocabulary
 Grammar
 Expressive Language
Overall
Language Overall
(parent questionnaire)
Small
Large
No
Yes
Low
N/A
Medium
No
None
Large
No
Yes
Large
Large
Small
No
No
No
Yes
Yes
Yes
No
Yes
No
None
Not reported
Not reported
Medium
McCartney et al.
Receptive Language Overall
Expressive Language
Overall
Total Language Overall:
(2011)
Mecrow et al.
(2010)
Wilcox et al.
(2011)
Receptive Language
 Vocabulary
 Grammar
Expressive Language
 Vocabulary
 Grammar
 Narrative
Receptive Language
 Receptive Language
Overall
 Receptive Vocab (Spring
measure)
Expressive Language
 Narrative
 Expressive Vocab (Spring
measure)
 Overall expressive
language
Medium
Small
No
No
N/A
Low
Not reported
Small
Small
Small/med**
N/A
Small*
Medium*
Small
Medium
No
Medium
Yes
Small-Large.
Large
Yes
Yes
Small
Yes
*p<0.05: within group change
**p<0.01 within group change
15
N/A
N/A
Low
Medium
Doctorate in Educational and Child Psychology
Polly Grant
Primary Outcome
Vocabulary
Four studies included vocabulary as one of the outcome measures; all showed small
to large effect sizes. Effect sizes reported for vocabulary for the two meta-analyses
were within the small to medium range (Law et al., 2004; Roberts & Kaiser, 2010).
One explanation for the large variations in effect sizes for the vocabulary outcome
measure may be attributed to the different methods of measurement. Wilcox et al.
(2011) used the VOCAB measure, which was developed specifically for that
investigation. While Wilcox et al. (2011) did report a moderate correlation for this
measure with standardised tests (the CELF), the correlation is for overall language
score, not specifically vocabulary.
In the Gallagher and Chiat (2009) study, the
outcome measure for vocabulary was the Renfrew Action Picture test (expressive)
and the BPVS (receptive), the reliability and validity of which was not reported in the
paper. The studies reporting small to medium effect sizes all used the CELF.
While all studies reported small to medium effect sizes for vocabulary, only two
reported effects to a statistically significant level. Wilcox et al. (2011) reported a
significant difference between groups in the area of vocabulary (p<0.0001), and
Mecrow et al. (2010) reported significant differences between groups for expressive
vocabulary.
The nature of the vocabulary intervention differed between studies with some
providing explicit vocabulary teaching in addition to embedding new words in the
context of stories (Wilcox et al., 2011), while others did not provide direct instruction
in vocabulary (Gallagher & Chiat, 2009). The intervention described in the Bickford16
Doctorate in Educational and Child Psychology
Polly Grant
Smith et al. (2005) study also appeared to address the semantic elements of
vocabulary, however little information is provided on the intervention, as reflected in
the low weighting in WOE C. In their meta-analysis on learning vocabulary, Stahl
and Fairbanks (1986) found that for vocabulary learning to be successful, the
intervention must include: definitional and contextual information. in-depth meaning
and multiple repetitions or exposures to new words. Of the reviewed studies, only
the intervention in the Wilcox et al. (2011) involved the teaching of vocabulary this
thoroughly.
Only the studies that achieved a ‘Medium’ for overall weight of evidence were also
able to achieve large effect sizes; Gallagher and Chiat (2009) for expressive
vocabulary and Wilcox et al. (2011), also for expressive vocabulary. The common
features that these studies shared in achieving the medium weight of evidence was
the intensity of the language intervention, both of which featured heavily in
vocabulary.
Interestingly however this explanation does not hold for receptive
vocabulary; the intensity of the intervention may have resulted in children feeling
more confident in their expressive language which may have led to higher scores.
Grammar
Grammar showed a medium to large effect across studies, and was the only
outcome measure to provided significant results across studies, both between
(Bickford-Smith et al., 2005; Gallagher & Chiat, 2009), and within group participants,
(Mecrow et al., 2010). The effect sizes and significant results are not related to the
quality of the study; two of the three were deemed to be of low quality. No trends
could be established between the quality of the study and the outcomes with regard
to grammar, as only one of the two ‘medium’ rated studies included a measure of
17
Doctorate in Educational and Child Psychology
Polly Grant
grammar. As two of the studies that reported the significant results (Bickford-Smith
et al., 2005 and Mecrow et al., 2010) were of low quality the results should be
interpreted with caution.
With regard to the nature of the interventions, neither the Gallagher and Chiat (2009)
nor the Bickford-Smith et al. (2005) study, include any explicit grammar teaching as
part of the intervention. Grammar may have been indirectly addressed in story
retelling part of the intervention (Gallaher & Chiat, 2009), and the linguistic concepts
activities. This may suggest that children with SLCN are able to learn correct
grammatical structures without explicit teaching, if they are provided with sufficient
modelled language.
Narrative
Only two studies included expressive narrative language as an outcome measure
(Mecrow et al., 2010 and Wilcox et al., 2011), one of which found significant
differences between participants (Wilcox et al., 2011) and the other no significant
changes were reported (Mecrow et al., 2010). The effect sizes ranged from small
(Mecrow et al., 2010 and Wilcox et al., 2011) to large (Wilcox et al., 2011). The
Wilcox et al. (2011) study use three different measures for this outcome thereby
presenting a strong case for the effectiveness of the intervention on narrative skills.
The intervention in the Wilcox et al. (2011) study promoted strategies targeting the
acquisition of complex language skills which may have contributed to this
significantly large effect in this area.
As only two of the studies included a measure of narrative skills, no trend could be
established with regard to the low vs, medium weight of evidence ratings. However,
18
Doctorate in Educational and Child Psychology
Polly Grant
the Wilcox et al. (2011) study which achieved the medium rating did produce both a
significant effect, between groups, and was able to achieve large effect sizes on
some measures. This was not the case for the Mecrow et al. (2010) study which
received a ‘low’ weighting for overall weight of evidence: it did not produce a
significant result and achieved only a small effect size.
Receptive Language
A measure of receptive language, as a composite score , was reported in two of the
studies (McCartney et al., 2011 and Wilcox et al., 2011) , and both reported no
significant difference between or within groups and small to medium effect sizes.
The CELF was used in both cases.
The control group used in McCartney et al. (2011) study was a speech and language
therapy control group. The large effect size and no significant difference between
groups might suggest that class based intervention was as effective as the traditional
speech and language model. However, as the within group change for the classbased intervention was also reported as insignificant then an alternate interpretation
might be suggested: that neither class based nor speech and language therapy
interventions are effective.
The study however, was underpowered as reflected in
the WOE A.
Expressive Language
A measure of expressive language was reported in three of the studies (Gallagher &
Chiat, 2009, McCartney et al., 2011, & Wilcox et al., 2011), none of which reported
significant results, and reported small to medium effect sizes. While the Law et al.
(2004) meta-analysis did not provide an effect size for the expressive language
19
Doctorate in Educational and Child Psychology
Polly Grant
category, it is consistent with their measure of the effect of intervention on overall
language score, which was small. Roberts and Kaiser’s meta-analysis on parental
interventions for SLCN reported a medium effect for expressive language.
Conclusion
This review evaluated five studies that provided class-based interventions for
children with SLCN. Three studies were rated low on the overall quality of evidence,
while two achieved a medium overall weight of evidence. The weight of evidence
however did not have an overall bearing on the findings, with both the low and
medium rated studies achieving a range of no effect to large effect sizes on the
various language skill categories.
The low to medium weight of evidence for these studies reflects the difficulty in
conducting research with special populations in mainstream schools. There are a
number of factors that make comparisons between studies difficult including different
measurement tools, heterogeneous population of SLCN, varying intensity levels of
the intervention (time, breadth of content), and the lack of reporting of the fidelity of
implementation.
While there are promising results with regard to the effect the
interventions for children with SLCN, the lack of controlled variables in the studies
make it difficult to determine the most important element of the intervention in
producing the effect. Similar methodological difficulties for language interventions
have been outlined in the review by Parkinson and Humphrey (2008).
Despite the methodological weaknesses, this review finds promising support for
class-based interventions for children with SLCN. With effect sizes consistent with
more traditional models of speech and language intervention (Law et al., 2004), and
the previously established need for a more consultative approach to speech and
20
Doctorate in Educational and Child Psychology
Polly Grant
language therapy (Bercow, 2008), then class based interventions offer a viable
intervention for children with SLCN. In recommending this approach to speech and
language provision, a number of factors need to be considered. These factors are
included in the form of recommendations below.
Recommendations
In order to further understand the critical components of an intervention, and for
whom it works, there are a number of recommendations to be made for future
research:
 Information about interventions: Information regarding the precise nature of
the intervention is needed to ensure that comparisons can be made across
studies, effective elements of the intervention can be identified, and
replication is possible.
 Measures: In order to compare data across studies, a degree of consistency
in measurement is necessary.
Measurement scores also need to be
disaggregated into component parts in order to ensure comparisons are
possible.
 Follow up: Follow up measures need to be taken after the intervention to
assess if progress during the intervention is sustained. None of the studies in
the current review reported any follow up measures.
 Participants: due to the broad scope of SLCN, it is imperative that researchers
provide information about the nature of the SLCN in order to allow for relevant
comparisons, and determine whether some interventions are more relevant
for a particular need.
 Role of school staff in supporting speech and language: Further research is
needed on the most effective ways in which school staff can support speech
21
Doctorate in Educational and Child Psychology
Polly Grant
and language interventions. Law et al., (2002) reported that the consultative
model can only work if there is someone there to consult; further research is
needed to assess effective systems for this.
In conclusion, this review provides promising support for class based language
interventions and it is recommended that schools, speech and language
therapists and Educational Psychologists collaborate to implement class-based
language instruction for children with SLCN.
References
Alston, E. & St. James-Roberts, I. (2005). Home environments of 10-month-old
infants selected by the WILSTAAR screen for pre-language difficulties. International
Journal of Language & Communication Disorders, 40, 123–136
Aram, D. M. & Nation, J. E. (1980). Preschool language disorders and subsequent
language and academic difficulties. Journal of Communication Disorders, 13, 159–
170.
Baxendale, J. & Hesketh, A. (2003). Comparison of the effectiveness of the Hanen
Parent Programme and traditional clinic therapy. International Journal of Language
and Communication Disorders Oct-Dec 2003. Vol.38, No. 4 , 397-415.
Becker, B. J. (1988). Synthesizing standardized mean-change measures. British
Journal of Mathematical and Statistical Psychology ,41, 257–278.
Bickford‐Smith, A., Wijayatilake, L., & Woods, G. (2005). Evaluating the
Effectiveness of an Early Years Language Intervention. Educational Psychology in
Practice, 21(3), 161–173.
Bercow, J. (2008). The Bercow Report: A review of services for children and young
people (0–19) with speech language and communication needs. Nottingham:
Department for Children, Schools and Families.
22
Doctorate in Educational and Child Psychology
Polly Grant
Botting, N. & Conti-Ramsden, G (2000). Social and behavioural difficulties in children
with language impairment. Child Language Teaching and Therapy 16, 105–20.
Cohen, J. (1988). Statistical power analysis for the behavioural sciences. New
Jersey: Lawrence Erlbaum Associates, Inc. Publishers. pp 283-286.
Department
of
Education,
2013
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/22569
9/SFR30-2013_Text.pdf
Durkin, K.& Conti-Ramsden, G. (2010). Young people with specific language
impairment: A review of social and emotional functioning in adolescence. Child
Language Teaching and Therapy, 26, 105–121.
Fricke, S., Bowyer-Crane, C., Haley, A. J., Hulme, C., & Snowling, M. J. (2013).
Efficacy of language intervention in the early years. Journal of Child Psychology and
Psychiatry, and Allied Disciplines, 54(3), 280–90.
Gallagher, A. L. & Chiat, S. (2009). Evaluation of speech and language therapy
interventions for pre-school children with specific language impairment: a
comparison of outcomes following specialist intensive, nursery-based and no
intervention. International Journal of Language & Communication Disorders / Royal
College of Speech & Language Therapists, 44(5), 616–38.
Gough, D. (2007). Weight of Evidence: a framework for the appraisal of the quality
and relevance of evidence. Research Papers in Education, 22(2), 213–228.
Hoff, E. & Naigles, L. (2002). How children use input to acquire a lexicon. Child
Development, 73, 418–433.
Huttenlocher, J., Haight, W., Bryk, A., Seltzer, M., & Lyons, T. (1991). Early
vocabulary growth: Relation to language input and gender. Developmental
Psychology, 27, 236–248
Kratochwill, T. R. (2003). Task force on evidence-based practice interventions in
school
psychology.
Retrieved
on
January
25th
2014
from:
www.switzerland.k12.in.us/Rti/RTI%20Resources%20Galore_Updated%206.28.09/S
SSP%20EBI%20Manual.pdf
Law, J., Garrett, Z., & Nye, C. (2004). The efficacy of treatment for children with
developmental speech and language delay/disorder: a meta-analysis. Journal of
Speech, Language, and Hearing Research : JSLHR, 47(4), 924–43.
Law, J., Lee, W., Roulstone, S., Wren, Y., Zeng, B., & Lindsay, G. (n.d.). ’ What
Works ': Interventions for children and young people with speech , language and
communication needs.
23
Doctorate in Educational and Child Psychology
Polly Grant
Law, J., Lindsay, G., Peacey, N., Gascoigne, M., Soloff, N., Radford, J., & Band, S.
(2002). Consultation as a model for providing speech and language therapy in
schools: a panacea or one step too far? Child Language Teaching and Therapy,
18(2), 145–163.
McCartney, E., Boyle, J., Ellis, S., Bannatyne, S., & Turnbull, M. (2011). Indirect
language therapy for children with persistent language impairment in mainstream
primary schools: outcomes from a cohort intervention. International Journal of
Language & Communication Disorders / Royal College of Speech & Language
Therapists, 46(1), 74–82.
Mecrow, C., Beckwith, J., & Klee, T. (2010). An exploratory trial of the effectiveness
of an enhanced consultative approach to delivering speech and language
intervention in schools. International Journal of Language & Communication
Disorders / Royal College of Speech & Language Therapists, 45(3), 354–67.
Muter, V., Hulme, C., Snowling, M.J., & Stevenson, J. (2004). Phonemes, rimes,
vocabulary and grammatical skills as foundations of early reading development:
Evidence from a longitudinal study. Developmental Psychology, 40, 665–681.
Parkinson, G., & Humphrey, N. (2008). Intervention for children with language
impairments: a model of evidence-based outcome research. Journal of Research in
Special Educational Needs, 8(1), 2–12.
Ripley, K., & Yuill, N. (2005). Patterns of language impairment and behaviour in boys
excluded from school. The British Journal of Educational Psychology, 75(Pt 1), 37–
50.
Roberts, M. & Kaiser, A. P. (2011). The Effectiveness of Parent-Implemented
Language Interventions: A Meta-Analysis. American Journal of Speech-Language
Pathology, 20(August), 180–199.
Steele, S. C. & Mills, M. T. (2011). Vocabulary intervention for school-age children
with language impairment: A review of evidence and good practice. Child Language
Teaching and Therapy, 27(3), 354–370.
Stahl, S.. & Fairbanks, M.M. (1986). The effects of vocabulary instruction: A modelbased meta-analysis. Review of Educational Research 56: 72–110.
Vygotsky, L. (1978) Mind in society. Cambridge, MA: Harvard University Press.
Wilcox, M. J., Gray, S. I., Guimond, A. B., & Lafferty, A. E. (2011). Efficacy of the
TELL language and literacy curriculum for pre-schoolers with developmental speech
and / or language impairment. Early Child Research Quarterly, 26, 278–294.
Yew, S. G. K. & O’Kearney, R. (2013). Emotional and behavioural outcomes later in
childhood and adolescence for children with specific language impairments: metaanalyses of controlled prospective studies. Journal of Child Psychology and
Psychiatry, and Allied Disciplines, 54(5), 516–24.
24
Doctorate in Educational and Child Psychology
Polly Grant
Appendix 1: Excluded Studies
Excluded Studies
Reason For Exclusion
1
Boyle, J., McCartney, E., O’Hare, A., & Forbes, J. (2009).
Direct versus indirect and individual versus group modes of
language therapy for children with primary language
impairment: principal outcomes from a randomized controlled
trial and economic evaluation. International Journal of
Language & Communication Disorders, 44(6), 826–846.
Criteria 8: The
intervention was
delivered by SLT
assistants
2
Davies, P., Shanks, B., & Davies, K. (2004). Improving
narrative skills in young children with delayed language
development. Educational Review, 56(3), 271–286.
Criteria 9: Participants.
Children did not have
SLCN.
3
Dickson, K., Marshall, M., Boyle, J., McCartney, E., O’Hare, A.,
& Forbes, J. (2009). Cost analysis of direct versus indirect and
individual versus group modes of manual-based speech-andlanguage therapy for primary school-age children with primary
language impairment. International journal of language &
communication disorders / Royal College of Speech &
Language Therapists, 44(3), 369–81.
Criteria 8: The
intervention was
delivered by SLT
assistants
4
Ebbels, S. (2007). Teaching grammar to school-aged children
with specific language impairment using Shape Coding. Child
Language Teaching and Therapy, 23(1), 67–93.
Criteria 8: The
intervention was
delivered by the
researcher
5
Lafferty, A. E., Gray, S., & Wilcox, M. J. (2005). Teaching
alphabetic knowledge to pre-school children with
developmental language delay and with typical language
development. Child Language Teaching and Therapy, 21(3),
263–277.
Criteria 5: Measures.
The study did not include
language measures
6
Loeb, D. F., Gillam, R. B., Hoffman, L., Brandel, J., & Marquis,
J. (2009). The effects of Fast ForWord Language on the
phonemic awareness and reading skills of school-age children
with language impairments and poor reading skills. American
journal of speech-language pathology / American SpeechLanguage-Hearing Association, 18(4), 376–87.
Criteria 5: Measures.
The study did not include
language measures
7
Motsch, H.-J., & Riehemann, S. (2008). Effects of “ContextOptimization” on the acquisition of grammatical
case in children
25
with specific language impairment: an experimental evaluation
in the classroom. International Journal of Language &
Communication Disorders / Royal College of Speech &
Criteria 7 Setting:
Intervention was
delivered in a special
school,
Doctorate in Educational and Child Psychology
Polly Grant
Language Therapists, 43(6), 683–98.
8
Munro, N., Lee, K., & Baker, E. (2008). Building vocabulary
knowledge and phonological awareness skills in children with
specific language impairment through hybrid language
intervention : a feasibility study.
Criteria 8: The
intervention was
delivered by SLT
assistants
9
Ritter, M. J., & Saxon, T. F. (2010). Classroom-Based
Phonological Sensitivity Intervention (PSI) Using a Narrative
Platform: An Experimental Study of First Graders At Risk for a
10 .Reading Disability. Communication Disorders Quarterly, (1),
3–12.
Criteria 5: Measures.
The study did not include
language measures (only
reading measures).
10
Sajaniemi, N., Suhonen, E., & Kontu, E. (2010). Verbal and
non‐verbal development in SLI children after early intervention.
Early Child Development and Care, 180(4), 519–534.
Criteria 8: Not explicit
who the change agent
was.
11
Smith-Lock, K. M., Leitao, S., Lambert, L., & Nickels, L. (2013).
Effective intervention for expressive grammar in children with
specific language impairment. International Journal of
Language & Communication Disorders / Royal College of
Speech & Language Therapists, 48(3), 265–82
Criteria 7: Setting.
Special School
12
Smith-Lock, K., Leitão, S., Lambert, L., Prior, P., Dunn, A.,
Cronje, J., & Nickels, L. (2013). Daily or weekly? The role of
treatment frequency in the effectiveness of grammar treatment
for children with specific language impairment. International
Journal of Speech-Language Pathology, 15(3), 255–67.
Criteria 7: Setting.
Special school setting
13
Tyler, A. a, Lewis, K. E., Haskill, A., & Tolbert, L. C. (2003).
Outcomes of different speech and language goal attack
strategies. Journal of speech, language, and hearing research :
JSLHR, 46(5), 1077–94. Retrieved from
http://www.ncbi.nlm.nih.gov/pubmed/14575344
Criteria 8: The
interventions were
delivered by graduate
interns.
14
Tyler, A. a., Gillon, G., Macrae, T., & Johnson, R. L. (2011).
Direct and Indirect Effects of Stimulating Phoneme Awareness
vs. Other Linguistic Skills in Pre-schoolers With Co-occurring
Speech and Language Impairments. Topics in Language
Disorders, 31(2), 128–144.
26
Criteria 8: The
interventions were
delivered by Senior or
Master’s level speechlanguage pathology
students who were
trained on the
interventions
Doctorate in Educational and Child Psychology
Polly Grant
15
Van Kleeck, A., Vander Woude, J., & Hammett, L. (2006).
Fostering literal and inferential language skills in Head Start
pre-schoolers with language impairment using scripted booksharing discussions. American Journal of Speech-Language
Pathology / American Speech-Language-Hearing Association,
15(1), 85–95.
Criteria 8: Intervention
delivered by graduate
and undergraduate
research assistants
16
Washington, K. N. (2013). The Association Between
Expressive Grammar Intervention and Social and Emergent
Literacy Outcomes for Pre-schoolers With SLI. American
Journal of Speech Language Pathology, 22.
Criteria 8 : Intervention
by Speech and language
pathologist.
17
Yoder, P., Camarata, S., & Gardner, E. (2005). Treatment
Effects on Speech Intelligibility and Length of Utterance in
Children with Specific Language and Intelligibility Impairments.
Journal of Early Intervention, 28(1), 34–49.
Criteria 8: Intervention
delivered by a
psychology undergrad
19
Ziolkowski, R. a., & Goldstein, H. (2008). Effects of an
Embedded Phonological Awareness Intervention During
Repeated Book Reading on Preschool Children With Language
Delays. Journal of Early Intervention, 31(1), 67–90.
27
Criteria 5: Measures.
Language measures
were not used, only
reading.
Doctorate in Educational and Child Psychology
Polly Grant
Appendix 2: Summary of Studies
Bickford-Smith et al. (2005)
Design
Participants
Intervention
Number: 6 children with
delayed language. Age:
3 years five months to 4
years 4 months.
Nursery school in outer
London
2 part intervention:
1) Directly
taught
language
programme
2) The creation of a language rich
nursery environment.
The directly taught language programme
focused on teaching vocabulary (10
words per week), teaching was
reinforced with Makaton signs, rebus
symbols, real objects and photographs.
Non randomized design:
The morning group at the
nursery was designated
as the intervention group.
The afternoon group was
the control group.
(** It is unclear from the
paper how many children
were in each group).
The group was taught by a member of
the nursery staff. The staff member
delivering the program was provided
with weekly support from a speech and
language therapist.
The creation of the language rich
environment involved: visually and
verbally reinforcing target words,
expanding children’s language when
repeating back, model language to
children.
Measures
Outcomes
CELF – P subtests:
Basic Concepts
(assesses a child’s
understanding of
modifiers relating to
attributes,
Dimension, quantity and
position).
● Sentence Structure
(assesses a child’s
understanding of early
acquired sentence
formation rules).
● Formulating Labels
(assesses a child’s ability
to name pictures that
represent nouns
And verbs).
The intervention group showed
significantly more progress than the
control group on only one of the
measures: sentence structure, which
assesses receptive language.
Test on receptive and
expressive knowledge of
the first 100 words
(vocabulary test).
Parental questionnaire to
assess which of the first
100 words their child
knew
Ran for 10 weeks
28
The intervention group showed
progress in the other areas (basic
concepts and formulating labels), but
not to a significant level.
Doctorate in Educational and Child Psychology
Participants
24 children aged 3 years 6
months to 5 years, recruited from
specialist speech and language
therapy waiting list in Lambeth,
London.
Polly Grant
Gallagher and Chiat (2009)
Intervention
Design
Children received group instruction in Randomized design:
the nursery setting. The instruction Children were
was conducted by nursery staff with randomly assigned to
the support of a speech and one of three
language therapist.
intervention groups:
1) Traditional
The intervention focused on the
speech and
following areas of language:
language
Understanding and use of linguistic
therapy
concepts such as colour, size, spatial
delivered by
prepositions.
Speech and
language
Understanding
and
use
of
therapist
grammatical markers e.g. past -ed
2) Consultative
endings, plural -s.
design: class
based
Understanding and use of utterances
intervention
including increasing numbers of key
3) Control group:
information words.
no intervention
Blind assessment
Understanding and use of a list of
topic-based vocabulary.
Attention to adult-led tasks for an
increasing length of time.
Time: one hour per week in small
groups.
Speech and language
therapist involved in 12 hours of
training. Instruction implemented by
nursery staff in between.
29
Measures
Reynell developmental
scales III comprehension
subtest : measures:
 Receptive
grammar.
 Expressive
grammar
Outcomes
Nursery Based Group made
more
progress
than
no
intervention group on one
measure (comprehension of
grammar), but showed no
significant differences on any of
the other measures.
British Picture Vocabulary
Scale: measures:
 receptive
vocabulary
Renfrew word finding test:
 Expressive
vocabulary
Intensive
group
showed
significantly more progress than
the nursery based group on all
measures with the exception of
expressive grammar, where
there were no significant
differences.
Renfrew Action Picture
test: measures:
 expressive
language.
Intensive
treatment
group
showed large effects on all
measures (ranging from d=1.26
to 2.76.

Parent/carer
questionnaire.
Measures;
Parental perception of
child’s language.
Doctorate in Educational and Child Psychology
Participants
38 children aged 6-11
(intervention group). All had
a
diagnosed
language
impairment.
On entry scored less than
1.25 SD on CELF on
receptive or expressive
language
Intervention
Individualised plan carried
out for each child – based
on the materials written for
the original RCT (McCartney
2007).
Intervention
delivered by school staff:
teachers, support teachers,
assistants.
Polly Grant
McCartney et al. (2011)
Design
Measures
Randomized
control Language measures:
(with historical control
UK
group).
CELF-3
a) Expressive
language
b) Receptive
language
Total language
31 children in historical
comparison group
30
Outcomes
No significant differences pre and post
intervention on each of the tests
(receptive and expressive language).
Doctorate in Educational and Child Psychology
Participants
35 children aged 4 years 2
months and 6 years 10
months.
Identified as School Action
plus. All scored 1.5 sd
below the mean on either
the expressive or receptive
subscale of the Preschool
Language scale 3
Intervention
Individual programme was
developed for each child
depending on their need.
Focus was on: attention
and
listening,
auditory
memory, receptive and
expressive
language,
phonological
awareness,
literacy an all aspects of
speech.
Polly Grant
Mecrow et al. (2011)
Design
Measures
Within subject, quasi Probe tasks:
experimental design.
Change on a probe task that
assessed the two targets being
addressed for that child
UK
CELF P measures:
Receptive
a) Lingusitic concepts
b) Basic concepts
c) Sentence structure
Expressive language:
a) Formulating labels
b) Word structure
c) Recalling sentences
context
2 intervention and 2 control
targets were identified for
each child.
Delivered by specialist
teaching assistants under
the supervision of speech
and language therapists
st
1
target: children performed
significantly better on target
intervention
when
measured
against the control (eta squared =
0.88), and when measured in
time (baseline vs follow up) ,
partial eta squared = 0.91)
Same for the second target.
in
Parent and teacher questionnaires
to measure their impressions of the
difficulties children experienced at
school and their impression of the
progress
of
the
language
intervention
31
Outcomes
Primary measure:
Significant standard score gains
in CELF scores on receptive and
expressive composite scales.
Doctorate in Educational and Child Psychology
Participants
118
children
with
Developmental
Speech/Language Impairment (aged 47 to 64
months) from 29 preschool classes in 19
schools in Phoenix, Arizona, USA.
Control group: 38
Intervention group: 80
Their status in terms of DSLI was determined
by
school district testing and all children
demonstrated
speech
and/or
language
standard scores that were more than 1.5 SD
below the
mean on a standardized test of speech or
language
Polly Grant
Wilcox et al., (2011)
Intervention
Design
TELL
programme Randomized
(Teaching Early Literacy controlled classes
and Language):
were
assigned
Focuses on:
randomly to TELL
1) Oral language skills: or
control
vocabulary
and conditions,
not
complex
language individual children)
(sentence length and (compounding
variables).
complexity).
Code focused skills :
Phonological awareness
alphabet knowledge, print
concepts, writing.
32
Measures
VOCAB:
To
measure
receptive and expressive
vocabulary.
Outcomes
On CELF –P2 measure : No
difference between TELL and
control.
CELF –P2: measures :
a) Core language
b) Receptive language
c) Expressive language
d) Language content
e) Language structure.
BUS MLU-5 Scores higher for
TELL children (after controlling
for pre-test scores and family
income).
Renfrew
Bus
Story:
measures:
a) Mean
length
utterance
b) Information score
c) Subordinate clause
BUS Information scores – no
difference.
Doctorate in Educational and Child Psychology
Polly Grant
Appendix 3: Weight of Evidence Criteria
A. Quality of Methodology
Measures
In order to receive a high rating the studies must have:




Reported reliability of .85 or above for all primary outcomes relevant to the
review question or used well referenced measures,
Use at least 2 methods of assessment
Use at least 2 sources of assessment.
Reported validity for all measures used or use a well referenced,
standardised or norm-referenced measure.
In order to receive a medium rating, the studies must have:


Reported reliability of 0.85 for some primary outcomes or use well referenced
measures.
Use at least 2 methods of assessment or used at least 2 sources of
assessment
(The above criteria must be met for 75% of primary outcomes)
In order to receive a low rating, the studies must have:

Reported reliability of .50 or above for some primary outcomes or used well
referenced measures.

Use at least 2 methods of assessment or used at least 2 sources of
assessment
(the above criteria must be met for 50% of primary outcomes)
Comparison Group
In order to receive a high rating, the study must have:



Used an active comparison group
Demonstrated group equivalence
Evidence of equivalent mortality and low attrition
In order to receive a medium rating, the study must have:


Used a ‘no intervention’ group
Demonstrated group equivalence
33
Doctorate in Educational and Child Psychology

Polly Grant
Evidence of equivalent mortality and low attrition.
In order to receive a low rating, the study must have:


A comparison group
One of the following criteria must be met: a)group equivalence established c)
equivalent mortality with low attrition.
NOTE: This is adapted from the Kratochwill criteria in the following way: Change
agent has been removed, as the studies looked at class based instruction. The
change agent is therefore the teacher or TA, which would not be practical or
appropriate to change.
Analysis
In order to receive a high rating the study must have:




Conducted an appropriate analysis
Controlled for family wise error (when appropriate)
Provided enough information for all effect sizes to be calculated
Had a sufficiently large sample size
In order to receive a medium rating the study must have:



Conducted an appropriate analysis
Provided enough information for some effect sizes to be calculated
Had a sufficiently large sample size
In order to receive a low rating the study must have demonstrated 2 of the following



Conducted an appropriate analysis
Provided enough information for some effect sizes to be calculated
Had a sufficiently large sample size
B. Relevance of Methodology

To receive a high weighting for methodological relevance the study must
have used a random assignment of participants to either an intervention
control or alternative intervention group. They must have demonstrated group
34
Doctorate in Educational and Child Psychology
Polly Grant
equivalences. Pre and post-test measure must have been obtained for all
groups and outcome measures.

To receive a medium weighting for methodological relevance the study must
have demonstrated group equivalences. A comparison group must have been
used. Pre and post-test measure must have been obtained for all groups
involved.

To receive a low weighting, pre and post-test measure must have been
obtained for all groups involved.
C. Relevance of evidence to the review question

High- T
The intervention took place more than twice per week.
The programme ran for over 20 weeks
The study reported a high level of fidelity to the intervention
The intervention was manualised.
The intervention covered more than 2 aspects of language: e.g. Vocabulary,
narration, grammar,
Staff delivering the intervention were provided with training.

MediumThe intervention took place more than once a week
The programme ran for 11-20 weeks
The study reported the level of fidelity to the intervention
The study met at least 1 of the following conditions:
The intervention was manualised, or
The intervention covered more than 2 aspects of language: e.g.
Vocabulary, narration, grammar, or,
Staff delivering the intervention were provided with training.

LowThe intervention took place once a week or less
The programme ran for up to 10 weeks
The fidelity to the programme was not reported.
D. Overall weight of evidence: WOE D
(WOE A + WOE B + WOE C) ÷3 = WOE D
35
Doctorate in Educational and Child Psychology
Polly Grant
Appendix 4: Coding Protocols
[adapted from Task Force on Evidence-Based Interventions in School Psychology, American Psychology
Association, Kratochwill, T.R. (2003)]
Coding Protocol
Name of Coder:
ucjtpgr
Date: 15/1/14
Full Study Reference in proper format:___
Bickford‐Smith, A., Wijayatilake, L., & Woods, G.
(2005). Evaluating the Effectiveness of an Early Years Language Intervention.
Educational Psychology in Practice, 21(3), 161–173.
Intervention Name (description of study): The
intervention was designed to improve language skills
in a nursery setting. This consisted of two parts: 1) A directly taught language programme.
This consisted of material written specifically for the language delayed children in the study
and focused on teaching vocabulary. 2) A language rich environment.
Study ID Number:__________________
Type of Publication:
Book/Monograph
X Journal Article
Book Chapter
Other (specify):
1.General Characteristics
A. General Design Characteristics
A1. Random assignment designs (if random assignment design, select one of the following)
Completely randomized design
Randomized block design (between participants, e.g., matched classrooms)
Randomized block design (within participants)
Randomized hierarchical design (nested treatments
A2. Nonrandomized designs (if non-random assignment design, select one of the following)
X Nonrandomized design
Nonrandomized block design (between participants)
Nonrandomized block design (within participants)
Nonrandomized hierarchical design
Optional coding for Quasi-experimental designs
A3. Overall confidence of judgment on how participants were assigned (select on of the following)
Very low (little basis)
Low (guess)
Moderate (weak inference)
High (strong inference)
X Very high (explicitly stated)
N/A
Unknown/unable to code
36
Doctorate in Educational and Child Psychology
Polly Grant
B Participants
Total size of sample (start of study): 6
(this was the number of children with delayed language
skills). Excluded EAL and those with delay in non verbal skills).
Intervention group sample size: Not
clear from the method described. The intervention group was
the morning group at the nursery, the control group was the afternoon group. Paper did not
state how many children with language delay were in the morning and afternoon groups.
Control group sample size:
Not clear – see above explanation.
C. Type of Program
Universal prevention program
Selective prevention program
Targeted prevention program
X Intervention/Treatment
Unknown
D. Stage of Program
X Model/demonstration programs (program written specifically for the language delayed children-
Used vocabulary from Living Language, Locke 1985).
Early stage programs
Established/institutionalized programs
Unknown
E. Concurrent or Historical Intervention Exposure
Current exposure
Prior exposure
X Unknown
Section 2
Key Features for Coding Studies and Rating Level of Evidence/Support
A Measurement (Estimating the quality of the measures used to establish effects)
(Rating Scale: 3= Strong Evidence, 2=Promising Evidence, 1=Weak Evidence, 0=No Evidence)
A1 The use of the outcome measures produce reliable scores for the majority of the primary outcomes (see
following table for a detailed breakdown on the outcomes)
X Yes
No
Unknown/unable to code
A2 Multi-method (at least two assessment methods used)
Yes
X No (Used CELF. Also used VOCAB, but only for one outcome measure- Vocabulary)
N/A
Unknown/unable to code
A3 Multi-source (at least two sources used self-reports, teachers etc.)
Yes
XNo
N/A
Unknown/unable to code
37
Doctorate in Educational and Child Psychology
Polly Grant
A4 Validity of measures reported (well-known or standardized or norm-referenced are considered good,
consider any cultural considerations)
X Yes validated with specific target group (CELF)
In part, validated for general population only
No
Unknown/unable to code
Overall Rating of Measurement:
3
2 X1
0
B Comparison Group
B1 Type of Comparison group
Typical intervention
Attention placebo
Intervention element placebo
Alternative intervention
Pharmacotherapy
X No intervention
Wait list/delayed intervention
Minimal contact
Unable to identify type of comparison
B2 Overall rating of judgment of type of comparison group
Very low
X Low
Moderate
High
Very high
Unable to identify comparison group
B3 Counterbalancing of change agent (participants who receive intervention from a single therapist/teacher etc.
were counter-balanced across intervention)
By change agent
Statistical (analyse includes a test for intervention)
Other
Not reported/None
B4 Group equivalence established (group equivalence not reported).
Random assignment
Posthoc matched set
Statistical matching
Post hoc test for group equivalence
B5 Equivalent mortality (not clear from numbers reported).
Low attrition (less than 20 % for post)
Low attrition (less than 30% for follow-up)
Intent to intervene analysis carried out?
Findings_____________
Overall Level of Evidence 1____
3= Strong Evidence
2=Promising Evidence
1=Weak Evidence
38
0=No Evidence
Doctorate in Educational and Child Psychology
Polly Grant
C Appropriate Statistical Analysis
Analysis 1____________________________________________________________________________
X Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable
Sufficiently large N (insufficient : only 6 children).
Analysis 2____________________________________________________________________________
Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable
Sufficiently large N
Analysis 3____________________________________________________________________________
Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable
Sufficiently large N
Overall Rating of Analysis:
3
2 X1
0
39
Doctorate in Educational and Child Psychology
Polly Grant
[adapted from Task Force on Evidence-Based Interventions in School Psychology, American Psychology
Association, Kratochwill, T.R. (2003)]
Coding Protocol
Name of Coder: ucjtpgr
Date:
15/1/14
Gallagher, A. L., & Chiat, S. (2009). Evaluation of speech
and language therapy interventions for pre-school children with specific language
impairment: a comparison of outcomes following specialist intensive, nursery-based and no
intervention. International journal of language & communication disorders / Royal College
of Speech & Language Therapists, 44(5), 616–38.
Full Study Reference in proper format:
Intervention Name (description of study): Evaluation
of speech and language therapy interventions
for pre-school children with specific language impairment: a comparison of outcomes
following specialist intensive, nursery-based and no intervention
Study ID Number:__________________
Type of Publication:
Book/Monograph
X Journal Article
Book Chapter
Other (specify):
1.General Characteristics
A. General Design Characteristics
A1. Random assignment designs (if random assignment design, select one of the following)
X Completely randomized design
Randomized block design (between participants, e.g., matched classrooms)
Randomized block design (within participants)
Randomized hierarchical design (nested treatments
A2. Nonrandomized designs (if non-random assignment design, select one of the following)
Nonrandomized design
Nonrandomized block design (between participants)
Nonrandomized block design (within participants)
Nonrandomized hierarchical design
Optional coding for Quasi-experimental designs
A3. Overall confidence of judgment on how participants were assigned (select on of the following)
Very low (little basis)
Low (guess)
Moderate (weak inference)
X High (strong inference)
Very high (explicitly stated)
N/A
Unknown/unable to code
B Participants
40
Doctorate in Educational and Child Psychology
Total size of sample (start of study):
Intervention group sample size:
Control group sample size
Polly Grant
24
2 intervention groups: 8 in each group
8
C. Type of Program
Universal prevention program
Selective prevention program
Targeted prevention program
X Intervention/Treatment
Unknown
D. Stage of Program
Model/demonstration programs
Early stage programs
Established/institutionalized programs
X Unknown
E. Concurrent or Historical Intervention Exposure
Current exposure
Prior exposure
X Unknown
Section 2
Key Features for Coding Studies and Rating Level of Evidence/Support
A Measurement (Estimating the quality of the measures used to establish effects)
(Rating Scale: 3= Strong Evidence, 2=Promising Evidence, 1=Weak Evidence, 0=No Evidence)
A1 The use of the outcome measures produce reliable scores for the majority of the primary outcomes (see
following table for a detailed breakdown on the outcomes)
X Yes
No
Unknown/unable to code
A2 Multi-method (at least two assessment methods used)
Yes
X No (multiple methods were used, but only one for each
N/A
Unknown/unable to code
outcome measure)
A3 Multi-source (at least two sources used self-reports, teachers etc.)
Yes
X No (Multiple sources used, but only one for each outcome
measure)
N/A
Unknown/unable to code
A4 Validity of measures reported (well-known or standardized or norm-referenced are considered good,
consider any cultural considerations)
X Yes validated with specific target group
In part, validated for general population only
No
41
Doctorate in Educational and Child Psychology
Polly Grant
Unknown/unable to code
Overall Rating of Measurement:
3
2 X1
0
B Comparison Group
B1 Type of Comparison group (The study involved 2 intervention groups)
Typical intervention
Attention placebo
Intervention element placebo
X Alternative intervention
Pharmacotherapy
No intervention
X Wait list/delayed intervention
Minimal contact
Unable to identify type of comparison
B2 Overall rating of judgment of type of comparison group
Very low
Low
Moderate
XHigh
Very high
Unable to identify comparison group
B3 Counterbalancing of change agent (participants who receive intervention from a single therapist/teacher etc.
were counter-balanced across intervention)
By change agent
Statistical (analyse includes a test for intervention)
Other
X Not reported/None
B4 Group equivalence established
X Random assignment
Posthoc matched set
Statistical matching
Post hoc test for group equivalence
B5 Equivalent mortality
X Low attrition (less than 20 % for post)
Low attrition (less than 30% for follow-up)
Intent to intervene analysis carried out?
Findings_____________
Overall Level of Evidence 3
3= Strong Evidence
2=Promising Evidence
1=Weak Evidence
42
0=No Evidence
Doctorate in Educational and Child Psychology
Polly Grant
C Appropriate Statistical Analysis
Analysis 1
X Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable (this
was not controlled-
multiple t-tests were used)
X Sufficiently large N (insufficient N: 8 in each group)
Analysis 2____________________________________________________________________________
Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable
Sufficiently large N
Analysis 3____________________________________________________________________________
Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable
Sufficiently large N
Overall Rating of Analysis:
3
2 X1
0
43
Doctorate in Educational and Child Psychology
Polly Grant
[adapted from Task Force on Evidence-Based Interventions in School Psychology, American Psychology
Association, Kratochwill, T.R. (2003)]
Coding Protocol
Name of Coder: ucjtpgr
Date: 15/1/14
McCartney, E., Boyle, J., Ellis, S., Bannatyne, S., &
Turnbull, M. (2011). Indirect language therapy for children with persistent language
impairment in mainstream primary schools: outcomes from a cohort intervention.
International journal of language & communication disorders / Royal College of Speech &
Language Therapists, 46(1), 74–82.
Full Study Reference in proper format:
Intervention Name (description of study: The
intervention was delivered in the classroom by a
member of staff. Language materials were supplied to staff. These focused on: vocabulary,
grammar and narrative.
Study ID Number:__________________
Type of Publication:
Book/Monograph
Journal Article
Book Chapter
Other (specify):
1.General Characteristics
A. General Design Characteristics
A1. Random assignment designs (if random assignment design, select one of the following)
Completely randomized design
Randomized block design (between participants, e.g., matched classrooms)
Randomized block design (within participants)
Randomized hierarchical design (nested treatments
A2. Nonrandomized designs (if non-random assignment design, select one of the following)
X Nonrandomized design
Nonrandomized block design (between participants)
Nonrandomized block design (within participants)
Nonrandomized hierarchical design
Optional coding for Quasi-experimental designs
A3. Overall confidence of judgment on how participants were assigned (select on of the following)
Very low (little basis)
Low (guess)
Moderate (weak inference)
High (strong inference)
Very high (explicitly stated)
N/A
Unknown/unable to code
B Participants
Total size of sample (start of study): 69
44
Doctorate in Educational and Child Psychology
Intervention group sample size:
Polly Grant
38
Control group sample size: 31
C. Type of Program
Universal prevention program
Selective prevention program
Targeted prevention program
Intervention/Treatment
Unknown
D. Stage of Program
Model/demonstration programs
Early stage programs
Established/institutionalized programs
Unknown
E. Concurrent or Historical Intervention Exposure
Current exposure
Prior exposure
Unknown
Section 2
Key Features for Coding Studies and Rating Level of Evidence/Support
A Measurement (Estimating the quality of the measures used to establish effects)
(Rating Scale: 3= Strong Evidence, 2=Promising Evidence, 1=Weak Evidence, 0=No Evidence)
A1 The use of the outcome measures produce reliable scores for the majority of the primary outcomes (see
following table for a detailed breakdown on the outcomes)
X Yes
No
Unknown/unable to code
A2 Multi-method (at least two assessment methods used)
Yes
No (only the CELF is used to measure language ability).
(Phab also used, to measure
phonological awareness, and NARA –II)
N/A
Unknown/unable to code
A3 Multi-source (at least two sources used self-reports, teachers etc.)
Yes
No
N/A
Unknown/unable to code
A4 Validity of measures reported (well-known or standardized or norm-referenced are considered good,
consider any cultural considerations)
XYes validated with specific target group
In part, validated for general population only (CELF Validated for general population).
No
Unknown/unable to code
45
Doctorate in Educational and Child Psychology
Overall Rating of Measurement:
3
2
Polly Grant
1X0
B Comparison Group
B1 Type of Comparison group
Typical intervention.
Attention placebo
Intervention element placebo
Alternative intervention
Pharmacotherapy
No intervention
Wait list/delayed intervention
Minimal contact
Unable to identify type of comparison
B2 Overall rating of judgment of type of comparison group
Very low
Low
Moderate
High
Very high
Unable to identify comparison group
B3 Counterbalancing of change agent (participants who receive intervention from a single therapist/teacher etc
were counter-balanced across intervention)
By change agent
Statistical (analyse includes a test for intervention)
Other
X Not reported/None
B4 Group equivalence established
Random assignment
Posthoc matched set
Statistical matching
Post hoc test for group equivalence
B5 Equivalent mortality
X Low attrition (less than 20 % for post)
Low attrition (less than 30% for follow-up)
Intent to intervene analysis carried out?
Findings_____________
Overall Level of Evidence 3 ____
3= Strong Evidence
2=Promising Evidence
1=Weak Evidence
46
0=No Evidence
Doctorate in Educational and Child Psychology
Polly Grant
C Appropriate Statistical Analysis
Analysis 1____________________________________________________________________________
X Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable (not applicable)
X Sufficiently large N Insufficient N
Analysis 2____________________________________________________________________________
Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable
Sufficiently large N
Analysis 3____________________________________________________________________________
Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable
Sufficiently large N
Overall Rating of Analysis:
3
2 X1
0
47
Doctorate in Educational and Child Psychology
Polly Grant
[adapted from Task Force on Evidence-Based Interventions in School Psychology, American Psychology
Association, Kratochwill, T.R. (2003)]
Coding Protocol
Name of Coder:
ucjtpgr
Date: 15/1/2014
Mecrow, C., Beckwith, J., & Klee, T. (2010). An
exploratory trial of the effectiveness of an enhanced consultative approach to delivering
speech and language intervention in schools. International journal of language &
communication disorders / Royal College of Speech & Language Therapists, 45(3),
354–67.
Full Study Reference in proper format :
Intervention Name (description of study): An
Individualised programme was developed for each
child depending on their need. The focus was on: attention and listening, auditory memory,
receptive and expressive language, phonological awareness, literacy an all aspects of speech.
Programme was delivered by specialist teaching assistants under the supervision of speech
and language therapists.
Study ID Number:__________________
Type of Publication:
Book/Monograph
Journal Article
Book Chapter
Other (specify):
1.General Characteristics
A. General Design Characteristics
A1. Random assignment designs (if random assignment design, select one of the following)
Completely randomized design
Randomized block design (between participants, e.g., matched classrooms)
Randomized block design (within participants)
Randomized hierarchical design (nested treatments
A2. Nonrandomized designs (if non-random assignment design, select one of the following)
Nonrandomized design
Nonrandomized block design (between participants)
Nonrandomized block design (within participants)
Nonrandomized hierarchical design
Optional coding for Quasi-experimental designs
A3. Overall confidence of judgment on how participants were assigned (select on of the following)
Very low (little basis)
Low (guess)
Moderate (weak inference)
High (strong inference)
Very high (explicitly stated)
N/A
Unknown/unable to code
48
Doctorate in Educational and Child Psychology
B Participants
Total size of sample (start of study):
Polly Grant
35
Intervention group sample size:_0______
Control group sample size:____0____
C. Type of Program
Universal prevention program
Selective prevention program
Targeted prevention program
Intervention/Treatment
Unknown
D. Stage of Program
Model/demonstration programs
Early stage programs
Established/institutionalized programs
Unknown
E. Concurrent or Historical Intervention Exposure
Current exposure
Prior exposure
Unknown
Section 2
Key Features for Coding Studies and Rating Level of Evidence/Support
A Measurement (Estimating the quality of the measures used to establish effects)
(Rating Scale: 3= Strong Evidence, 2=Promising Evidence, 1=Weak Evidence, 0=No Evidence)
A1 The use of the outcome measures produce reliable scores for the majority of the primary outcomes (see
following table for a detailed breakdown on the outcomes)
X Yes
No
Unknown/unable to code
A2 Multi-method (at least two assessment methods used)
Yes
X No (Multimethods were used, but not to measure the same language outcome)
N/A
Unknown/unable to code
A3 Multi-source (at least two sources used self-reports, teachers etc.)
Yes
No (Multisources were used, however not to measure the language outcomes )
N/A
Unknown/unable to code
A4 Validity of measures reported (well-known or standardized or norm-referenced are considered good,
consider any cultural considerations)
Yes validated with specific target group
In part, validated for general population only (only the CELF Is validated for the group, the IEP measures
were not).
49
Doctorate in Educational and Child Psychology
Polly Grant
No
Unknown/unable to code
Overall Rating of Measurement:
3
2 X1
2
B Comparison Group
B1 Type of Comparison group
Typical intervention
Attention placebo
Intervention element placebo
Alternative intervention
Pharmacotherapy
No intervention
Wait list/delayed intervention
Minimal contact
Unable to identify type of comparison
B2 Overall rating of judgment of type of comparison group
Very low
Low
Moderate
High
Very high
Unable to identify comparison group
B3 Counterbalancing of change agent (participants who receive intervention from a single therapist/teacher etc
were counter-balanced across intervention)
By change agent
Statistical (analyse includes a test for intervention)
Other
X Not reported/None
B4 Group equivalence established
Random assignment
Posthoc matched set
Statistical matching
Post hoc test for group equivalence
B5 Equivalent mortality
X Low attrition (less than 20 % for post)
Low attrition (less than 30% for follow-up)
Intent to intervene analysis carried out?
Findings_____________
Overall Level of Evidence 1
3= Strong Evidence
2=Promising Evidence
1=Weak Evidence
50
0=No Evidence
Doctorate in Educational and Child Psychology
Polly Grant
C Appropriate Statistical Analysis
Analysis 1____________________________________________________________________________
X Appropriate unit of analysis
X Familywise/experimenter wise error rate controlled when applicable
Sufficiently large N (insufficient N)
Analysis 2____________________________________________________________________________
Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable
Sufficiently large N
Analysis 3____________________________________________________________________________
Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable
Sufficiently large N
Overall Rating of Analysis:
3
2 X1
0
51
Doctorate in Educational and Child Psychology
Polly Grant
[adapted from Task Force on Evidence-Based Interventions in School Psychology, American Psychology
Association, Kratochwill, T.R. (2003)]
Coding Protocol
Name of Coder: ucjtpgr
Date: 15/1/2014
Full Study Reference in proper format : Wilcox,
M. J., Gray, S. I., Guimond, A. B., & Lafferty, A.
E. (2011). Early Childhood Research Quarterly Efficacy of the TELL language and
literacy curriculum for pre-schoolers with developmental speech and / or language
impairment, 26, 278–294. doi:10.1016/j.ecresq.2010.12.003
Intervention Name (description of study):
Teaching Early Literacy and Language [TELL]
Study ID Number:__________________
Type of Publication:
Book/Monograph
Journal Article
Book Chapter
Other (specify):
1.General Characteristics
A. General Design Characteristics
A1. Random assignment designs (if random assignment design, select one of the following)
X Completely randomized design
Randomized block design (between participants, e.g., matched classrooms)
Randomized block design (within participants)
Randomized hierarchical design (nested treatments
A2. Nonrandomized designs (if non-random assignment design, select one of the following)
Nonrandomized design
Nonrandomized block design (between participants)
Nonrandomized block design (within participants)
Nonrandomized hierarchical design
Optional coding for Quasi-experimental designs
A3. Overall confidence of judgment on how participants were assigned (select on of the following)
Very low (little basis)
Low (guess)
Moderate (weak inference)
X High (strong inference)
Very high (explicitly stated)
N/A
Unknown/unable to code
B Participants
Total size of sample (start of study): 118
52
Doctorate in Educational and Child Psychology
Intervention group sample size:
Polly Grant
80
Control group sample size:38
C. Type of Program
X Universal prevention program
Selective prevention program
Targeted prevention program
Intervention/Treatment
Unknown
D. Stage of Program
Model/demonstration programs
X Early stage programs
Established/institutionalized programs
Unknown
E. Concurrent or Historical Intervention Exposure
Current exposure
Prior exposure
X Unknown
Section 2
Key Features for Coding Studies and Rating Level of Evidence/Support
A Measurement (Estimating the quality of the measures used to establish effects)
(Rating Scale: 3= Strong Evidence, 2=Promising Evidence, 1=Weak Evidence, 0=No Evidence)
A1 The use of the outcome measures produce reliable scores for the majority of the primary outcomes (see
following table for a detailed breakdown on the outcomes)
XYes
No
Unknown/unable to code
A2 Multi-method (at least two assessment methods used)
Yes
No (Multiple methods were used but they measured different language outcomes.
N/A
Unknown/unable to code
A3 Multi-source (at least two sources used self-reports, teachers etc.)
Yes
XNo Mutliple sources were used, but they measured different language outcomes.
N/A
Unknown/unable to code
A4 Validity of measures reported (well-known or standardized or norm-referenced are considered good,
consider any cultural considerations)
X Yes validated with specific target group (not reported for all measures)
In part, validated for general population only
No
Unknown/unable to code
53
Doctorate in Educational and Child Psychology
Overall Rating of Measurement:
3
2 X1
Polly Grant
0
B Comparison Group
B1 Type of Comparison group
Typical intervention
Attention placebo
Intervention element placebo
Alternative intervention
Pharmacotherapy
X No intervention
Wait list/delayed intervention
Minimal contact
Unable to identify type of comparison
B2 Overall rating of judgment of type of comparison group
Very low
Low
Moderate
X High
Very high
Unable to identify comparison group
B3 Counterbalancing of change agent (participants who receive intervention from a single therapist/teacher etc
were counter-balanced across intervention)
By change agent
Statistical (analyse includes a test for intervention)
Other
X Not reported/None
B4 Group equivalence established
Random assignment
Posthoc matched set
Statistical matching
X Post hoc test for group equivalence
B5 Equivalent mortality
X Low attrition (less than 20 % for post)
Low attrition (less than 30% for follow-up)
Intent to intervene analysis carried out?
Findings_____________
Overall Level of Evidence 2
3= Strong Evidence
2=Promising Evidence
1=Weak Evidence
54
0=No Evidence
Doctorate in Educational and Child Psychology
Polly Grant
C Appropriate Statistical Analysis
Analysis 1____________________________________________________________________________
X Appropriate unit of analysis
XFamilywise/experimenter wise error rate controlled when applicable
X Sufficiently large N
Analysis 2____________________________________________________________________________
Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable
Sufficiently large N
Analysis 3____________________________________________________________________________
Appropriate unit of analysis
Familywise/experimenter wise error rate controlled when applicable
Sufficiently large N
Overall Rating of Analysis:
3 X 2
1
0
55