Download an electronic copy of the
slides and a handout here:
projectgladstudy.educationnorthwest.org
Lessons Learned from
An Experimental Study of Project GLAD®
Theresa Deussen & Claudia Rodriguez-Mojica
EL Alliance Conference – March 14, 2014
Do you tweet?
@TLDeussen
#ProjectGLAD
#educationnw
Today’s Agenda
• Describe Project GLAD and how it works
• Report Year 1 results from our experimental study
• Take a closer look at what our findings mean for
closing the achievement gap
• Share some reflections about how professional
development might have a bigger impact
Language and content
SIOP
Sheltered Instruction Observation Protocol
QuEST
Quality English and Science Teaching
Project GLAD
Guided Language Acquisition Design
Project GLAD®
(Guided Language Acquisition Design)
Key program elements
35 instructional strategies
Usable with any curriculum
Intended as a coherent package that builds
•Readiness and motivation to learn
•Content knowledge
•Ability to converse at a high level about the
topic
•Ability to read and write at a high level about
the topic
What does it look like?
Project GLAD Input Chart
(One of 35 instructional strategies)
11
How does this compare to what
you do in your school?
I have never had
training that has
been this good!
Research Questions
What is the impact of Project GLAD® teacher
training on fifth-grade students’ achievement
in ELA and science?
– For ELs
– For non ELs?
Study population
30 Idaho schools
21 districts
50% located in rural communities
2250 grade 5 students
65% Free/Reduced-Price Lunch
33% Latino
62% White
13% ELs
Cluster Randomized Trial (CRT)
30 schools agreed
to participate
15 received Project GLAD®
15 had “business as usual”
Outcome measures
English language arts
•Reading comprehension
•Vocabulary
•Essay writing
Science
•Rocks & minerals unit test
•State science assessment
Year 1 Literacy Outcomes
ELs only
Year 1 Literacy Outcomes
ELs only
Year 1 Literacy Outcomes
ELs only
485
Treatment
Control
480
475
0.21
0.24
470
465
460
455
Vocabulary
Comprehension
Pretest
What’s an effect size again?
Difference between the Tx and C
Standard deviation of the group
Measure
Vocabulary
Comprehension
ELs
Non ELs
.21~
.24~
.04
.04
.32*
.27~
.05
.22
.05
.02
.19
.12
.21~
.13
.08
.14
.12
.07
.23
.13
Writing
Ideas
Organization
Voice
Word Choice
Sent. Fluency
Conventions
Rocks & Minerals
State Science
Measure
Vocabulary
Comprehension
ELs
Non ELs
.21~
.24~
.04
.04
.32*
.27~
.05
.22
.05
.02
.19
.12
.21~
.13
.08
.14
.12
.07
.23
.13
Writing
Ideas
Organization
Voice
Word Choice
Sent. Fluency
Conventions
Rocks & Minerals
State Science
Measure
Vocabulary
Comprehension
ELs
Non ELs
.21~
.24~
.04
.04
.32*
.27~
.05
.22
.05
.02
.19
.12
.21~
.13
.08
.14
.12
.07
.23
.13
Writing
Ideas
Organization
Voice
Word Choice
Sent. Fluency
Conventions
Rocks & Minerals
State Science
Measure
Vocabulary
Comprehension
ELs
Non ELs
.21~
.24~
.04
.04
.32*
.27~
.22
.05
.05
.02
.19
.12
.21~
.13
.14
.08
.12
.07
.23
.13
Writing
Ideas
Organization
Word Choice
Voice
Sent. Fluency
Conventions
Rocks & Minerals
State Science
What’s a good effect size?
Use empirical comparisons.
Literacy
Effect sizes for ELs
Reading
Writing
Reading
Vocabulary
Ideas
Organization
Project
GLAD
0.24
0.21
0.32
0.27
SIOP
0.16*
0.19*
0.31**
QuEST
0.26
* Small sample with developers involved in training.
** Estimated based on data provided in Echevarria, Short & Powers 2006.
Literacy
Effect sizes for ELs
Reading
Writing
Reading
Vocabulary
Ideas
Organization
Project
GLAD
0.24
0.21
0.32
0.27
SIOP
0.16*
0.19*
0.31**
QuEST
0.26
* Small sample with developers involved in training.
** Estimated based on data provided in Echevarria, Short & Powers 2006.
Literacy
Effect sizes for ELs
Reading
Writing
Reading
Vocabulary
Ideas
Organization
Project
GLAD
0.24
0.21
0.32
0.27
SIOP
0.16*
0.19*
0.31**
QuEST
0.26
* Small sample with developers involved in training.
** Estimated based on data provided in Echevarria, Short & Powers 2006.
Literacy
Effect sizes for ELs
Reading
Writing
Reading
Vocabulary
Ideas
Organization
Project
GLAD
0.24
0.21
0.32
0.27
SIOP
0.16*
0.19*
0.31**
QuEST
0.26
* Small sample with developers involved in training.
** Estimated based on data provided in Echevarria, Short & Powers 2006.
Literacy
Effect sizes for ELs
Reading
Writing
Reading
Vocabulary
Ideas
Organization
Project
GLAD
0.24
0.21
0.32
0.27
SIOP
0.161
0.192
0.312
QuEST
1
2
0.26
Small sample with developers involved in training.
Estimated based on data provided in Echevarria, Short & Powers 2006.
Science
Effect sizes for ELs
Unit Test
State Test
Project
GLAD
0.19
0.12
QuEST1
0.16
(NA)
1
QuEST also had positive impacts for nonELLs.
Back to our research questions
What is the impact on Project GLAD
on students’ reading, vocabulary,
writing, and science achievement?
For ELLs?
For nonELLs?
Starting lower means you need a
bigger boost.
Vocabulary
Comprehension
Can Project GLAD close that gap?
Multiple years
Additive effect?
Compounding effect?
Reflections
Content of the PD
Factors affecting implementation
Content of the PD
Does it prepare teachers to make a difference?
Yes
Multiple representations of concepts
Structured interactions with academic focus
Use of primary language
Focus on academically useful words
Powerful science instruction
No
Content of the PD
Does it prepare teachers to make a difference?
Yes
Multiple representations of concepts
X
Structured interactions with academic focus
X
Use of primary language
Focus on academically useful words
Powerful science instruction
No
Content of the PD
Does it prepare teachers to make a difference?
Yes
Multiple representations of concepts
X
Structured interactions with academic focus
X
No
Use of primary language
X
Focus on academically useful words
X
Powerful science instruction
X
Powerful science instruction
Inquiry-based
Evidence
• Collect
• Interpret
• Communicate
Scaffolding + FOSS kits
• (ES = +1.39)
What about implementation?
Implementation
Average number of strategies
per week (surveys)
Average quality rating
(observations)
48
Average
12.5
Range
0-22
69%
19-100%
Average number of strategies
per week (surveys)
Average quality rating
(observations)
49
Average
12.5
Range
0-22
69%
19-100%
Average number of strategies
per week (surveys)
Average fidelity rating
(observations)
50
Average
12.5
Range
0-22
69%
19-100%
The frequency and fidelity of
implementation varied significantly
across teachers.
Average number of strategies
per week (surveys)
Average fidelity rating
(observations)
51
Average
12.5
Range
0-22
69%
19-100%
Factors affecting implementation
Does the PD make it easy for teachers to implement?
Yes
On-going
Collective participation
Explicit protocols for team work
Focus on solutions, not activities
Coherence
No
Factors affecting implementation
Does it make it easy for teachers to implement?
Yes
On-going
X
Collective participation
X
Explicit protocols for team work
X
X
Focus on solutions, not activities
Coherence
No
X
X
Teachers also told us what made it hard to
implement.
Lack of time…
Story map—this takes too much time to
develop and implement. This is something to
do in the summer.
Mostly I haven’t used some of the strategies
as a matter of time. Social studies is only 30
minutes.
Lack of materials…
I was worried about the home school
connections because I sent things home and
I thought about the paper I would have to
use. We have a copy limit per month.
Lack of confidence in using the
strategy…
Don’t feel competent as yet to do certain
strategies like Team Tasks that are linked to
other strategies. Same is true with Group
Frame, ELD Review and the Process Grid
and Mind Map. I’m not experienced enough
to do the reading strategies either.
Collaboration may support Project
GLAD implementation
…[collaboration] changed how I
implemented the strategies and how often I
did them. The other person gave me an
incentive to do things more often and if she
hadn't been there I would have done far
less.
What else might help?
Reading
Writing
Small groups
Charts
Behavior
Walls
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
Standards
Scouts
Literacy awards
T-graph for social skills
Team points
Personal interaction
10/2 lecture
Picture file cards
Observation chart
Inquiry chart
Cognitive content dictionary
Graphic organizer
Pictorial input
Comparative input
Narrative input
Sentence patterning chart
Chants/poetry
Story map
Mind map
Process grid
Team task
Expert group
ELD group frame
ELD review
Numbered heads
Cooperative strip paragraph
Writers’ workshop
Learning logs
Interactive journals
Portfolios
Big books
Clunkers/links
Focused reading
Ear-to-ear reading
Home/school
REPEATED _________________
Used
*
*
*
*
*
*
*
Revisit
R1
R2
R3
R4
R5
R6
R7
R1
R1
R2
R2
R3
R4
R5
R1
R2
R3
R4
R1
R1
R1
R1
R1
R1
R1
R1
R1
R1
R1
R1
R1
R1
R1
R1
R2
R2
R2
R2
R2
R2
R2
R2
R2
R2
R2
R2
R2
R2
R2
R2
R3
R3
R3
R3
R3
R3
R3
R3
R3
R3
R3
R3
R3
R3
R3
R1
R2
R1
R2
R5
R6
R7
R4
R4
R4
R4
R4
R4
R4
R4
R4
R5
R5
R5
R5
R5
R5
R5
R5
R5
R6
R6
R6
R6
R6
R6
R6
R6
R4
R4
R4
R4
R4
R5
R5
R5
R5
R5
R6
R6
R6
R6
R3
R4
R5
R6
R3
R4
R5
60
R8
R9
R10
R7
R7
R7
R7
R7
R8
R8
R8
R8
R9
R9
R9
R10
R7
R7
R8
R8
R7
R8
R9
R7
R8
R9