Examining Teacher Quality and Teacher
Preparation for English Language Learners:
the Use of Multiple Data Sources
Maria Coady, Alice Jo, Ramandeep Brar
& Yang Qi
Florida Association of Teacher Educators
October 8, 2011
Overview of ELL Preparation
Nationally, most teachers are inadequately prepared to
teach ELLs (Gándara et al., 2005; Walker et al., 2004)
– Karabenick & Noda (2004) report that teachers lack basic
foundational knowledge about ELL issues, despite the fact that
88% teach ELLs
– Colon-Muniz, Brady and SooHoo (2010) reported that teachers
who had been prepared to teach ELLs at universities performed
better in their practice in teaching ELL students.
Florida’s requirements to prepare mainstream teachers
of ELLs (through in-service since 1990, and pre-service
since 2001) are unique.
ELLs in Florida Context
FDOE reported 231,801 ELLs in 2009-2010 (8.8% of
total enrollment)
Additionally, FDOE reported 204,287 former ELLs in
2009-2010 (7.7% of total enrollment)
Most ELLs (66%) are enrolled in the elementary grades,
including 40% enrolled in grades K-2 (2009-2010)
Source: http://www.fldoe.org/aala/omsstat.asp
ELL Achievement in Florida
The Florida Context: Preservice
Teacher Preparation Requirements
• Teacher education programs have
prepared candidates through an “infused”
ESOL endorsement program since 2001
– Minimum of 2 ESOL stand-alone courses
taught by ESOL faculty
– ESOL Performance Standards are addressed
and assessed in the program
– ESOL content is “infused” in coursework
throughout the program
– Field experience requirement (varies by
course)
Project DELTA
(Developing English Language and
Literacy Through Teacher Achievement)
Federally funded, post-training assessment grant to
examine the impact of an ESOL-infused elementary
education program on teacher practices and ELL
achievement through
– Quantitative data from Education Data Warehouse
(EDW) linking teacher and student files
– Survey of UF graduates’ sense of preparedness and
efficacy in teaching ELLs
– Interviews with UF teacher graduates
– Case study observations in elementary, mainstream
math and reading classes and follow up interviews
with those teachers
DELTA Research Questions
How (and how well) do teacher graduates from a (ESOL
infused) teacher preparation program (ProTeach) work
with ELLs?
A. What practices do teacher graduates use to facilitate learning
(language, literacy & content) for ELLs in their classrooms?
(Case Studies)
B. What do teacher graduates believe they have learned in their
teacher preparation program? (Survey, Interviews)
C. How well do English learners in the classrooms of UF’s teacher
graduates (across various preparation pathways) perform on 3rd
grade state standardized tests? (EDW Metadataset)
Theoretical Framework:
Quality Teachers of ELLs
Teacher Quality and
Teacher Effectiveness
Terms frequently used interchangeably
Current climate to associate teachers with
student performance as one method of
teacher evaluation (quantitative measures)
Annual Evaluation 100%=50% student growth + 50% inperson evaluation
50% instructional evaluation = 20% teacher growth +
30% teacher status score
Our Focus Today...
Present findings from a large-scale, mixed
methods study using data from Florida’s
Educational Data Warehouse (EDW), case
study teachers of ELLs, program graduates
(survey and interviews);
Discuss the use of mixed methods to explore
quality teachers of ELLs;
Demonstrate how various methods illuminate
specific characteristics of quality teachers of
ELLs.
Mixed Methods
Both quantitative (EDW, survey) and
qualitative (observations, interviews,
document analysis, survey) methods are
used in data collection and analysis.
– EDW
– Survey
– Observations
– Interviews
– Documents
The Education Data
Warehouse
Only a handful of states have statewide
datasets matching students to teachers
for any length of time (NC, WA, NY)
Florida Department of Education has
developed the EDW as a large dataset
related to schools in Florida
http://edwapp.doe.state.fl.us/home.aspx
EDW Metadataset
www.edwapp.doe.state.fl.us/home.aspx
Final Dataset for Analysis
-Non-ESE, non-retained ELLs, not enrolled in
ESOL course, in grades 3, 4, and 5 with complete
FCAT and enrollment data matched to
– Individual non-UF teachers
(n= 71,194 students; 23,985 teachers)
– Individual UF teachers
(n=1,100 students; 358 teachers)
DELTA – EDW Findings
Do different UF teacher preparation paths make a
significant difference in ELL student
achievement?
UF teacher preparation pathway codes
(n=12) were aggregated into four groups:
– A [5 Year ESOL Endorsement]
– B [5 Year ESE + ESOL Endorsement]
– C [5 Year No ESOL Endorsement]
– D [4 year Bachelor’s degree]
DELTA -- EDW Findings
Student Mean FCAT Score
UF Teacher
Preparation Paths
A: 5 Year ESOL
Math
Reading
306.42*
282.64*
B: 5 Year ESOL + ESE
310.42
294.52
C: 5 Year No ESOL
291.53*
270.64*
D: 4 Year Bachelor’s
307.26
278.82
Survey Methods and Design
Five sections of the survey with 10-12
items in each (total 49 statements)
Social and cultural dimensions of teaching
for ESOL students
Content area instruction
Language and literacy development
Curriculum and classroom organization
Assessment issues in teaching ESOL
students
Survey
Survey Questions
1. In what instructional areas related to ELLs
do teacher education program graduates
feel most prepared? Least prepared?
2. In what instructional areas related to ELLs
do teacher education program graduates
feel most effective? Least effective?
Survey Findings
• Teachers feel most prepared and effective:
a. creating a welcoming and affirming (valued)
atmosphere in the classroom
b. using graphic organizers
• Teachers feel least prepared and effective using students’
first language as a resource for learning (bilingual
materials, home resource)
• Implications for the development of bilingualism and
learning theory (connecting background to new learning)
Interviews
19 teacher-graduates, conducted by telephone, explored
teachers’ beliefs
– Interview data (n=19)
Criteria for participation:
– Graduate of UF elementary education program with
infused ESOL endorsement (two stand-alone ESOL
courses completed)
– Two or more years of teaching experience
– Experience teaching elementary ELLs
Interview Questions:
- ESOL Preparation
- Languages Other Than English (LOTE)
- Program Recommendations
Interview Findings
Overall positive evaluation of the program (all but 1
graduate felt positive about their preparation, especially
compared with some of their colleagues)
Emphasis on central role of field experiences in building
confidence and developing competence in teaching
ELLs
Recommendations to expand field experiences, ensure
elementary field placements, connect theory and
practice, and provide access to teaching resources
Importance of LOTE proficiency and/or experiences to
connect with ELLs and make classrooms more
comfortable and accessible
Case Studies
Teacher Cases(n=6) teacher graduates in 5
school districts in north/central Florida
Graduated from an ESOL-infused pre-service
teacher preparation program
Classroom observation (math & reading over 18
month period)
– Audio, video, observation protocol, material artifacts,
photos
Case Studies Findings
Preliminary findings of two case study teachers
100%
90%
88%
85%
80%
74%
72%
70%
60%
50%
40%
30%
20%
28%
26%
15%
12%
10%
0%
Feedback (462)
Questioning (397)
ELL interaction
Instruction (270)
Classroom management
(50)
Non ELL interaction
Case Studies Findings
Preliminary findings of two case study teachers
Case study teachers – observed no interactions
in which teachers planned for ELLs and made
specific modifications based on language
proficiency level of the ELL;
Teachers’ generally addressed language
demands of classroom/text/activities in a
spontaneous manner:
– Feedback, Questioning, Instruction, Management
Initiation, Response, Evaluation (IRE) pattern of
language use
Case Study Teacher
Use of Mixed Methods in
Determining Teacher Quality
Different research methodologies are
complementary, such that they paint a
more holistic picture of teacher graduates’
work, beliefs, and effectiveness with ELs.
The methodologies used highlight distinct
findings to answer the research questions
The methodologies may not provide
enough answers (“why” or “how”); other
methods may need to be employed
Conclusion
Multiple methods paint a picture that
addresses questions of quality teachers of
ELLs that may not be evident by simply
using quantitative (assessment) data;
Multiple methods seek to address not only
relationships between variables (teachers
and student) but illuminate how and why;
Multiple methods allow for improvement in
teacher education and programmatic
feedback.
Questions? And
Contacts
Maria Coady, Ph.D.
mcoady@coe.ufl.edu
Project DELTA website
http://education.ufl.edu/project-delta/