Assessing English Language Learners:
Pre-Referral, Process, Pathways, and Evaluation
Pedro Olvera, PsyD, LEP
California Association of School Psychologists
Sacramento, California
April 18, 2013
Olvera, P. CASP 2013
1
Evaluation
Traditional and
Emerging Models
Pathways
Nonverbal, Modified,
English, and Native
Language, Combination.
Process
(Language of Assessment,
Instrument Selection, etc).
Pre-Referral
(Intervention, SST, RTI, Etc.)
Olvera, P. CASP 2013
2
Phase 1: Pre-Referral
Considerations in Assessing ELLs
Olvera, P. CASP 2013
3
Objectives
Developing Knowledge of Bilingualism/English Language Learners (ELL)
Effective General Education Teaching Strategies for ELLs
ELL and Reading Development (Normal v. Abnormal)
ELLs and Written Language (Normal v. Abnormal)
Evidence-Based Interventions for Reading and Writing for ELLs
Overview of Basic Interpersonal Communication Skills (BICS)
Overview of Cognitive Academic Language Proficiency (CALP) and the
Impact on Learning
• Assessing Primary Language for Assessment Purposes
•
•
•
•
•
•
•
4
Olvera, P. CASP 2013
Terms
• English Language Learner (ELL): “An active learner of the English language
who may benefit from various types of language support programs. This term is
used mainly in the U.S. to describe K–12 students”.
• ESL (English as a Second Language): “Formerly used to designate ELL
students; this term increasingly refers to a program of instruction designed to
support the ELL. It is still used to refer to multilingual students in higher education”.
• LEP (Limited English Proficiency): “Employed by the U.S. Department of
Education to refer to ELLs who lack sufficient mastery of English to meet state
standards and excel in an English-language classroom” (NCTE; 2008, p. 3).
(NCTE; 2008, p. 3).
Olvera, P. CASP 2013
5
Terms
• Native Language (IDEA, 1997) :
 The language normally used by the child and not the parents if there is a
different between the two.
 In your contact with the child, the language most used by the child in the
home or learning environment.
 For a child who is deaf or blind or has no written language, the mode of
communication most used by the child (sign language, Braille, or oral
communication).
Olvera, P. CASP 2013
6
Terms
Primary Language: A student's primary language is identified by the Home
Language Survey as the language first learned, most frequently used at home, or
most frequently spoken by the parents or adults in the home. Primary language is
also referred to as L1. (R30-LC; CDE)
Olvera, P. CASP 2013
7
Terms
Primary Language v. Native Language
• Primary Language (L1)- what student learned first.
• Native Language (L1 or L2)- What the student speaks in the present.
** If there is a difference between the two**
Olvera, P. CASP 2013
8
Myths
Ten Myths Regarding ELLs
Olvera, P. CASP 2013
9
Myths
(NCTE, 2008)
Myth #1
Many ELLs have disabilities, which is why they are often
overrepresented in special education (SPED).
Truth
Lack of support and inappropriate assessment practices is
why ELLs are placed in SPED. Early intervention and
good teaching can prevent overrepresentation.
Olvera, P. CASP 2013
10
Myths
(NCTE, 2008)
Myth #2
Children learn a second language quickly and easily.
Truth
Can take anywhere from 7-10 years to acquire full
proficiency in a second language (Rhodes, Ochoa, &
Ortiz, 2005).
Olvera, P. CASP 2013
11
Myths
(NCTE, 2008)
Myth #3
When an ELL is able to speak English fluently (orally),
he or she has mastered it.
Truth
Need to assess language in order to determine depth and
breadth fluency. Oral language skills to do not necessarily
translate to academic skills – BICS & CALPS (Cummins,
1984).
Olvera, P. CASP 2013
12
Myths
(NCTE, 2008)
Myth #4
All ELL students learn English in the same way.
Truth
Best approaches involve individualized approaches.
Integration of vocabulary, visuals, native language ,a
and language modulation (Gersten & Baker, 2000).
Olvera, P. CASP 2013
13
Myths
Myth #5
All ELLs are immigrants and no English is spoken in
the home.
Truth
In truth, ELLs consist of heterogeneous populations.
One study demonstrated that 57% were US born. Of
these, 27% are second generation and 30% are third
generation (NCTE, 2008).
Olvera, P. CASP 2013
14
Myths
Myth #5
All ELLs are low SES, have limited education, and
usually have lower academic expectations.
Truth
ELLs are from diverse populations and represent
various levels of SES, educational, and linguistic
backgrounds (NCTE, 2008).
Olvera, P. CASP 2013
15
Myths
Myth #6
Learning two or more languages simultaneously will
negatively impact a child’s fluency in both languages
and thus impact academic performance.
Truth
Students with little to no academic or cognitive
development in primary language tend to fall behind
after fourth grade (Thomas and Collier, 2002).
Olvera, P. CASP 2013
16
Myths
Myth #7
All children from minority language home (nonEnglish) learn in the same manner.
Truth
Amount of L1 exposure in L1 was greatest predictor of
the ELL’s second language academic success.
(Thomas and Collier, 2002).
Olvera, P. CASP 2013
17
Myths
Myth #8
Native English speakers (L1) will lag academically and
linguistically if they are enrolled in two way immersion
programs (dual immersion).
Truth
In addition to learning an L2 language, children from English (L1) homes
maintained their L1 and tested at above the 50th % on English tests. In
addition, gains were made in L2 each successive year. No evidence
was found that suggested that two-way immersion programs impeded
the academic ability of native English speakers (Thomas and Collier,
2002).
Olvera, P. CASP 2013
18
Myths
Myth #9
ELLs tend to be overrepresented in special education in all
districts.
Truth
Nationally, overrepresentation tends to occur in smaller districts
(<10,000). This seems to be due to lack of appropriate language
support programs/resources (NCT, 2008).
Olvera, P. CASP 2013
19
Myths
Myth #10
ELL families are uninterested and/or passive in their children’s
education.
Truth
Most immigrant parents, Mexican/Latino in particular, see themselves as
the purveyor of morality and manners (bien educados). Education is
seen as a specialist’s job (Sparks, 2009). Latino parents tend to see
themselves in a supportive role in the home environment (Olvera &
Olvera, 2012).
Olvera, P. CASP 2013
20
Guidelines
We will integrate key California regulations with
regards to children that are English Language
Learners (ELL’s) as outlined in (Figueroa & Newsome,
2006) and by Olvera as appropriate.
Olvera, P. CASP 2013
21
Guidelines
Education Code (EC) 56303: “A pupil shall be
referred for special education instruction and services
only after the resources of the regular education
program have been considered, and when
appropriate, utilized”
Olvera, P. CASP 2013
22
Guidelines
Section 56303, Education Code (described):
The absence of documented attempts to modify the student's general
education class program precludes any placement in special
education.
Because each student is guaranteed the right of an education in the
LRE, evidence shall be presented which shows that attempts have
been made to continue the student in a general education classroom.
Special education shall not act upon a referral without documented
program modifications.
Olvera, P. CASP 2013
23
Effective Classroom Teaching Strategies
Olvera, P. CASP 2013
24
Effective Instruction
• Five critical instructional factors for teaching ELLs in
classroom settings (Gersten &Baker, 2000):
• Vocabulary as a curricular anchor (connecting what they
know with new knowledge).
• Visuals to reinforce concepts and vocabulary
• Cooperative learning and peer tutoring strategies (PALS).
• Native language used strategically (L1 & L2)
• Modulation (regulation) of cognitive and language demands
(with support for these demands)
Olvera, P. CASP 2013
25
Effective Instruction
• National Council of La Raza & Education Alliance at Brown
University Recommend the Following (NCLR, 2005):
• Pre-Teaching the Language (preview of language that will be
used for lesson)
• Provide Meaningful Experience (connect with student)
• Record the Experience (writing, drawing, etc.).
• Model Experience (show what is expected)
• Group Students with other Learners (ELL w/non-ELL)
• Pairing of Students (PALS)
• Monitor Comprehension (Non-Threatening)
• Verbal Elaboration of Short-Answer (To increase language)
Olvera, P. CASP 2013
26
Effective Instruction
• The National Council of Teachers of English (NCTE,
2008) recommends the following:
• Present ELLs with challenging curricular content
(authentic, challenging, and rigorous)
• Set high expectations for ELLs
• Use technology effectively
• Recognize and value socio-cultural factors
• Position native languages and home environments as
resources (bridge home-school gap)
Olvera, P. CASP 2013
27
Effective Instruction
• Other Programs (Wendling & Mather, 2009) that
have demonstrated efficacy for ELLs:
– Success for All: successforall.net
– Reading Mastery: http://www.mcgrawhill.co.uk/sra/readingmastery.htm
– Read Well: http://store.cambiumlearning.com/read-well/
– Peer Assisted Learning Strategies:
http://store.cambiumlearning.com/read-well/
Olvera, P. CASP 2013
28
Response to Intervention
(RTI)
• US Department Institute of Education Sciences (IES)
Publication: Effective Literacy and English Language
Instruction for English Learners in the Elementary
Grades (Gersten et al., 2001) offers five
recommendations:
1.
2.
3.
4.
5.
Screen for reading and monitor progress
Provide intensive small-group reading intervention
Provide extensive vocabulary instruction
Develop academic English
Peer-assisted regular learning opportunities
Olvera, P. CASP 2013
29
Response to Intervention
• Six recommendations for implementing an effective program for ELLs
(English Instruction +Other Instruction; Brown-Chidsey, Bronaugh, &
McGraw, 2009, p. 132):
1.
2.
3.
4.
5.
6.
Screen in order to identify language development in L1 & L2.
Provide English (ELD) instruction as well as other skills (content): (ELD+ L2
Reading Instruction)
Monitor progress in English (development) and other skills being taught.
Adapt English instruction and assess progress
Adapt skills instruction (based on data)
Consider learning difficulties in both languages.
Possible referral for SPED Assessment
Olvera, P. CASP 2013
30
Response to Intervention
• Reading Interventions (Tier 1; Butterfield, 2010):
• Teach essential skills and strategies.
• Provide differentiated instruction based on assessment results
and adapt instruction to meet students' needs.
• Provide explicit and systematic instruction with lots of practice
with and without teacher support and feedback, including
cumulative practice over time.
• Provide opportunities to apply skills and strategies in reading
and writing meaningful text with teacher support.
• Don't just "cover" critical content; be sure students learn it.
• Monitor student progress regularly and reteach as necessary.
Olvera, P. CASP 2013
31
Response to Intervention
• Resources for ELLs (Brown-Chidsey, Bronaugh, &
McGraw, 2009, p. 137):
• AIMS Web Spanish Version
• Fostering Academic Success for English Language
Learners: What do we Know?
www.wested.org/policy/pubs/fostering
• Indicadores Dinamicos del Exito en la Lectura (DIBELS
Spanish)
• Teaching Literacy in English to K-5 English Learners
dww.ed.gov/priority_area/priority_landin.cfm?PA_ID
Olvera, P. CASP 2013
32
Summary
In Summary,
“For ELLs, in order for instruction to be effective the
assessment as well as instruction must be both
linguistically and culturally appropriate” (Butterfield,
2010, p. 20)
Olvera, P. CASP 2013
33
Pre-Referral
Considerations
Key Features for Success:
•
•
•
•
•
•
•
Schoolwide Commitment to High Expectations
Using L1 & L2 (intervention and skills building)
Students working together (PALS)
ELD + Core Content
Evidenced-based Instruction/Intervention
Progress Monitoring
SPED Assessment/consideration after all other resources
have been exhausted
Olvera, P. CASP 2013
34
SPED Assessment Considerations
Olvera, P. CASP 2013
35
Guidelines
CA Regs. 56320 (b)(1):
•Tests are to be provided and administered in the pupil’s primary language
unless clearly not feasible.
•Primary language = The language the student first learned, or the
language which is spoken in the person’s home.” (CA Regs. 3001(x))
•Key Issues:
• Have they been instructed in the primary language or has primary language
been used to teach academic content (ELD)??
• Is CALP (i.e, BVAT or CELDT) developed in their primary language?
Olvera, P. CASP 2013
36
Primary Language
Assessment
• Case Study:
• Johnny is a 2nd grade student referred for assessment due to concerns in reading
(fluency and comprehension).
• He grew up in a Spanish-speaking home. His parents are from Mexico and speak
only Spanish.
• His CELDT Score (Overall) indicates a level 3 (Intermediate). He is still an ELL
paper.
• He has been educated in English-only and speaks in English to his friends with the
occasional Spanish word (so his teachers will not understand what he is saying).
• Please identify primary and native language and his CALP score.
Olvera, P. CASP 2013
37
Identifying your ELL Student
The Bilingual Student
(Not all Bilinguals are Created Equal)
Olvera, P. CASP 2013
38
Bilingualism
(Adapted from Valdes & Figueroa, 1994, p. 16)
Type
Stage
Language Usage
First Generation-Foreign Born
A
Newly
Arrived
Understands little English. Learns a few words and phrases.
After Several years of residence
Ab
Type 1
Understands enough English to take care of essential everyday needs. Speaks enough
English to make self heard.
Ab
Type 2
Is able to function capably in the work domain where English is required. May still
experience frustration in expressing self fully in English. Uses immigrant language in all
other contexts.
Second Generation U.S. Born
Ab
Preschool
Acquires immigrant language first. May be spoken to in English by by relatives or
friends. Will normally be exposed to TV in English.
Ab
School Age
Acquires English. Uses it increasingly to talk to peers and siblings. Views English TV
extensively. Maybe literate only in English if schooled exclusively in English.
Olvera, P. CASP 2013
39
Identifying your ELL Student
• Types of Bilinguals:
• Limited Bilingualism (Subtractive Bilingualism/Arrested
Development)- When first language is replaced by new language
(English).
• Also referred to as semilingualism (Diaz, 1983). Students
develop limited levels of academic proficiency in both L1 & L2.
• Assessment Implications: Can seem SLI & LD. When
assessed in both languages is low in both (False Positive). Rule
out SLI.
Olvera, P. CASP 2013
40
Identifying your ELL Student
• Types of Bilinguals:
• Partial Bilingualism: Students reached native like proficiency in one of their
languages (English). Limited in primary (L1). No positive or negative effects in
regards to learning in L2 (Diaz-Rico & Weed, 2002).
• Assessment Implications: Assess in Native Language (L2). Assessment in
L1 may be biased.
• Semlingualism: Students are limited in both languages (L1 &L2). May be
“stuck” at a level 2 CALP in both languages.
• Assessment Implications: Assess in Native Language (L2) and English (L2).
Olvera, P. CASP 2013
41
Identifying your ELL Student
• Types of Bilinguals:
Proficient Bilingualism (Diaz-Rico & Weed, 2002): High
levels of proficiency in L1 & L2. Also known as Additive
bilingualism (opposite of subtractive bilingualism).
Byproduct of Dual Immersion programs.
Assessment implications: No need to assess in L1.
Olvera, P. CASP 2013
42
Identifying your ELL Student
• Most students in my experience have been the
following:
– Parent born in another country.
– Educated in the U.S. (English)
– Prefer to Speak in English
• We tend to treat them like:
– They were born in another country
– Educated in their primary language
– Prefer to speak L1
Olvera, P. CASP 2013
43
Basic Interpersonal Communication Skills (BICS)
Cognitive Academic Language Proficiency Skills (CALPS)
How we measure language proficiency
Olvera, P. CASP 2013
44
BICS
• Basic Interpersonal Communication Skills (BICS)• Socially acquired and exercised
• Precursor to CALP
• Context embedded
• Face to face/playground/lunch time communication
• Cognitive Academic Language Proficiency
• Academic in nature (Reading, writing, academic
discussions, etc.)
• Subsequent to BICS
• Context Reduced
Olvera, P. CASP 2013
45
CALP
Cognitive – Instruction is cognitively challenging and
requires students to use higher order thinking abilities
rather than the low-level memorization and application
skills that are tapped by typical worksheets or drill andpractice computer programs (Grigorenko, 2005, p. 39).
Olvera, P. CASP 2013
46
CALP
Academic - academic content (science, math, social
studies, art, etc.) should be integrated with language
instruction so that students acquire the specific
language or registers of these academic subjects
(Grigorenko, 2005, p. 39).
Olvera, P. CASP 2013
47
CALP
Language - the development of critical language
awareness should be fostered throughout the program
by encouraging students to compare and contrast their
languages (e.g. phonics conventions, grammar,
cognates, etc.) and by providing students with extensive
opportunities [to practice] (Grigorenko, 2005, p. 40).
Olvera, P. CASP 2013
48
CALP
Cognitive
(Higher Order
Thinking)
Language
Academic
(Language
Embedded)
(Content)
Proficiency
Olvera, P. CASP 2013
49
Language Acquisition Stages
(Jim Cummins, 1984)
CALP Language Acquisition Stages:
•
•
•
•
•
Stage I: Pre-production (0-6 months)
Stage II: Early production (6-months- 1 year)
Stage III: Speech emergence (1-3 years)
Stage IV: Intermediate fluency (3-5 years)
Stage V: Advanced Fluency (5-7 years)
Olvera, P. CASP 2013
50
50
CALP Examples
(Olvera & Cerrillo-Gomez, 2011)
Olvera, P. CASP 2013
51
Language Development
•
CALP Levels correspond to California English Development Test
(CELDT) and BVAT, WMLS, and WJ III
CALP level
CALP Level 1:
CELDT* Level
Beginning
BVAT/WMLS**/WJ III
Negligible-Very Limited
Preproduction
CALP Level 2: Early
Early Intermediate
Production
CALP Level 3: Speech Intermediate
Emergence
CALP Level 4:
Early Advanced
Intermediate Fluency
Very Limited-Limited
Limited to Fluent
Fluent to Advanced
CALP Level 5:
Advanced
Advanced
Advanced Fluency
*California English Language Development Test (CELDT)/ WMLS- Suggests 6 CALP Levels.
Olvera, P. CASP 2013
52
Language Development
• Why does CALP in L1 Matter for Bilinguals?
• Threshold Hypothesis: “Children must attain a
critical level, or threshold, of linguistic proficiency in
order to avoid cognitive deficit and allow their
bilingualism to their cognitive growth.”
(Diaz-Rico & Weed, 2002, p.42).
Olvera, P. CASP 2013
53
Olvera, P. CASP 2013
54
CALP
• Small scale study (N=23) demonstrated that
increasing CALP has been linked to significant
increases in reading standardized test scores Study
included SPED students. (Grigorenko, 2005).
• Interestingly, the SPED students did not benefit.
• The point: CALP significantly impacts reading
scores.
Olvera, P. CASP 2013
55
CALP
• Another study (N=77) demonstrated that the best
predictor of L2 reading growth was primary
language development. Furthermore, the higher
language proficiency in L1 the tendency to have
higher L2 proficiency (Laija-Rodriguez, Ochoa,&
Parker, 2006).
• Point 1: Correlation of L1 & L2 Reading
• Point 2: Higher L1= Higher L2
Olvera, P. CASP 2013
56
CALP
• Thomas & Collier (1997) looked at a data set of ELL Reading Scores
across five states (N= 42, 317).
• Looked at long-term effectiveness of various types of programs
serving ELLs. Programs included:
•
•
•
•
•
•
Two-way immersion (TWI)
One-Way Immersion Developmental + Content (OWD)
Transitional Bilingual Education + Content ESL (TBE + CESL)
Transitional Bilingual Education + Traditional ESL (TBE + TESL)
English as a Second Language + Content (ESL + Content)
ESL Pullout- Traditional
• Grades: K-12
• Normal Curve Equivalent (NCE) scores (mean of Native English=
50).
(cont’d)
Olvera, P. CASP 2013
57
CALP
• Thomas & Collier (1997)
• Results: All programs demonstrated steady upward progress in
regards to reading scores up to around fourth grade.
• By fifth grade, only TWI had hit NCE 50.
• By 9th grade, only TWI & ODE had surpassed the NCE of 50. TWI
had achieved 61 and ODE 52.
• All other programs demonstrated either stagnate or declining
performance after the fifth grade.
• Implications:
– All demonstrated initial success through the first four years of schooling.
– However, those programs that used primary CALP to learn English, had better
long-term outcomes.
Olvera, P. CASP 2013
58
California Department of Education
(Language Policy and Leadership Office)
Two-Way BE
70
Late-Exit BE +
Content ESL
60
NCE
50
Early-Exit BE +
Content ESL
40
Early-Exit BE +
Trad. ESL
30
20
ESL thru Academic
Content
10
ESL Pullout-Trad.
0
K
2
4
6
8
Grade
10
Olvera, P. CASP 2013
12
Native English
Speakers
59
CALP
• Thomas & Collier (2002)
• N= 5 School Sites ( Maine, Oregon, Texas, and Florida)
• Looked at the academic performance of ELLs in different types of
bilingual programs.
– Standardized Tools:
• Iowa Test of Basic Skills
• Stanford 9
• Terra Nova
• California Test of Basic Skills
• SABE (Spanish Achievement)
Olvera, P. CASP 2013
60
CALP
• Programs that were analyzed:
– Immersion (EO)
– Transitional Bilingual
– Maintenance
– Dual Immersion
• Results of NCE Scores (Mean= 50)
– Immersion- NCE= 25 by 11th grade
– Content Based ELL- NCE= 35 by 12th grade
Olvera, P. CASP 2013
61
CALP
• Implications From Study:
– Replicated the findings from the 1997 study.
– Positive results from maintenance and dual
immersion programs.
– English (L1) students scored above the national
average for English achievement and learned a
second language.
Olvera, P. CASP 2013
62
Possible Reasons for Low CALP
• Possible reasons for low CALP Scores:
 Can the student’s difficulty in acquiring English proficiency be attributed to his
or her insufficient development in his or her first language?
 Can the student’s academic difficulties or failure in an English-only academic
setting be attributed to his or her not having attained CALP in English?
 Was the student given ample instructional time in his or her first language to
(1) develop CALP in this language and (2) demonstrate ability somewhat within
the average range of academic performance? (Rhodes et al., 2009, p. 73)
Olvera, P. CASP 2013
63
Supports to Build CALP
Given the importance of CALP, how do we go about increasing CALP in
English?
• Cognitive Academic Language Learning Approach (CALLA):
Incorporates explicit teaching of learning strategies within content areas.
The goal is to enrich academic language while teaching academic subjects
(Chamot & O’Malley, 1994).
• Total Physical Response (TPR): Based on the association of language
and body movement (Asher, 1982).
• Content Based ELD: ELD Lessons are organized around academic
content.
How are we building CALP??
Olvera, P. CASP 2013
64
Guidelines
CA Regs 3023 (a):
•Assessment must done by someone who is competent in
the oral and written skills of the individual’s primary
language and who has a knowledge and understanding of
the cultural and ethnic background of the pupil?
•If not available, an interpreter must be used.
•Does your report document it?
• Does report indicate that validity may have been affected?
NASP Recommended Practices for Working with Interpreters
(NASP- Emilia C. Lopez)
Olvera, P. CASP 2013
65
What you will observe in
ELL’s with Reading Problems
(A Review of the Literature)
Olvera, P. CASP 2013
66
Academic and Learning Problems That ELLs may
Experience
• Possible Characteristics of ELLs with learning disabilities (GopaulMcNicol & Thomas-Pressword, 1998):
• Discrepancies between verbal and nonverbal learning.
• Perceptual disorders- alphabetic home language versus nonalphabetic (L1 & L2).
• Language Disorders- processing language, following directions, and
processing complex language (L1 & L2).
• Metacognitive Deficits- Slow language/information processing (low
CALP in L1 & L2))
• Motor Disorders- Lack of previous experience may impact
graphomotor skills (pencil).
Olvera, P. CASP 2013
67
Academic and Learning Problems That ELLs may
Experience
• Possible Characteristics of ELLs with learning disabilities
(Gopaul-McNicol & Thomas-Pressword, 1998):
• Social Emotional Functioning- Academic frustration and low
self-esteem. Can lead to low self-esteem and social skill
problems.
• Difficulty attending and focusing- distractibility, short attention
span, high motor activity (finger tapping, fidgety, etc.). Due to
cognitive overload from immersion in another language.
• Culture/Language Shock- Long term-acculturation stress,
withdrawal, anger, or sadness.
Olvera, P. CASP 2013
68
Academic and Learning Problems That ELLs may
Experience
• Possible Characteristics of ELLs with learning
disabilities (Gopaul-McNicol & Thomas-Pressword,
1998):
• Reading:
Low skills due to low interest
Slow oral reading
Short perceptual span (reading word by word)
Omission, substitution, and words in oral reading
Lack of comprehension
Lack of interest in reading overall
Olvera, P. CASP 2013
69
Understanding Warning Signs
• What are the warning signs (predictors) that an
ELL may have a reading disability (Gorman
2009, p. 252) ?
• History of speech/language delay or impairment in the
native language (L1).
• Communication difficulties at home (L1).
• Significant family history of learning disability/reading
disability.
• Difficulty developing literacy skills in the native language
despite instruction (L1).
• Limited progress compared with similar peers despite
high-quality intervention (L1 & L2).
Olvera, P. CASP 2013
70
Cross Linguistic Indicators of Reading Disability
• What are common core cross-linguistic
processing indicators of a reading disability
(Gorman, 2009, p. 250)?
• Phonological processing (Ga) deficits appear to be the most robust diagnostic
indicators of RD across languages and grade levels.
• In addition to word reading, working memory (Gsm), syntactic awareness
(Gc), and rapid naming (Glr) measures also contribute to identification of RD.
• Of skills measured in kindergarten, the single best predictor of ELLs’ word
reading and reading comprehension skills in later grades appears to be
phonological processing skills (Ga), followed by letter identification.
Olvera, P. CASP 2013
71
Academic and Learning Problems That ELLs may
Experience
• Possible Characteristics of ELLs with learning
disabilities (Roserberry-McKibben 2007):
• Writing:
Difficulties with spoken language can interact with writing.
Problems with following narrative (verbal) and reproducing
those narratives in written form.
The interaction of spoken language and reading
problems will impact writing as student progresses in school
(e.g., debates, expressing opinions, reading to learn).
Olvera, P. CASP 2013
72
ELL Learning Difficulties
Learning Issue (CAPPELL, 2011, P. 11)
Reason Difficulty Seen in Typical ELLs
Academic Learning difficulties
ELLs often have difficulty with grade-level
academic language and concepts
because it takes at least five years for
nonnative speakers to display nativespeaker like functioning in academics.
Language disorder
Lack of fluency and correct syntax is a
natural part of learning a new language.
Students may require more “wait time” as
they process an utterance in one
language and translate into another. This
“wait time” may be misinterpreted as a
language processing issue.
Olvera, P. CASP 2013
73
ELL Learning Difficulties
Learning Issue (CAPPELL, 2011, P. 11)
Reason Difficulty Seen in Typical ELLs
Attention and memory problems
ELLs may have difficulty paying attention
and remembering if they cannot relate
new information to their previous
experiences in their respective cultures.
ELLs may also be experiencing
exhaustion due to the task of learning in a
language in which they are not yet
proficient.
Withdrawn behavior
When students are learning new language
and adapting to a new culture a “silent
period” is normal. Also, this behavior might
be appropriate in the student’s behavior.
Olvera, P. CASP 2013
74
ELL Learning Difficulties
Learning Issue (CAPPELL, 2011, P. 11)
Reason Difficulty Seen in Typical ELLs
Aggressive behavior
The student may not understand appropriate
school behavior and language in the USA.
Also this behavior may be appropriate in the
student’s culture.
Social and Emotional problems
When students are learning to live in a new
culture and using a new language, social and
emotional problems often develop.
Olvera, P. CASP 2013
75
Difference or Disorder
Area
ELL
(Examples)
LD/SLI
(Examples)
Cognitive Ability
Achievement is usually commensurate
with measured cognitive ability and
length of school experience. Generally
better on nonverbal
Achievement is below cognitive
potential in some areas. Uneven
learning ability (all over the map.)
Academic Progress
Progress in L1 is based upon
appropriateness of language of
instruction. During L2 transition period
English may slow down.
May show progress in some areas and
lag in others. May make uneven
patterns of progress.
Language
L1 is age appropriate . All nonverbal
communication is culturally appropriate.
May not know word, but knows concept.
Sentence structure is appropriate to L1.
Difficulties w/auditory processing,
vocabulary, and difficulties transferring
concepts to everyday language.
Social-Emotional
May be socially isolated due to difference
in culture. May prefer to interact with
own culture. Communication is socially
appropriate.
May be socially inadequate. May lack
confidence.
Adapted from Larry P. Task Force Report (CDE, 1989).
Olvera, P. CASP 2013
76
The Process:
Determining Appropriate Language of Assessment
Olvera, P. CASP 2013
77
Evaluation
Discrepancy,
PSW, Larry P., Etc.
Pathways
Nonverbal, Modified,
English, and Native
Language, Combination.
Process
(Language of Assessment, Instrument
Selection, etc).
Pre-Referral
(Intervention, SST, RTI, Etc.)
Olvera, P. CASP 2013
78
Outcomes
• Reviewing the SST file and Interventions for Appropriate Referrals
• Assessing Exclusionary Factors (Cultural, Environmental and Economic
Disadvantage)
• Review key California guidelines that must be included in all ELL
psychoeducational reports
• Selecting Appropriate for Language of Assessment (L1 & L2)- Introduction to the
Multidimensional Assessment Model for Bilingual Individuals (Rhodes, Ochoa, and
Ortiz, 2005).
• Brief Overview of Appropriate use of Interpreters
• Introduce and Practice the Cultural Language Test Classifications (C-LTC) &
Cultural Language Interpretative Matrix (CLIM)- (Flanagan, Ortiz, Alfonso, 2007)
• Present Case Studies that Incorporate California Guidelines for Assessing ELL’s
and C-LTC & CLIM.
Olvera, P. CASP 2013
79
Table 1. Summary of Primary Language Assessment & Available Languages:
Tests that Measure CALP
(Olvera & Cerrillo-Gomez, 2011)
Test
Bilingual Verbal Abilities Test (Munoz-Sandoval,
Cummins, Alvarado, & Ruef, 1998).
Arabic, Chinese (Simplified and Traditional), English,
French, German, Haitian-Creole, Hindi, Italian, Japanese,
Korean, Polish, Portuguese, Russian, Spanish, Turkish, and
Vietnamese.
Woodcock Johnson III (Verbal)/Bateria III
(Verbal)
English and Spanish
Basic Inventory of Natural Languages (Herbert,
1986).
Arabic ,Armenian, Cambodian, Cantonese,
Chinese Creole , Dutch , English, Farsi , French , German ,
Greek , Hindi, Hmong , Ilocano, Inupiaq, Italian, Japanese,
Korean
Laotian ,Navajo Pilipino , Polish Portuguese,
Russian Spanish, Tagalog, Taiwanese, Toishanese,
Ukranian , Vietnamese
Woodcock-Munoz Language Survey – Update
(Woodcock & Sandoval, 2001).
English and Spanish
California English Language Development Test
(CDE, 2009).
IDEA Oral Language Proficiency Test (Dalton,
1991)
English
English and Spanish
Olvera, P. CASP 2013
80
Case Study #1
•
•
•
Sergio is a 12-year-old student in the 7th grade. He arrived to the United States from Mexico
as a 6th grader. He is the youngest of three siblings and lives with both of his parents. His
language arts teacher is concerned that he is making minimal progress in reading and
writing, for which reason she referred him to the pre-referral problem-solving team. At the
meeting, the parents inform the school that Sergio had received average to above average
grades across all academic areas while attending elementary school in Mexico.
The pre-referral team requested that English-language proficiency be administered in both
Spanish (L1) and English (L2) to establish language dominance. The Woodcock-Muñoz
Language Survey was administered by the bilingual specialist and the following was
obtained: Applied Language Proficiency (ALP) score in Spanish equivalent to a CALP level
of 5 out of 6 and the English ALP equivalent to a CALP level of 2 out of 6.
The team determined that Sergio demonstrated advanced level proficiency in Spanish and
very limited proficiency in English as evidenced by his performance. Because of his
advanced proficiency in his primary language, the team concluded that Sergio needed more
intensive English as a Second Language (ESL) services and agreed to meet again in 3
months to assess his progress. The language assessment clarified that a special education
referral would not be appropriate.
Olvera, P. CASP 2013
81
Case Study #2
Suzy is currently is a third grade student and is classified as ELL. She was born in the United
States to Peruvian-born immigrant parents. Her educational records indicate that she speaks
Spanish at home and has been educated in English only. She has been having difficulties in
language arts. Specific problems include poor phonemic awareness and fluency. According to
her pre-referral file, she has been receiving interventions and has been part of the response
to intervention program for the past year and a half. Suzy has made limited progress.
The pre-referral team recommended that she be assessed by the IEP team. Because she is
still classified as an English Language Learner (ELL), the school psychologist administered
the Oral Language sections of the WMLS (English & Spanish). The results indicated the
following: English CALP = 3 (Limited) & Spanish CALP = 1 (Negligible).
Although Suzy’s assessment results indicated that her English skills are limited, she is more
English-proficient than Spanish. Her negligible results in Spanish indicated that she has
minimal conversation skills in that language. The results led the school psychologist to
proceed with assessment in English.
Olvera, P. CASP 2013
82
Remember!!
• Possible reasons for low CALP Scores:
 Can the student’s difficulty in acquiring English proficiency be attributed to his or
her insufficient development in his or her first language?
 Can the student’s academic difficulties or failure in an English-only academic
setting be attributed to his or her not having attained CALP in English?
 Was the student given ample instructional time in his or her first language to (1)
develop CALP in this language and (2) demonstrate ability somewhat within the
average range of academic performance? (Rhodes et al., 2009, p. 73)
Olvera, P. CASP 2013
83
Select Most Appropriate Language
Multidimensional Assessment Model for Bilingual
Individuals (MAMBI)- Ochoa & Ortiz, 2005, p. 171
See attached Attachment 1- MAMBI Handout.
Olvera, P. CASP 2013
84
Select Most Appropriate Language
• Using the MAMBI (Ochoa & Ortiz, 2005):
• Looks at the following variables:
• Program history (ESL, bilingual, EO).
• Grade (K-7th)
• CALP Level (Language Proficiency)
• Minimal Proficiency (CALP 1-2)
• Emergent (CALP 3)
• Fluent (CALP 4-5)
• Helpful in determining the most appropriate language of
assessment.
•
•
•
•
Nonverbal (NV)
L1 (Primary)
L2 (Second language)
Bilingual (L1 & L2)
Olvera, P. CASP 2013
85
Select Most Appropriate Language
• Interpreting the MAMBI:
– O= Optimal mode of assessment
– √= Secondary or optional mode of assessment
– √*= Not recommended for K-2 but may be informative
for 3-4 (results may be underestimated).
– √#= Not recommended for K-1 but informative for 2-4
(may be underestimated).
Olvera, P. CASP 2013
86
Select Most Appropriate Language
• What would you recommend for:
• Student A:
Student that has received bilingual education
Low L1 and Emerging 2?
3rd Grade
• Student B:
Student has received EO (no ELD/ESL)
Low L1 & Low L2
5th Grade
Olvera, P. CASP 2013
87
Select Most Appropriate Language
• Student C:
Student that has received EO (No ESL)
Low L1 and Fluent 2?
7th Grade
• Student D:
Previously in Bilingual (now receiving ELD/ESL)
Fluent in L1 and Emerging in L2
2nd Grade
Olvera, P. CASP 2013
88
Key Guidelines
• Key California guidelines that must be included in
all ELL through your assessment pre-referral and
reports.
• See attached: Bilingual Template Evaluation Rubric
(Olvera, 2012; Handout #2)
• We will be reviewing this later.
Olvera, P. CASP 2013
89
Exclusionary Factors
Relevant Exclusionary Factors (EF):
A child will not be learning disabled IF his/her learning
problems are:“Primarily the result of Environmental, Cultural
or Economic Disadvantage.”
Olvera, P. CASP 2013
90
Exclusionary Factors
How do we define EFs?
“There is very limited information or research about
how this federal requirement is to be implemented”
(Rhodes, Ochoa, & Ortiz, p. 34).
Olvera, P. CASP 2013
91
Exclusionary Factors
Is it easy to establish the impact of EFs?
“A thorough assessment of the impact of
environmental, cultural, and economic factors on a
student’s performance is a difficult task and requires
consideration of of multiple factors” (Rhodes, Ochoa,
& Ortiz, p. 51).
Olvera, P. CASP 2013
92
Exclusionary Factors
Furthermore,
“it is important to note that as the number of
cultural and linguistic differences in a student’s
background increase, the greater the likelihood
that poor academic performance is attributable
primarily to such differences rather than a
disability” (Mascolo & Flanagan, 2011).
Olvera, P. CASP 2013
93
Exclusionary Factors
Additionally,
“The test [exclusionary factors] is not failed when such
external factors are present or when they may be
contributing to poor performance. The test is failed
only when those factors are determined to be primarily
responsible for the academic performance deficit”
(Flanagan, Ortiz, Alfonso, & Mascolo, 2006)
Olvera, P. CASP 2013
94
Exclusionary Factors
My conclusion: It is clinical judgment!
Our job is to analyze, address, and assess in order
determine impact of the exclusionary factor and
determine if it is the primary reason for
academic/cognitive difficulties.
See Attached Exclusionary Factor Checklist (See
Handout #3)
Olvera, P. CASP 2013
95
Accounting for Language of
Assessment in your Report
(Possible Statements)
Olvera, P. CASP 2013
96
Select Most Appropriate Language
Sample Statements for Language of Assessment Rationale:
Assessment Mode: L1 & L2
“Student is currently identified as an English Language Learner
(ELL); however, he prefers to speak English. He has a history of
receiving bilingual instruction at Jones Elementary. In addition to
an overall CELDT levels of Intermediate , Johnny has
demonstrated English and Spanish CALP levels of 3/5 (WJ III &
Bateria 3) on standardized assessment. Therefore, assessment
will be conducted in English, Spanish, and Nonverbal modalities
by a bilingual school psychologist”.
Olvera, P. CASP 2013
97
Select Most Appropriate Language
Sample Statements for Language of Assessment Rationale:
Assessment: In L2 (English)
“Johnny, whose home language is Portuguese, has been R-FEP
since 3rd grade. He has a history of receiving English-only
instruction at Jones Elementary. In addition to an overall CELDT
level of Intermediate, student demonstrated an English CALP
level of 3/5 (BVAT ELP = 90) and a Bilingual (English +
Portuguese) Verbal Ability (BVA) of 90 (SS) . Given Johnny’s
FEP status and current CALP scores, assessment will be
conducted in English.”
Olvera, P. CASP 2013
98
Select Most Appropriate Language
Sample Statements for Language of Assessment Rationale:
Assessment: Native Language Assessment
“Johnny, a 3rd grade student at Jones Elementary, whose home
language is Spanish, has been educated in El Salvador from K2nd grade. He has been classified as an English Language
Learner (ELL). In addition to an overall CELDT level of Early
Intermediate, Johnny demonstrated English CALP levels 1/5
(WJ III) and a verbal CALP score of 4/5 (Bateria III). Given his
demonstrated Spanish dominance and a history of Spanish
academic instruction, all cognitive and academic assessment
will be conducted Spanish”
Olvera, P. CASP 2013
99
Select Most Appropriate Language
Sample Statements for Language of Assessment Rationale:
Assessment: Interpreter and Nonverbal
“Student is currently identified as an English Language Learner (ELL).
His home language is Tagalog. Although he has BICS level English,
his academic English is limited. He has a history of English academic
instruction. In addition to an overall CELDT level of Early
Intermediate, language dominance exams were administered and the
following was obtained: English CALP 2/5 (BVAT ELP = 83) and his
Bilingual Verbal Ability (BVA) SS= 103). Given Johnny’s CALP levels,
assessment will be conducted in a nonverbal manner and verbal
parts of the assessment will be interpreted by qualified/certified fluent
interpreter.”
Olvera, P. CASP 2013
100
Interpreters
• According to the Clinical Evaluation of Language
Fundamentals Manual- 4th Edition (CELF-4;
Semel,Wiig, Secord, 2006), the following should be
considered when utilizing interpreters:
• Training varies widely (university, court, community, etc).
• Rigorous training required to interpret assessment
(SLI/Psycho-ed).
• Training should include: Language development, Psych-Ed
issues, testing practices, cultural factors, behavior
management, scoring, prompting, cueing, etc.
Olvera, P. CASP 2013
101
Interpreters
Please remember 
“It is not appropriate to enlist the aid of a school staff member,
parent, or sibling at the last minute to administer a test as
comprehensive as CELF-4 Spanish without training in these
issues.”
(Semel,Wiig, & Secord, 2006, p. 12)
Olvera, P. CASP 2013
102
Evaluation
Discrepancy,
PSW, Larry P., Etc.
Pathways
Nonverbal, Modified,
English, and Native
Language, Combination.
Process
(Language of Assessment, Instrument
Selection, etc).
Pre-Referral
(Intervention, SST, RTI, Etc.)
Olvera, P. CASP 2013
103
Assessment Pathways:
Given the RIOT data and language assessment
determination, various assessment pathways will be
discussed.
Olvera, P. CASP 2013
104
Outcomes
• Review Assessment Pathways
• Pros and Cons of the Abovementioned Assessment
Pathways
• Review Key California Guidelines that Must be
Included in all ELL Psychoeducational Reports
• Review Commonly Used Assessments with ELLs
• Case Study Review
Olvera, P. CASP 2013
105
Five Primary Pathways of Assessment:
• Modified/Adapted/Accommodations
• Nonverbal/Language-Free/Language-Reduced
Assessment
• Native Language/Primary Language (L1)
• English Only Testing
• Flexible/Dynamic (Bilingual)
Olvera, P. CASP 2013
106
Pathways to Assessing ELLs
Option #1: Modified/Adapted/Accommodations
Examples:
repeating instructions, accepting responses in both languages,
removing time, using interpreter, translation of documents,
rewording terms, and allowing extra explanation,
Pros:
Ensure understanding, more sensitive to individualized needs,
accommodate non-Spanish ELL’s.
Cons:
Violate standardization and thus validity/reliability of assessment.
Other
Considerations:
Always report all modifications and adaptations in your report and
how they may impact test results. You may only want to report raw
scores.
Olvera, P. CASP 2013
107
Pathways to Assessing ELLs
Option #2 Language-Free/Language Reduced Assessment
Examples:
Nonverbal instruments (UNIT, Leiter, KABC, WNV,
Etc.).
Pros:
Reduce Language and more universal in that it can
apply to a wider linguistic audience.
Limited in range of abilities tested. See Attachment 4
-CHC Coverage Handout (See Attachment #5).
Language is still present
Cons:
Other
Most disabilities are verbal in nature. You can never
Considerations totally reduce language.
:
Olvera, P. CASP 2013
108
Pathways to Assessing ELLs
Option #3: Native Language Testing (Primary Language)
Examples:
Assessing student with Spanish (or other language) norms. Test
publishers like the BVAT, WISC IV, and Bateria 3.
Pros:
Ensure linguistic appropriateness of the assessment. When student
has received instruction in another language. Maintain
standardization of assessment.
Cons:
Adequate norm representation. Instruction in the language of
assessment.
Remember that bilingualism is complex and thus capturing your exact
student can be challenging. Cultural factors.
Other
Considerations:
Other than Spanish, finding other native language
assessmentOlvera,
mayP. CASP
be difficult.
2013
109
Pathways to Assessing ELLs
Option #4: English-Only- Testing using English testing modalities.
Examples:
Although student is ELL, all assessment is conducted in English.
Pros:
Most research in this area, most of our ELL’s have been instructed in this
language.
We can predict that as the verbal demand increases scores will be lower
than English-speaking peers.
We can predict that as the visual demand increases scores will be
comparable to English-speaking peers.
Cons:
Testing may be perceived as bias and eligibility may not be based on
numbers alone.
Other Considerations:
Try not to taint assessment by modifying.
Olvera, P. CASP 2013
110
Pathways to Assessing ELLs
Option #5 (Olvera): Flexible/Dynamic Approach
Native language and
English
Bilingual instruction, Dual Immersion, and English Instruction.
Nonverbal and Native
Language
Instruction in another country and you have minimal access to instruments in
that language.
Modified/Accommodation Working with interpreter and have to translate items/instructions and also want to
s and Nonverbal
consider standard administration of items.
English and
Working with interpreter and also want to administer items in students language
Modified/Accommodation of instruction (English).
s
Nonverbal, Native
Language, and English
Bilingual instruction, Dual Immersion, and English Instruction. Also, if you are
concerned about language impairment.
Olvera, P. CASP 2013
111
Assessment Pathways
Modifications/Ad
apted/Accommo
dations
Native
Language
Mitigate the
effects of
test bias for
CLD
children.
When
student has
received
instruction/
exposure in
L1
Nonverbal
When you
do not have
access to
the student’s
L1
English
Only
When
instruction
has only
been in
English
Flexible and Dynamic
Olvera, P. CASP 2013
112
C-LTC/C-LIM
Flanagan, Ortiz, & Alfonso (2007)
Tools to help you select appropriate tools and
interpret findings:
• Cultural Language Test Classifications (C-LTC)
• Cultural Language Interpretative Matrix (C-LIM)
Olvera, P. CASP 2013
113
Broad
Ability
Catell-Horn Carroll (CHC)
Definition
Batteries
Fluid Reasoning (Gf)
The ability to reason, form concepts, and solve problems using
unfamiliar information or novel procedures. A good predictor of
intelligence
WJ; WECH; KABC;
SB; CAS
Crystallized Abilities
(Gc)
The breadth and depth of a person’s acquired knowledge, the ability to
communicate this knowledge (verbally) and the ability to reason using
previously learned experiences or procedures).
WJ; WECH; KABC;
SB; CAS; CELF;
CASL; PPVT; EVT;
CREVT; RO-WPVT;
TOLD
Long Term-Retrieval
(Glr)
The generation, perception, analysis, synthesis, storage retrieval,
manipulation, and transformation of visual patterns and stimuli. Other
skills involved include mental rotation and perception of spatial
configurations.
WJ; KABC; CAS;
CTOPP; WRAML
Visual Processing (Gv)
The ability to store information and efficiently retrieve it later.
WJ; WECH; KABC;
SB; CAS
Auditory Processing
(Ga)
The ability to apprehend, and hold information in immediate awareness
and then use it in a few minutes.
WJ, CTOPP, TAPS;
TOLD
Short Term Memory
(Gsm)
The perception, analysis, and synthesis of patterns among auditory
stimuli as well as the discrimination of subtle differences in patterns of
sound and speech when presented under distorted conditions. Important
in the development of speech/language abilities. Mostly encompasses
phonological awareness/processing abilities.
WJ; KABC; CAS;
CTOPP; WRAML;
CELF; TOLD;
Processing Speed (Gs)
The ability to perform automatic cognitive tasks, an aspect of cognitive
efficiency.
WJ; KABC; CAS;
CTOPP
Olvera, P. CASP 2013
114
CHC
Broad Ability
English
Spanish
NV
Instructions in Spanish
Gf
WJ; WECH; KABC; SB;
CAS;
Bateria &
WECH
CTONI; KABC,
DAS; UNIT
KABC & DAS
Gc
WJ; WECH; KABC; SB;
CAS; CELF; CASL; PPVT;
EVT; CREVT; RO-WPVT;
TOLD
Bateria; WECH;
CELF; PPVT;
NA
KABC
Glr
WJ; KABC; CAS; CTOPP;
WRAML;
Bateria
Leiter
NA
Gv
WJ; WECH; KABC; SB;
CAS
Bateria &
WECH
CTONI; KABC,
DAS; UNIT
KABC & DAS
Ga
WJ, CTOPP, TAPS; TOLD
Bateria & TAPS
NA
TAPS
Gsm
WJ; KABC; CAS; CTOPP;
WRAML; CELF; TOLD;
Bateria, TAPS;
WECH; CELF;
KABC, UNIT,
KABC & DAS
Gs
WJ; KABC; CAS; CTOPP
Bateria &
WECH
Leiter and UNIT
DAS
Olvera, P. CASP 2013
115
C-LTC/C-LIM
(Flanagan, Ortiz, & Alfonso 2007)
• Cultural Language Test Classifications (C-LTC)
• Data driven method to help establish validity of
assessment profile/profile.
• Based on evidence of ELD students and English
cognitive assessment performance.
• Addresses the issue of Difference v. Disorder
• Uses CHC Theory
• Used to help practitioner select tools that are lower in
culture and language demand.
Olvera, P. CASP 2013
116
Cultural Language Interpretative Matrix
(Flanagan, et al., p. 177)
Olvera, P. CASP 2013
117
C-LTC/C-LIM
(Flanagan, Ortiz, & Alfonso 2007)
• How do you use the C-LTC?
• Find the assessment/instrument that you will you or can
potentially use.
• Look at the patterns of linguistic and cultural loading as you
select and/or interpret assessment profile.
• Helpful in selecting parts/aspects of test, according to CHC,
in a cross battery assessment model.
Olvera, P. CASP 2013
118
Cultural Language Interpretative Matrix
(Flanagan, et al., p. 369)
Olvera, P. CASP 2013
119
Cultural Language Interpretative Matrix
(Flanagan, et al., p. 371)
KABC 2
DEGREE OF LINGUISTIC DEMAND
TRIANGLES (Gv-SR,Vz)
Hand Movements (Gsm-MS; GvMV)*
Pattern Reasoning (Gf-I, Gv-Vz)*
Face Recognition (Gv-MV)
Atlantis (Glr-MA, L1)
Atlantis Delayed (Glr-MA, L1)
HIGH
MODERATE
LOW
LOW
MODERATE
HIGH
NUMBER RECALL (Gsm-MS)
Block Counting (Gv-Vz)
Rebus (Glr-MA)
Rebus Delayed (Glr-MA, L1)
Conceptual Thinking (Gv-Vz; Gf-I)*
Rover (Gv-SS; Gf-RG)*
WORD ORDER (Gsm-MS, WM)
Gestalt Closure (Gv-CS)
Story Completion (Gf-I, RG; Gc-K0,
Gv-Vz)*
KABC 2
See Handout p. 20
Olvera, P. CASP 2013
Expressive Vocabulary (Gc-VL)
Riddles (Gc-VL, LD; Gf-RG)*
Verbal Knowledge (Gc-VL, K0)
120
DEMAND CLASSIFICATIONS OF THE CELF-4
(ORTIZ, 2005)
MODERA
TE
LOW
LOW
MODERATE
Number Repetition-Forward (Gsm)
Number Repetition-Backward (Gsm)
HIGH
Familiar Sequences (Gsm)
Concepts and Following Directions
(Gc/Gsm)
Recalling Sentences (Gsm)
HIGH
Sentence Structure (Gc-LS)
Word Structure (Gc)
Semantic Relationships (Gc)
Word Definitions (Gc)
Expressive Vocabulary (Gc)
Word Classes-Expressive (Gc)
Word Classes-Receptive (Gc)
Understanding Spoken Paragraphs
(Gc)
Formulated Sentences (Gc)
Sentence Assembly (Gc)
CELF 4
See Handout p. 32
Olvera, P. CASP 2013
121
Matrix of Cultural Loading and Linguistic Demand
Classifications of the CASL
(Ortiz, 2005)
MODERATE
HIGH
MODERATE
Grammatical Morphemes (Gc-MY)
Grammaticality Judgment (Gc-MY)
Syntax Construction (Gc-MY, OP)
Paragraph Comprehension (Gc-LS)
Sentence Comprehension (Gc-LS)
Sentence Completion (Gc-LD)
Basic Concepts (Gc-VL)
Antonyms (Gc-VL)
HIGH
LOW
LOW
Idiomatic Language (Gc-LD, )
Nonliteral Language (Gc-LD, )
Meaning from Context (Gc-LS)
Inference (Gc-LS)
Ambiguous Sentences (Gc-VL)
Pragmatic Judgement (Gc-LD, K0)
Synonyms (Gc-VL)
CASL
See Handout p. 34
Olvera, P. CASP 2013
122
DEMAND CLASSIFICATIONS OF THE CTOPP
(ORTIZ, 2005)
LOW
MODERATE
MODERATE
Rapid Digit Naming (Glr-NA)
Blending Nonwords (Ga-PC:S)
Sound Matching (Ga-PC:A)
Rapid Color Naming (Glr-NA)
Rapid Letter Naming (Glr-NA)
Blending Words (Ga-PC:A)
Segmenting Nonwords (Ga-PC:A)
Phoneme Reversal (Ga-PC:A,
Gsm-MW)*
Elision (Ga-PC:A)
HIGH
Memory for Digits (Gsm-MS)
Nonword Repetition (Gsm-MS)
HIGH
Rapid Object Naming (Glr-NA)
Segmenting Words (Ga-PC:A)
CTOPP
See Handout p. 35
Olvera, P. CASP 2013
123
Olvera, P. CASP 2013
124
C-LTC/C-LIM
(Flanagan, Ortiz, & Alfonso 2007)
What is the C-LIM?
Helps determine if excessive bias exists within a given
assessment profile.
Helps assess the validity of the assessment profile with
respect to cultural and linguistic loadings.
Olvera, P. CASP 2013
125
C-LTC/C-LIM
(Flanagan, Ortiz, & Alfonso, 2007)
•
How do you interpret findings (Flanagan, Ortiz, & Alfonso, 2007, p. 180):
1.
Is the uppermost left-hand corner (Low/Low)- The highest average score (when
compared to the rest)?
2.
Is the lowermost (right-hand) corner (High/High)- the lowest average (when
compared to the rest)?
3.
The the other cell average fall between the highest cell (Low/Low) and the
lowest cell (High/High) and follow a declining pattern (from the top left to the
bottom right)?
4.
If yes, test results may indicate high cultural and linguistic impact. Results may
be invalid.
5.
If no, the results may be valid and cultural and linguistic factors may not be an
issue.
ALWAYS USE IN THE CONTEXT OF OTHER DATA (RIOT).
Olvera, P. CASP 2013
126
Is Low/Low
Highest?
YES
Is High/High
NO
Lowest?
cultural and
linguistic may
not be
impacting.
YES
Are all other scores
between Low/Low &
High/High?
YES
Possible cultural and
Olvera, P. CASP
2013
linguistic
impact.
127
Olvera, P. CASP 2013
128
C-LIM/C-LTC
Sample Statement:
“Student’s profile does not follow a descending
pattern as would be expected in cases in which
culture and language may have impacted
cognitive performance. Therefore, this examiner
has concluded that these test results appear valid
and reliable”.
Olvera, P. CASP 2013
129
Olvera, P. CASP 2013
130
C-LIM/C-LTC
Sample Statement:
“Student’s profile follows a descending pattern as
would be expected in cases in which culture and
language may have impacted cognitive results.
Therefore, it is in examiner’s opinion that cognitive
assessment results may not be valid and reliable
for this student.”
Olvera, P. CASP 2013
131
Olvera, P. CASP 2013
132
C-LIM/C-LTC
Sample Statement:
“Student’s profile does not follow a descending
pattern as would be expected in cases in which
culture and language may have impacted cognitive
performance. Therefore, this examiner has concluded
these test results appear valid and reliable.”
Olvera, P. CASP 2013
133
C-LTC/C-LIM
(Flanagan, Ortiz, & Alfonso, 2007)
• Let’s Practice: WISC IV
Composite
Score
Verbal Comprehension
Similarities = 5
Vocabulary= 4
Comprehension= 6
Perceptual Reasoning
Block Design = 9
Picture Concept=13
Matrix Reasoning= 10
Working Memory
Digit Span= 3
Letter-Number= 4
Processing Speed
Coding= 6
Symbol Search=7
Olvera, P. CASP 2013
134
Olvera, P. CASP 2013
135
C-LTC/C-LIM
(Flanagan, Ortiz, & Alfonso, 2007)
• Let’s Practice:
• CELF 4
Composite
Score
Receptive Language
Concepts and Following Directions= 6
Word Classes= 11
Sentence Structure= 4
Expressive Language=
Word Structure= 6
Recalling Sentences= 7
Formulated Sentences= 7
Core Language
Concepts and Following Directions= 6
Word Structure= 6
Recalling Sentences= 7
Formulated Sentences= 7
Language Content
Concepts and Following Directions= 6
Word Classes= 11
Expressive Vocab= 9
Language Structure
Sentence Structure= 4
Recalling Sentences= 7
Formulated Sentences= 7
Olvera, P. CASP 2013 Sentence Structure= 4
136
Olvera, P. CASP 2013
137
C-LTC/C-LIM
(Flanagan, Ortiz, & Alfonso, 2007)
Let’s Practice: CAS
Composite
Score
Planning
Matching Numbers= 8
Planned Connections= 8
Attention
Expressive Attention= 13
Number Attention=11
Simultaneous
Nonverbal Matrices= 8
Verbal Spatial Relations=4
Successive
Word Series=6
Sentence Repetition=4
Olvera, P. CASP 2013
138
Olvera, P. CASP 2013
139
Putting it All Together
Consult
C-LTC
Program
History?
Assess
Language
(L1 & L2)
MAMBI
C-LIM
Interpretation
Olvera, P. CASP 2013
140
C-LTC/C-LIM
(Flanagan, Ortiz, & Alfonso (2007)
• In an advanced Cross Battery Model:
– The C-LTC can be used to select all abilities/constructs (Gv,
Gc, Ga….) that have the lowest degree of culture and
language.
 For example: WISC and KABC
 Goal would be to hit all 7 abilities with the lowest degree of culture
and language demands.
 Recommendations for this: District In-service on Cross Battery
Assessment (process).
Olvera, P. CASP 2013
141
CHC
Broad Ability
English
Spanish
NV
Instructions in Spanish
Gf
WJ; WECH; KABC; SB;
CAS;
Bateria &
WECH
CTONI; KABC,
DAS; UNIT
KABC & DAS
Gc
WJ; WECH; KABC; SB;
CAS; CELF; CASL; PPVT;
EVT; CREVT; RO-WPVT;
TOLD
Bateria; WECH;
CELF; PPVT;
NA
KABC
Glr
WJ; KABC; CAS; CTOPP;
WRAML;
Bateria
Leiter
NA
Gv
WJ; WECH; KABC; SB;
CAS
Bateria &
WECH
CTONI; KABC,
DAS; UNIT
KABC & DAS
Ga
WJ, CTOPP, TAPS; TOLD
Bateria & TAPS
NA
TAPS
Gsm
WJ; KABC; CAS; CTOPP;
WRAML; CELF; TOLD;
Bateria, TAPS;
WECH; CELF;
KABC, UNIT,
KABC & DAS
Gs
WJ; KABC; CAS; CTOPP
Bateria &
WECH
Leiter and UNIT
DAS
Olvera, P. CASP 2013
142
Cross Battery Website
Cross Battery Assessment Website
http://www.crossbattery.com/
Olvera, P. CASP 2013
143
Key Issues
• In summary,
• We discussed five pathways to assess ELLs.
• Assessing ELLs is a fluid and dynamic process.
• District language designation, CALP, history of instruction,
country of origin, and home language/exposure will generally
guide you in determining language of assessment.
• Recommend using CHC as cognitive model.
• C-LTC/C-LIM is a tool to help you interpret findings based on
clinical studies of ELL performance on that particular test.
Must be interpreted in context of other factors.
Olvera, P. CASP 2013
144
Evaluation
Discrepancy,
PSW, Larry P., Etc.
Pathways
Nonverbal, Modified,
English, and Native
Language, Combination.
Process
(Language of Assessment, Instrument
Selection, etc).
Pre-Referral
(Intervention, SST, RTI, Etc.)
Olvera, P. CASP 2013
145
Evaluation
of ELL-related Psychoeducational Case Study Data:
Traditional and Emerging Models
Olvera, P. CASP 2013
146
Intended Outcomes
• Review ELL-related due process cases
• Review the Bilingual Evaluation Rubric
• Review validity and reliability for commonly used
instruments with ELLs
• Compatibility of School Psychology (SP) and Speech and
Language Pathologist (SLP) Assessment
• Review different identification approaches (Evaluation)
• Review case studies which incorporate Assessment
Pathways
• Systemic implications and review
Olvera, P. CASP 2013
147
ELL Related Due Process Cases:
What can we Learn?
Disclaimer: Reviewing of these cases does not imply legal counsel. Please discuss issues with
your district administration and attorneys. Review of cases strictly for educational purposes.
Olvera, P. CASP 2013
148
Due Process Cases
OAH Case #: 2009010744
Facts:
•
•
•
•
•
•
15 year old Student is ELL and was born in Mexico.
Has been in US since 6 years old.
Enrolled in an English-immersion program.
Spanish spoken at home.
Speaks in English and Spanish at school.
Answers in English during class work.
Olvera, P. CASP 2013
149
Due Process Cases
• Issues:
– Did district appropriately assess in all areas of suspected
disability (psychoeducational and speech/language)?
– Should district fund an IEE (psychoeducational and
speech/language)
– “Student contends that the District’s assessments were
inappropriate because the District did not administer them to
her in Spanish, which is her native language”.
Olvera, P. CASP 2013
150
Due Process Cases
• School Psychologist (Monolingual English):
– Classroom observation
– No interpreter used
– Kaufman Assessment Battery for Children 2nd Edition (KABC
2)- Mental Processing Index (MPI)- (Minimal English)
– Comprehensive Test of Nonverbal Intelligence (CTONI 2).
– VMI 5
– Supplemented findings with parent, teacher, and classroom
observations due to student’s ELL status.
Olvera, P. CASP 2013
151
Due Process Cases
• Examiner Provided:
– Redirection
– Repetition
– Provided Clarification
• Psychologist noted that student understood during testing.
• English Academics- WJ III Achievement
Olvera, P. CASP 2013
152
Due Process Cases
• Two Speech and Language Pathologists (SLP):
– Monolingual and Bilingual
– Observed
– Reviewed records
• Receptive One Word Picture Vocabulary Test (ROWPVT)- English
• Expressive One Word Picture Vocabulary Test (EOWPVT)- English
• Bilingual SLP:
– Observations, teacher interviews, language samples (English & Spanish).
– No problems noted in either language.
Olvera, P. CASP 2013
153
Due Process Cases
Conclusions:
“The District met its burden of showing that its cognitive
development assessment, sensory motor processing
assessment, speech and language assessment, academic
achievement assessment, and social/emotional/behavioral
assessment were appropriate.”
Olvera, P. CASP 2013
154
Due Process Cases
OAH Case #: 2010050822
Facts:
• 19-year old Student is ELL and was born in rural Central America.
• Has been in US since 16-years old.
• Attended school in Honduras from K-5th grade.
• Has lived in foster homes.
• When arrived in district and placed in 11th grade.
• Beginning in English (CELDT) & LDA (Limited in Primary & NonLiterate).
• Grades in 11th and 12th ranged from F-B (ELD & Art).
Olvera, P. CASP 2013
155
Due Process Cases
• Issues (Expert Witness Opinion):
– District did not assess cognitive abilities appropriately.
– District did not assess in all areas of suspected disability
(psychoeducational and speech/language).
– District engaged in cultural bias by assuming that low
test scores were indicative of limited educational
background.
– District failed to gather information from family.
Olvera, P. CASP 2013
156
Due Process Cases
• School Psychologist (Bilingual):
RIOT
BVAT- Very Limited in ELP and BVA
CTONI (Composites Borderline-Low Average)
Vineland (TRF)- Borderline
VMI- Borderline
TAPS (Bilingual)- Low Average to Average (across
composites).
– BASC- No major concerns (Teacher & Student).
Socialization at the at-risk.
–
–
–
–
–
–
Olvera, P. CASP 2013
157
Due Process Cases
• School Psychologist (Bilingual):
– Bateria III (Ach)- Composites ranged from 60’s to 70’s.
Consistent with classroom grades and achievement.
– Conclusion: No severe discrepancy in scores.
– SP stated:
“(Student’s) limited school experience cannot be ruled out
as a primary factor in his performance of standardized tests of
intellectual achievement, psychological processing, and
academic achievement.”
Olvera, P. CASP 2013
158
Due Process Cases
• Expert Witness: Bilingual Clinical
Neuropsychologist.
– Opinion Regarding Assessment:
– CTONI- Not valid due to composite scatter (borderlinelow average) and thus district could not establish a
significant discrepancy.
– Expert stated that scatter was due to executive
functioning, attention, and memory problems. More
testing should have been completed.
Olvera, P. CASP 2013
159
Due Process Cases
• Expert Witness: Bilingual Clinical
Neuropsychologist.
– Opinion Regarding Assessment:
– TAPS Bilingual: Scattered scores render scores
inaccurate. Claimed scatter was due to executive
functioning, attention, and memory. More testing should
have been completed.
Olvera, P. CASP 2013
160
Due Process Cases
• Judges statements regarding Expert Opinion (EO):
– Credentials render him an excellent witness.
– EO relied only on scores and not a variety of sources (interview,
observations, etc.)
– EO did not assess student
– Did not observe student in class.
– “Did not address the impact that Student’s limited educational
background had on his intellectual abilities, psychological
processing, and academic achievement.”
– Did not reconcile that student, though MR/ID-borderline level
scores, was making progress academically in the classroom and
state scores.
Olvera, P. CASP 2013
161
Due Process Cases
• Speech and Language Pathologist (Bilingual):
– RIOT
– CELF 4 (Spanish)- Composites ranged from Borderline to Low
average. Overall Score Borderline Range (Limited Schooling).
Social Deficits (Clasificacion Pragmatica). CELF was not
administered in its’ entirety.
– Expressive One Word (Spanish/Bilingual)- Informal
Administration- Low level scores due to limited schooling.
– Receptive One Word (Spanish/Bilingual)- Informal AdministrationLow level scores due to limited schooling.
– Interpersonal Communication Checklist- Age appropriate levels.
– Clinical Discourse Analysis- Immature skills.
Olvera, P. CASP 2013
162
Due Process Cases
• Speech and Language Pathologist (Bilingual):
– Conclusion “Student was not eligible for special
education as a pupil with a speech and language
disorder.”
– Any deficits in skills were due to limited schooling.
– Deemed a very credible witness by the judge.
Olvera, P. CASP 2013
163
Due Process Cases
• Expert Witness: Bilingual Clinical
Neuropsychologist.
– Opinion Regarding SLP Assessment:
– Should not have administered parts of the test (CELF).
– EOWPVT (Bilingual) & ROWPVT (Bilingual) should not
be used informally.
– “He wanted [SLP] to use instruments normed for
Student, but he could not name any such assessments”.
Olvera, P. CASP 2013
164
Due Process Cases
• Judges statements regarding Expert Opinion (EO):
– “Did not offer any persuasive evidence against the
appropriateness of the District speech-language
assessment of Student.”
Olvera, P. CASP 2013
165
Due Process Cases
• Conclusions:
– Was district’s assessment appropriate?
– SP utilized an assessment battery that was appropriate. The
battery consisted of technically sound instruments, including
the CTONI-2, the BVAT, the Vineland II TRF, the Beery VMI tests,
the TAPS-3:SBE, the BASC-2, and the Bateria III. Where
appropriate, [SP] expressed caution with the interpretation of
test results when Student was not included in the normative
population of the assessment instrument. The battery
consisted of assessments and materials that were valid for the
intended purpose of the particular instrument. SP utilized the
assessment instruments in accordance with the instructions
provided by the test producers.”
Olvera, P. CASP 2013
166
Due Process Cases
• Conclusions:
– Was district’s assessment appropriate?
• SP utilized an assessment battery that was free of bias and
that was designed to yield accurate information on
Student’s academic, developmental and functional skills.
Except for the BVAT, SP conducted the psychoeducational
assessment of Student in Spanish which is his dominant
language.
Olvera, P. CASP 2013
167
Due Process Cases
• Conclusions:
– Did district failed to gather information from family?
• Student was 18 at the time of assessment and the attorney
did not allow parent contact.
– Did district appropriately assess in all areas of
suspected disability (psychoeducational and
speech/language)?
• SP performed a global assessment of Student that covered
his intellectual functioning, psychological processing and
academic achievement.
Olvera, P. CASP 2013
168
Due Process Cases
• Conclusions:
– What about SLP evaluation?
• The speech-language assessment performed was appropriate. This
evaluation satisfied the procedural requirements for an initial
evaluation of a pupil referred for possible special education assistance.
• SLP utilized a variety of assessment tools and strategies in his speech
language evaluation of Student.
• Very qualified and experienced SLP.
• Student failed to present evidence showing the need for SLP to administer
every subtest in the CELF-4:Spanish. In this regard, SLP was entitled to
exercise his professional judgment in conducting the speechlanguage assessment of Student.
Olvera, P. CASP 2013
169
Due Process Cases
Order:
“The psychoeducational assessment and speechlanguage assessment of Student performed by the
District were appropriate. Student is not entitled to
receive at public expense independent education
evaluations regarding such assessments.”
Olvera, P. CASP 2013
170
Due Process Cases
• What can we learn?
– Language dominance was established.
– An ELL student was assessed in English based on English education
(English Immersion). Student spoke primarily in English.
– Several sources of information were considered (RIOT) in the context of
standard scores.
– Verbal and Nonverbal measures were considered.
– Nondiscriminatory assessments were recognized.
– Examiners were qualified and trained.
– Examiners were allowed to use formal and informal measures.
– Examiners disclosed statements about validity and reliability when nonstandardized instruments were used.
– Examiners were allowed to use professional discretion!
Olvera, P. CASP 2013
171
Brief Overview Commonly Used
Assessments:
Reliability and Validity Issues Related to
ELLs.
(Peer Reviewed Perspectives)
Olvera, P. CASP 2013
172
Commonly Used Instruments
Test
Language
Norm
Sample
Reliability
Validity
Miscellaneous
Bilingual
Verbal
Abilities
Test
(BVAT)
English & 16+
Languages
8,818
Subtests (.80.90).
Good validity
with other
measure of
language.
Allows
demonstration of
home language.
Few bilinguals in
sample.
Ages 590+
Lack of control for
acculturation
factors.
Untranslatable
words were not
included.
Muñoz-Sandoval, Cummins, Alvarado, & Ruef, 1998
Olvera, P. CASP 2013
173
Commonly Used Instruments
Test
Language
Norm
Sample
Reliability
Validity
Miscellaneous
WoodcockMuñoz
Language
SurveyRevised
(WMLS-R)
English and
Spanish
8,782
Subtests (.76.97)
Good
correlations
with other
tests of
language.
CALP in English and
Spanish
Composites
(.88-.98)
Comprehensive
(includes academics)
Can be time
consuming
Ages 2
years to
Adult.
Spanish version not
clear about
SES, gender, and/or
geographic location of
sample.
Alvarado, Ruef, and Schrank, 2005
Olvera, P. CASP 2013
174
Commonly Used Assessments
Cognitive Measures
Verbal
Olvera, P. CASP 2013
175
Commonly Used Instruments
Test
Language
Norm
Sample
Reliability
Validity
Miscellaneous
Bateria III
Spanish
1,413
Mexico, Cuba,
Colombia,
Argentina, Chile,
Costa
Rica, Dominican
Republic,
Ecuador,
Guatemala,
Honduras, Peru,
Puerto Rico,
Uruguay,
Venezuela,
and one
unreported
location
1, 293
(USA)
Subtest
(Median= .80-.93)
Cluster
(All Ages= .86.95).
Confirmatory
Factor Analysis
supports strong
support for the
CHC Model.
Normed on native
Spanish speakers
both in and outside
USA.
Ages 2 yrs
to 90+
120 from
Latin
America
How do you factor
in bilingual
students?
Helpful in
comparing English
v. Spanish CALP.
Schrank, McGrew, Ruef, Alavardo, Muñoz-Sandoval, Woodcock, 2005
Olvera, P. CASP 2013
176
Commonly Used Instruments
Test
Language
Norm
Sample
Reliability
Validity
Miscellaneous
Wechsler
Intelligence
Scale for
Children 4thSpanish (WISC
IV Spanish)
Spanish
851 (ELLs
that had
been
educated in
the US <5
years).
Subtest
( .74-.90).
Good concurrent
validity (UNIT &
CELF).
Direct translation of the
English.
Mexico, Cuba,
Puerto Rico, and
the Dominican
Republic, and
Central and South
America.
Cluster
(.82-.97).
< 5 years of US education.
Bilingual Spanish speakers in
sample.
Equated norms not actual
separate Spanish norms.
However, percentiles are
based on parent education &
years of schooling (student).
Ages 6-0 to 1611
Practice effect when
administering English and
Spanish.
Clinton, 2007
Olvera, P. CASP 2013
177
Commonly Used Instruments
Test
Language
Norm
Sample
Reliability
Kaufman
Assessment
Battery for
Children 2nd
Edition
(KABC 2)
Mental
Processing Index
(MPI)
(Minimal
Language)
3, 025
across
USA.
Internal
reliability
(split half)=
.72- .92 for
all ages.
Ages 3-18
Validity
Confirmatory
Factor Analysis
(CFA)
demonstrates
good measures
of Gf, Gsm, Gv,
Test- Retest Gc, and Glr.).
= .72-.82.
Good
correlations with
WISC IV & WJ
III.
Miscellaneous
Neuropsychological
model (Luria).
Processing oriented.
Recommended for
bilingual ELLs.
Translated directions are
a direct translation of the
English.
Not based on ELL
samples.
Criticism about lack of
utility for educational
planning.
Olvera,
CASP 2013 2005
Kaufman, Lichtenberger, Fletcher-Janzen,
&P.Kaufman,
178
Commonly Used Assessments
Language-based Measures
Olvera, P. CASP 2013
179
Commonly Used Instruments
Test
Language
Norm
Sample
Reliability
Validity
Miscellaneous
Clinical
Evaluation
of Language
Fundamenta
ls 4th Edition
(Spanish)CELF-4
Spanish
Spanish
1,100
Subtest
Test –
Retest:
.68-.91
(All ages).
Demonstrate
appropriate
levels of
validity for
interpretation
of Spanish
scores,
validity of
constructs,
and
correlations
with other
similar
measures.
Bilingual children with
Spanish being primary
language.
Cuba,
Dominican
Republic,
Mexico, Puerto
Rico, Central
and South
America, and
other.
Cluster:
.79-.88
Ages 5-21
Semel, Wiig, and Secord, 2006.
Olvera, P. CASP 2013
30% lived in homes
that spoke a language
other than Spanish as
follows:
96% English
3.2 % Other (French,
Italian, and
Portuguese).
180
Commonly Used Instruments
Test
Language
Norm Sample
Reliability
Validity
Miscellaneous
Expressive
One Word
(SpanishBilingual
Edition)
Spanish
1,050
(bilingual)
.92-.98
Good content
validity
(rigorous item
selection
procedure).
Co-Normed with
ROWPVT
Approximated
U.S. Hispanic
population
Expressive
One Word
Very good
construct
validity.
Ages 4-12:11
Excellent reviews.
Considers dialectical
nuances (normed on
Hispanics in US).
Slight
overrepresentation of
Mexican Americans.
Manual claims that it can
be used as an
intelligence test (solely
verbal). Should not be
used in that manner.
Brownell, 2001
Olvera, P. CASP 2013
181
Commonly Used Instruments
Test
Language
Norm Sample
Reliability Validity
Receptive
One
Word
(SpanishBilingual
Edition)
Spanish &
English
(combined
language).
1,050
(bilingual)
.92-.95
Approximated
U.S. Hispanic
population
Bilingual
Expressiv Test
e One
Word
Ages 412:11
Miscellaneous
Good content Co-Normed with the
validity
EOWPVT
(rigorous item
selection
Excellent reviews.
procedure).
“Very good measure
Very good
of bilingual receptive
construct
ability.”
validity.
Slight
overrepresentation of
Mexican Americans.
Spanish uses ‘Tu”
And “usted” which
may confuse some.
Krach, 2005
Olvera, P. CASP 2013
182
Commonly Used Instruments
Test
Language
Norm Sample
Reliability
Test of
Auditory
Processing
Skills 3Bilingual
(TAPS 3
Bilingual )
English &
Spanish
851 Spanish
(bilingual
children) in the
US and Puerto
Rico.
.53-.90
Validity is well
(index level). documented.
Test-retest
.60-.95 (14
Days).
West Coast USA
(N= 671).
Ages 5-0- 1811
Validity
Good construct
validity.
Good predicative
validity of
academic scores
(WJ III)
Miscellaneous
Not a direct translation.
Data regarding
subcategories of Hispanics
not provided.
Norm sample does not
exactly approximate US
population.
Good reviews of overall
utility.
Reviewers indicated that
other factors are measured
as well (verbal ability).
Good attention to dialectical
differences.
Vetter, 2010.
Olvera, P. CASP 2013
183
Commonly Used Instruments
Test
Language
Norm Sample
Test de
Vocabulario
en Imagenes
Peabody
(TVIP)
Spanish
(Mexican
and Puerto
Rican)
Mexican
Manuel
Version= 1, 219 does not
(Mexico City)
state
reliability.
Puerto Rican
Version =
All
1, 488
reliability
studies
based on
the PPVT
(English).
Ages 2-616-11
Reliability
Validity
Miscellaneous
Established
content,
construct, and
concurrent
validity.
Uses dialectically
appropriate Mexican and
Puerto Rican Spanish.
Good
correlations
with other
receptive
vocabulary
assessments
Good reviews for
receptive language
measure.
Good norm sample.
Need to update norms
(1986).
Indicates
that it is
similar to
English.
Gonzalez, 2001
Olvera, P. CASP 2013
184
Commonly Used Assessments
Non-Verbal Measures
Olvera, P. CASP 2013
185
Commonly Used Instruments
Test
Language
Universal
Nonverbal
Intelligence
Test (UNIT)
Nonverbal &
2,100
Mediated
Language Tasks
Universal
Gestures
Norm
Sample
ELLS &
Bilingual
Education
Reliability
Validity
Miscellaneous
Subtests
(.64- .91).
Strong
concurrent
validity with
WISC III &
Bateria R.
Limited range of CHC
abilities (non verbal).
Composites
(.83-.93).
No evidence of
predicting achievement
mentioned in manual.
Bracken & McCallum, 1998
Olvera, P. CASP 2013
186
Commonly Used Instruments
Test
Language Norm
Sample
Reliability
Leiter-R
Nonverbal
Subtests
(.69-.90)
1,719
Nationally
Representative.
(Included
ELLs).
Validity
Good validity
demonstrated
when
compared
Composites against the
(.80s to .90s) WISC III.
for 11-20
years olds.
Miscellaneous
Studies of bias at the
subtest have
demonstrated few
significant differences
between Caucasian and
Hispanic samples.
Based on CHC Model.
Good predictor of
academics.
Excellent peer reviews.
Roid & Miller, 2002
Olvera, P. CASP 2013
187
Commonly Used Instruments
Test
Language
Norm
Sample
Reliability
Validity
Miscellaneous
Kaufman
Assessment
Battery for
Children 2nd
Edition (KABC
2)
Nonverbal & MPI
(Minimal
Language)
3, 025
across
USA.
Internal
reliability=
..72- .92 for
all ages.
Confirmatory
Factor Analysis
(CFA)
demonstrates
good measures
of Gf, Gsm, Gv,
Gc, and Glr.).
Direct Translation of the
English.
Test- Retest
= .72-.82.
Not based on ELL samples.
NVI subtests can be
administered with gestures.
Good correlations Criticism about lack of utility
with WISC IV &
for educational planning.
WJ III.
NVI correlates best with
math (.60-.67), reading
(.60), and writing (.50-.60),
and oral language (.50-.60)
Kaufman, Lichtenberger, Fletcher-Janzen, & Kaufman, 2005
Olvera, P. CASP 2013
188
Commonly Used Instruments
Test
Language
Norm
Sample
Reliability
Validity
Miscellaneous
Comprehensiv
e test of
Nonverbal
Intelligence 2nd
Edition
(CTONI 2).
Nonverbal
2, 827
Composite
reliability
ranged from
.90-.95
(internal
consistency)
, 80 (or
greater) for
test retest,
and
interscorer=
.95.
Good construct
and criterion
prediction validity.
Decreases language and
motor demands.
Ages 6-89
years.
Instructions
English, Spanish,
Chinese, French,
Tagalog,
Vietnamese,
German and
Korean.
Good predictor of
intelligence and
achievement
scores.
Multiple languages
(directions).
Not standardized using the
non-English languages.
Possible bias on test items (
i.e., American football).
Delen, Kaya, & Ritter, 2012
Olvera, P. CASP 2013
189
Commonly Used Instruments
• What did I learn?
– There are no perfect tests.
– Know limitations
– Know Strengths
– Interpret within the context of outside data (RIOT)
Olvera, P. CASP 2013
190
Is SP and SLP Assessment Compatible from a CHC
Perspective?
Olvera, P. CASP 2013
191
Is SP and SLP Assessment Compatible
from a CHC Perspective?
CHC Area
/Battery
KABC 2
TAPS
CELF
WRAML
Fluid Reasoning
(Gf)
X
Crystallized
Abilities (Gc)
X
Long TermRetrieval
(Glr)
X
X
Visual Processing
(Gv)
X
X
Auditory
Processing (Ga)
Short Term
Memory (Gsm)
X
VMI
X
X
X
X
X
X
X
X
X
X
Processing Speed
(Gs)
Olvera, P. CASP 2013
192
Is SP and SLP Assessment Compatible
from a CHC Perspective?
CHC Area
/Battery
WISC IV
Fluid Reasoning
(Gf)
X
Crystallized
Abilities (Gc)
X
Long TermRetrieval
(Glr)
Visual Processing
(Gv)
CTOPP
TOLD 4
X
X
Processing Speed
(Gs)
X
X
X
X
Short Term
Memory (Gsm)
VMI
X
X
Auditory
Processing (Ga)
CAS
X
X
X
X
X
X
X
X
Olvera, P. CASP 2013
193
Is SP and SLP Assessment Compatible
from a CHC Perspective?
CHC Area
/Battery
WJ III
CTOPP
Fluid Reasoning
(Gf)
X
Crystallized
Abilities (Gc)
X
Long TermRetrieval
(Glr)
X
Visual Processing
(Gv)
X
Auditory
Processing (Ga)
X
X
Short Term
Memory (Gsm)
X
X
Processing Speed
(Gs)
X
CASL
WRAML
VMI
X
X
X
X
Olvera, P. CASP 2013
X
X
194
Is SP and SLP Assessment Compatible
from a CHC Perspective?
WJ III/WISC IV
CELF -4
Understanding
Directions
Concepts/Followi
ng Directions
Sentence Recall
Recalling
Sentence
Similarities
Word Classes
Picture
Vocabulary
Expressive Voc
Vocabulary
Word Definitions
Story Recall
Understanding
Spoken
Paragraph
Sound
Awareness
Phonological
Awareness
Olvera, P. CASP 2013
195
Is SP and SLP Assessment Compatible
from a CHC Perspective?
WJ-III
WISC-IV/WIATIII
TOLD-P:4
Picture Vocabulary
Picture Vocabulary
Vocabulary
Oral Vocabulary
Receptive Voc
(WIAT III)
Syntactic
Understanding
Sentence Mem
(wiat iii)
Sentence Imitation
Auditory Attention
Word
Discrimination
Sound Awareness
Phonemic Analysis
Word Articulation
Similarities
Relational
Vocabulary
Morphological
Olvera, P. CASPCompletion
2013
196
ELL Case Studies:
Evaluation of the Data (Traditional
and Emerging Approaches)
Olvera, P. CASP 2013
197
Different Identification Approaches
(Traditional and Emerging Models)
Discrepancy
RTI
PSW
(Third
Method)
Academic
Skills
Tradition Model
IQ/ACH Discrepancy
Psychological Processing
(CA-Larry P)
Non-IQ based assessment
of basic psychological processes
Cross Battery Approach
Cognitive Hypothesis Testing (Hale)
Discrepancy- Consistency (Naglieri)
Looks at performance in cognitive abilities
to related academic areas.
Olvera, P. CASP 2013
198
Different Identification Approaches
Traditional Discrepancy:
•A specific learning disability is revealed through a
severe discrepancy between the pupil’s intellectual
ability and academic achievement. (Cal. Code Regs.,
tit. 5, § 3030, subd. (j)(4)(A).)
•Plus or minus 4 points (SEM).
Olvera, P. CASP 2013
199
Different Identification Approaches
Pattern of Strengths of Weaknesses (PSW):
“a pattern of strengths and weaknesses in performance, achievement, or both, relative to
intellectual development, that is determined by the team to be relevant to the identification of a
specific learning disability” ( Applegate, et al., 2010).
Criteria
Criteria #1:Does the student demonstrate a weakness (SS= <80*) in academic performance (CA
Standards below peers) and/or achievement relative to age or grade level standards?
Criteria #2: Does the student have a weakness (SS= 80*) in one or more of the basic psychological
processes (verified by more than one data source)?
Criteria #3: Is the identified basic psychological process related to the area of academic
underachievement?
Criteria #4: Are identified weaknesses (SS= <80*)present in an otherwise typical pattern (SS=
>90*) of functioning (i.e. the student also demonstrates strengths in some areas of achievement
and psychological processing)?
If yes to the above, then the student exhibits a pattern of strengths and weaknesses relevant to
the identification of a specific learning disability.
* General guidelines. District can decide.Olvera, P. CASP 2013
200
Different Identification Approaches
• CHC Operational Definition (Flanagan, Alfonso, & Mascolo,
2011).
Criteria
Criteria #1: Difficulties in one or more areas of academic achievement
( 7 IDEA Areas).
Criteria #2: Review of Exclusionary Factors.
Criteria #3: Disorder (SS= <80) in one or more of the basic psychological processes.
Criteria #4: Unexpected Underachievement- learning skill profile exhibiting significant
variability indicating processing areas of strength and weakness
Criteria #5: Specific learning disability has an adverse impact on educational
performance.
If yes to the above, then the student has a learning disability.
Olvera, P. CASP 2013
201
Different Identification Approaches
•
Larry P. Psychological Processing Approach (CDE, 1989, p. 66) ; Section 3030j (b) & (c)).
Criteria (the following factors must be discussed).
1. Patterns of current achievement are not commensurate with intellectual/cognitive
functioning.
2. Achievement levels are below the expected range of norms for the current classroom.
3. Rate of achievement has become more disparate from peers as increasingly complex tasks
are introduced
4. Documented have not been successful?
5. Teacher, parent, and pupil reports identify problem area and impact pupils achievement at
home and at school
6. Cognitive/Intellectual functioning is established as within the average range and
achievement levels are significantly below expectancy.
Olvera, P. CASP 2013
202
Case Studies
We will be reviewing various case studies.
Olvera, P. CASP 2013
203
Case Studies
• Five Pathways Have Been Discussed.
• Modified/Adapted/Accommodations
• Nonverbal/Language-Free/Language-Reduced
Assessment
• Native Language/Primary Language (L1)
• English Only Testing
• Flexible/Dynamic (Bilingual)
Olvera, P. CASP 2013
204
Examples
•Case #1: Jose Bilingual
•Case #2: Sally
•Case #3: Andy L1 & L2
•Case #4: Suzie “Nonverbal”
•Case #5: Andy L1 & L2 (see Handout #5)
Olvera, P. CASP 2013
205
In Closing
PRE-REFERRAL
• How are we teaching our ELLs in the General Educational Setting
(Tier 1):
 Are they receiving effective instruction?
 Has it been grounded in research and is it evidence-based?
• How about ELD instruction?
 Is it connected to the core?
 Does it build CALP?
Olvera, P. CASP 2013
206
In Closing
PRE-REFERRAL
• Intensive Interventions (Tier 2):
Is there a school-wide commitment to high expectations for all
ELLs?
Are they evidenced-based?
Are we progress monitoring?
Is it culturally and linguistically appropriate?
Is SPED the very last consideration after all other options have
been exhausted?
Olvera, P. CASP 2013
207
In Closing
Process
– Determining Language of Assessment
• Have you determined language of assessment?
• Have you used a standardized assessment (BVAT, CELDT, Etc.)?
• Have you determined BICS and CALP (L1 & L2).
• Are able to or do you have a competent assessor?
• Is your assessor trained?
• Do you document who and how the student was assessed?
– Does the student exhibits patterns which would be expected in ELL children
with learning difficulties?
• Academics
• Language?
• Classroom observation?
Olvera, P. CASP 2013
• Family history and background.
208
In Closing
Process
• Selecting Language of Assessment:
– MAMBI
– Clinical Judgment!
– Exclusionary Factors
Olvera, P. CASP 2013
209
In Closing
Systemic Perspectives
What we’ve Learned in our Journey Together
Pathways
•SPED Assessment (Tier 3):
• Assessment Methods
– Modified/Adapted/Accommodations
–
–
–
–
Language-Free/Language-Reduced Assessment
Native Language/Primary Language
English Only Testing
Combination of the Above
Olvera, P. CASP 2013
210
In Closing
Pathways
• Tools to help you analyze and interpret data:
– Cultural Language Test Classifications (C-LTC)
– Cultural Language Interpretative Matrix (C-LIM)
– Do you have a theory/approach that you all follow which will help with using a
common language?
Olvera, P. CASP 2013
211
In Closing
Systemic Perspectives:
What we’ve Learned in our Journey Together
Pathways
• SPED Assessment (Tier 3):
• Assessment tools
– Do you select tools that are low in culture and language demand?
– Do SPs and SLPs sit down together and jointly select and discuss tools?
– Is there a committee composed of SPs and SLPs that collaborate and work
together and make test purchase recommendations?
Olvera, P. CASP 2013
212
In Closing
Systemic Perspectives:
What we’ve Learned in our Journey Together
Evaluation of Case Data
• Evaluation of Assessment Data:
– How do you evaluate your data and make diagnostic decisions?
• Discrepancy?
• Pattern of Strengths and Weaknesses?
• CHC Operational Definitions?
• Alternative?
– Is there a district team that can organize and make recommendations as to how
determine eligibility? Should include SPs, SLPs, SPED, Admin, etc.
Olvera, P. CASP 2013
213
In Closing
Evaluation of Case Data
• Report Format:
– Is there a standard report that you use for ELLs?
– Is there a district group that can recommend a format?
– Should be made up of SPs, SLPs, SPED, etc.
– See Bilingual Evaluation Rubric
Olvera, P. CASP 2013
214
The End: Questions or Comments?
Polvera@apu.edu
Dr.pedro.olvera@gmail.com
Drs.olvera@neuroedclinic.com
http://www.neuroedclinic.com/
Olvera, P. CASP 2013
215