Academic Language Assessment
Emily Phillips-Galloway
Summer Institute
August, 2011
Team Leader: Paola Uccelli
Team Coordinator: Emily Phillips-Galloway
Team Members: Alejandra Meneses, Ph. D. (postdoc)
Emilio Sánchez, Ph. D. (visiting scholar)
Chris Barr, Ph. D. (psychometrician)
Christina Dobbs, Ed. D. candidate (RA)
Outline

Why is academic language important?

How do we define academic language?

The Academic Language Assessment (ALA)


Design and dimensions

Administration and Sample items

ALA & Common Core Standards

Relevance of results
Questions?
Intervention and
outcomes
Intervention
Outcomes
Teacher
Knowledge &
Capacity
Engaging
Topics &
Materials
Structured
Discussion
Purposeful
Reading/
Expository
Writing
Academ
ic
Langua
Complex
ge
Reasoning
Deep Reading
Comprehension
Perspective
Taking
CCDD Theory of Change, Adapted from Grant, Figure 2.
Why is academic language important?
90
THE GAP BETWEEN READING WORDS & COMPREHENDING TEXT
(LESAUX ET AL., 2010)
Grade 4
80
Percentile Rank
Grade 5
70
Grade 6
60
Grade 7
Grade 8
50
40
30
20
10
0
Word Reading
Fluency
Oral Language
Reading Comprehension
Lesaux, Crosson, Kieffer & Pierce, 2010
Academic Language or the language of school
•
required for academic success in challenging literacy tasks,
such as reading textbooks or writing school-valued genres
across content areas (Snow & Uccelli, 2009)
•
different from everyday language: many students who are
highly successful in communicating in informal contexts may
struggle at school (Halliday, 2004)
•
special challenge: learning language forms valued in school
is a challenge for all students, but it is especially challenging
for those with minimal exposure to and use of such language
outside of school (Schleppegrell, 2001, 2004)
AL: Definition
“knowing and being able to use general and content-specific
vocabulary, specialized or complex grammatical structures,
and multifarious language functions and discourse
structures—all for the purpose of acquiring new knowledge
and skills, interacting about a topic, or imparting information to
others” (Bailey, 2007)
AL across domains (Bailey & Butler, 2003
“language can be more or less
academic –that is furnished with
fewer or more of the traits that are
typical of academic language; we
have no basis for postulating a
separate category of language that
has passed some threshold
qualifying it as academic” (Snow &
Uccelli, 2009: 114-115)
Academic language: History textbook
(10th grade)
Although a war had begun, the American
colonists still debated their attachment to
Great Britain. A growing number, however,
favored independence. They heard the
persuasive arguments of colonial leaders
such as Patrick Henry, John Adams, and
Benjamin Franklin. These leaders used
Enlightenment ideas to justify independence.
The colonists had asked for the same
political rights as people in Britain, they said,
but the king had stubbornly refused.
Therefore, the colonists were justified in
rebelling against a tyrant[who had broken the
social contract.]
Autonomous text with
authoritative,
detached stance
Words that link ideas
explicitly
(connectives)
Words that link the
same participants
(reference chains)
Complex information
(complex words)
Densely packed
information
(complex syntax)
Example from Schleppegrell (2007), TNEELD
How will we assess academic language proficiency?:
The academic language assessment (ALA)

Purpose of the ALA
a. To understand the development of academic language during
the
middle school years
b. To explore academic language as predictor of deep
comprehension
- a crucial dimension of the intervention

Administration of the ALA
Group-administered test
Pre and post intervention (control & treatment group)
The Common Core Standards and ALA
Dimension
Text Type
Common Core Standards
Awareness of
AL
AL production
Definition
6th, 7th grade: Establish and maintain a formal style.
Definition
6th, 7th, 8th grade: Develop the topic with relevant facts, definitions, …
5th, 6th, 7th grade: Produce clear and coherent writing in which the development and
organization are appropriate to task, purpose, and audience.
8th grade: Demonstrate command of the conventions of standard English grammar
and usage when writing or speaking
Discourse
Structure
- global
- local
Linking ideas
- reference
- connectives
Dense
information
- words
- sentences
Argumentation
Exposition
Exposition
Sentences
Sentences
5th grade: Provide logically ordered reasons that are supported by facts and details.
6th, 7th grade: Introduce a topic; organize ideas, concepts, and information, using
strategies such as definition, classification, comparison/contrast, and cause/effect…
5th , 6th, 8th grade: Provide a concluding statement or section related to the information
or explanation presented.
5th grade: Link ideas within and across categories of information using words, phrases,
and clauses (e.g., in contrast, especially).
5th grade: Acquire and use accurately grade-appropriate general academic and domainspecific words and phrases, including those that signal contrast, addition, and other
logical relationships (e.g., however, although, nevertheless, similarly, moreover, in
addition).
6th grade: Use appropriate transitions to clarify the relationships among ideas and
concepts
7th, 8th grade: Use words, phrases, and clauses to create cohesion and clarify the
relationships among claims), reasons, and evidence.
4th, 5th, 6th: Choose words and phrases to convey ideas precisely.
5th, 6th, 7th: Use precise language and domain-specific vocabulary to inform
about or explain the topic.
8th grade: Acquire and use accurately grade-appropriate general academic and
domain-specific words and phrases…
Dimension 2
Knowledge of School Genres &
Discourse Structures
ability to draw on knowledge of how
texts (and text types) are constructed
both
1. globally (i.e., whole text) and
2. locally (i.e., paragraph-level)
Dimension 1
Academic Language
Register Awareness
1. knowledge of how formal
and informal language
registers differ and when
each is appropriate
2. ability to discern
author's/speaker's stance
(detached, authoritative)
Dimension 3
Knowledge of Words that Link Ideas
or Participants in Discourse
1. knowledge of how to connect
persons referenced in texts to
nouns or pronoun (e.g., she - Mary)
2. knowledge of general AL words
(e.g., therefore, however) that
connect ideas
AL skills which
support deep
comprehension
of texts
Dimension 4
Information Packing/
Unpacking Skills
ability to understand
information-dense texts by
employing knowledge of
1. complex words
(morphology) and
2. complex sentences
(syntax)
Assessing Dimension 1: Academic Language Register Awareness
What are we assessing? 1. knowledge of how formal and informal language registers
differ and when each is appropriate 2. ability to discern author's/speaker's stance
(detached, authoritative)
How will we assess this dimension?
Think: Which definition of
‘clown’ was written for
adults?
Assessing Dimension 1: Academic Language Register Awareness
What are we assessing? 1. knowledge of how formal and informal language registers
differ and when each is appropriate 2. ability to discern author's/speaker's stance
(detached, authoritative)
How will we assess this dimension?
X
X
X
X
Assessing Dimension 2: Knowledge of School Genres & Discourse Structures
What are we assessing? 1.
ability to draw on knowledge of
how texts (and text types) are
constructed globally (i.e., whole
text).
Another reason is that uniforms make students all
equal because no one has more expensive clothes.
In conclusion, wearing uniforms in school is better
because it solves many problems.
This task assesses
students’ knowledge
of how whole texts,
specifically
argumentative texts,
are constructed
(global discourse
structure awareness)
What ‘clues’ would you use to help you
to order this paragraph?
Finally, uniforms cost less, so families will spend less
money.
I think students should wear uniforms to school.
One reason is that with uniforms, students will pay less
attention to clothes and more attention to learning.
For example, a student at a school with uniforms said,
“everyone pays attention to the teacher. Nobody thinks about
clothes because we all look the same.”
Assessing Dimension 2: Knowledge of School Genres & Discourse Structures
What are we assessing? 1.
ability to draw on knowledge of
how texts (and text types) are
constructed globally (i.e., whole
text).
4
Another reason is that uniforms make students all
equal because no one has more expensive clothes.
6
In conclusion, wearing uniforms in school is better
because it solves many problems.
5
This task assesses
students’ knowledge
of how whole texts,
specifically
argumentative texts,
are constructed
(global discourse
structure awareness)
What ‘clues’ would you use to help you
to order this paragraph?
Finally, uniforms cost less, so families will spend less
money.
1
I think students should wear uniforms to school.
2
One reason is that with uniforms, students will pay less
attention to clothes and more attention to learning.
3
For example, a student at a school with uniforms said,
“everyone pays attention to the teacher. Nobody thinks about
clothes because we all look the same.”
Assessing Dimension 2: Knowledge of School Genres & Discourse Structures
What are we assessing? 1. ability
to draw on knowledge of how texts
(and text types) are constructed
locally (i.e., paragraph-level).
Different Towns: Springfield & Portcity
This task assesses
students’ knowledge
of how
argumentative and
expository texts are
constructed at the
paragraph level (local
discourse structure
awareness).
People in Springfield and Portcity live very
different lives. Springfield is a small town and most
people know each other. However, Portcity is a large
and very crowded city. Besides, in Springfield people
tend to have big houses with large gardens full of
flowers and trees.
_________________________________________________________________________________
_________________________________________________________________________________
Assessing Dimension 2: Knowledge of School Genres & Discourse Structures
What are we assessing? 1. ability
to draw on knowledge of how texts
(and text types) are constructed
locally (i.e., paragraph-level).
Different Towns: Springfield & Portcity
This task assesses
students’ knowledge
of how
argumentative and
expository texts are
constructed at the
paragraph level (local
discourse structure
awareness).
People in Springfield and Portcity live very
different lives. Springfield is a small town and most
people know each other. However, Portcity is a large
and very crowded city. Besides, in Springfield people
tend to have big houses with large gardens full of
flowers and trees.
_________________________________________________________________________________
In contrast, in Portcity people live in small homes or apartments
and rarely have a garden.
_________________________________________________________________________________
.
Assessing Dimension 3: Knowledge of Words that Link Participants or Ideas in
Discourse
What are we assessing?
1.knowledge
of how to connect persons referenced in texts to nouns or pronoun
(e.g., she – Mary)
This task assesses
students’ knowledge
of how to track
persons or ideas
mentioned in texts
across/ within
sentences (this is
called anaphora
resolution).
Assessing Dimension 3: Knowledge of Words that Link Participants or Ideas in
Discourse
What are we assessing?
1.knowledge
of how to connect persons referenced in texts to nouns or pronoun
(e.g., she – Mary)
This task assesses
students’ knowledge
of how to track
persons or ideas
mentioned in texts
across/ within
sentences (this is
called anaphora
resolution).
Assessing Dimension 3: Knowledge of Words that Link Participants or Ideas in
Discourse
What are we assessing?
2. knowledge of how general AL words (e.g., therefore, however ) connect ideas.
These tasks assesses
students’ knowledge
of words that connect
ideas in academic
texts (‘connectives’).
Assessing Dimension 4: Information Packing & Unpacking Skills
What are we assessing? 1. Ability to read/write complex words (morphology) and complex
sentences (syntax) which convey multiple ideas concisely.
1. activity. The children are very ___________________________________.
2. ethnicity. The city has many ________________________________ groups.
Choose the picture that
represents the
sentence: “The book
on the scarf is blue”
This task assesses
students’
understanding of the
construction of
complex words
structures that
support concise
writing and reading.
This task evaluates students’ abilities
to comprehend syntax.
(Kieffer’s Decomposition task; TROG-2 item)
Assessing Dimension 4: Information Packing & Unpacking Skills
What are we assessing? 1. Ability to read/write complex words (morphology) and complex
sentences (syntax) which convey multiple ideas concisely.
active
1. activity. The children are very ___________________________________.
ethnic
2. ethnicity. The city has many ________________________________
groups.
Choose the picture that
represents the
sentence: “The book
on the scarf is blue”
This task assesses
students’
understanding of the
construction of
complex words
structures that
support concise
writing and reading.
This task evaluates students’ abilities
to comprehend syntax.
(Kieffer’s Decomposition task; TROG-2 item)
A Task to Assess Multiple Dimensions Simultaniously: The Definition
Production Task
“Animal with silky fur on it black brown or white with four legs. You
can ride on it. It lives on a farm. It’s used to go places.”
–prototypical grade 4 student
“A four-legged farm animal that can be ridden for
transportation”
-prototypical grade 8 student
Are students’ scores on the AL Assessment relevant for school
performance? Yes….
Figure 1: Academic Language Results by grade and MCAS_ELA
1
0.9
0.8
AL Percent Correct
0.7
0.6
0.5
NI
W
0.4
Prof_Adv
0.3
0.2
0.1
0
4th
5th
6th
Grade
7th
8th
Your questions?
•
Bailey, A. & Butler, F. (2003). An Evidentiary Framework for Operationalizing Academic Language for Broad Application to K-12 Education: A Design Document CSE
Report 611. CRESST/University of California, Los Angeles.
•
Bailey, A. (2007). The language demands of the school: Putting academic English to the test. New Haven, CT: Yale University Press.
•
Halliday, M.A.K. (2003). The Language of Early Childhood, Vol. 4 in The Collected Works of M.A.K. Halliday. J. Webster (ed.). London: Continuum.
•
Lesaux, N.K., Crosson, A., Kieffer, M.J., & Pierce, M. (2010). Uneven profiles: Language Minority learner’s word reading, vocabulary and reading comprehension
skills. Journal of Applied Developmental Psychology, 31, 475-483.
•
Schleppegrell, M. (2001). Linguistic features of the language of schooling. Linguistics and Education, 14 (4), 431-459.
•
Snow, C. E., & Uccelli, P. (2009). The Challenge of Academic Language. Olson, D. R., & N. Torrance (Eds.). The Cambridge Handbook of Literacy, pp. 112-133.
Cambridge, UK: Cambridge University Press.
Appendix
AL Assessment Dimensions Mapped to Tasks
Dimension
Task
Text Type
Awareness of AL
Identification of AL
Definition
AL production
Writing AL
Definition
Discourse Structure
Global (anagram)
Argumentation
Local (text continuation)
Argumentation
Exposition
Linking ideas
Packing dense
information
Same participants
(reference chains)
Exposition
Explicit relations (connectives)
Sentences
Complex words (morphology)
Sentences
Complex sentences (syntax)
Sentences
A communicative model of Academic
Language
text
Representing
Message
Representation of self & audience
−
-Lexical pre
-Information
-Thematic de
Re
-Genre knowl
Me
Awareness of AL
• formal vs. informal
• stance (detached, authoritative)
fleS
Self
Self
Self
Commu
Socioc
Representation of message
−
•
•
•
−
−
−
School genres
Argumentation
Exposition
Definition
Discourse Structure
Local structure
Global structure
-L
-I
-T
-G
C
Organization of discourse
− Linking ideas
• Explicit relations (connectives)
• Same participants (reference)
− Packing dense information
• Complex words (morphology)
• Complex sentences (syntax)
− Lexical precision
(Snow & Uccelli, 2009)