Diagnosing and Assessing the
Needs of Heritage Learners
&
Implications for Instruction
Dr. Cynthia Martin
Agenda



Overview of the findings of a joint
NHLRC/ACTFL study
The ACTFL OPI – Assessment Criteria
Implications for assessment, instruction
and curricula
Limited language exposure
results in:




Incomplete grammar
Limited vocabulary
Limited pragmatic competence
BUT: near-native pronunciation and fluency are
in a high range
The NHLRC/ACTFL OPI Project


The goal is to gain an understanding of the
linguistic, exposure, and experiential factors that
contribute to HLLs speaking proficiency.
Languages: Russian and Spanish
Expected Outcomes




Describe the range of oral proficiency profiles of heritage
speakers by level and linguistic biographies.
Annotate descriptors within the ACTFL Proficiency
Guidelines – Speaking to encompass the range of
profiles that heritage speakers demonstrate.
Enhance OPI tester training to assure that testers are
accurately assessing the unique profiles of some
heritage speakers
Inform instructional practices that target the linguistic
strengths and weaknesses of heritage speakers
ACTFL Personnel





Project Director: Elvira Swender, Director, ACTFL
Professional Programs
Project Evaluator: Ray T. Clifford, Brigham Young
University
Russian Language Specialist: Cindy Martin,
University of Maryland
Spanish Language Specialist: Mildred Martinez
Rivera, ACTFL Consultant
Project Coordinator: Jeanmarie O'Leary, ACTFL
Heritage Criteria
 Learned
heritage language in an informal
setting
 home, community, etc.
 Uses heritage language with family,
friends, co-workers, etc.
 Lives in US
 Has received majority of formal education
in English-speaking institutions
 18 years or older
 Intermediate, Advanced, or Superior
speakers
Challenges
Identifying heritage speakers who met all
the criteria




Were willing to complete the background survey
Were willing to take an OPIc
 Especially at the Intermediate level
Sample sizes
 Spanish (41)
 Russian (50)
What is the ACTFL Rating Scale?
Hierarchy of global
tasks
 Four major levels
 Major levels divided
into sublevels

ACTFL Rating Scale
What are the major levels?
How are they defined?
SUPERIOR
ADVANCED
INTERMEDIATE
NOVICE
What are the major levels?
How are they
defined?
SUPERIOR
ADVANCED
INTERMEDIATE
NOVICE
Can communicate minimally with
formulaic and rote utterance, lists
and phrases
What are the major levels?
How are they defined?
SUPERIOR
ADVANCED
INTERMEDIATE
NOVICE
Can create with language, ask
and answer simple questions on
familiar topics, and handle a
simple situation or transaction
Can communicate minimally with
formulaic and rote utterance, lists
and phrases
What are the major levels?
How are they defined?
SUPERIOR
ADVANCED
INTERMEDIATE
NOVICE
Can narrate and describe in all
major time frames and handle a
situation with a complication
Can create with language, ask
and answer simple questions on
familiar topics, and handle a
simple situation or transaction
Can communicate minimally with
formulaic and rote utterance, lists
and phrases
What are the major levels?
How are they defined?
SUPERIOR
ADVANCED
INTERMEDIATE
NOVICE
Can support opinion,
hypothesize, discuss topics
concretely and abstractly, and
handle a linguistically
unfamiliar situation.
Can narrate and describe in all
major time frames and handle a
situation with a complication
Can create with language, ask
and answer simple questions on
familiar topics, and handle a
simple situation or transaction
Can communicate minimally with
formulaic and rote utterance, lists
and phrases
Assessment Criteria-Speaking
Proficiency
Level
Global Tasks and
Functions
Context/ Content
Accuracy/
Comprehensibility
Text
Type
Discuss topics extensively,
supports opinions and
hypothesize. Deal with a
linguistically unfamiliar
situation.
Most formal and informal
settings/ Wide range of general
interest topics and some special
fields of interest and expertise
No pattern of errors in basic
structures. Errors virtually never
interfere with communication or
distract the native speaker from
the message
Extended
discourse
Narrate and describe in major
time frames and deal effectively
with unanticipated complication.
Most informal and some formal
settings/ Topics of personal and
general interest
Understood without difficulty by
speakers unaccustomed to dealing
with non-native speakers
Paragraphs
Intermediate
Create with language, initiate,
maintain, and bring to a close
simple conversations by asking
and responding to simple
questions.
Some informal settings and
limited number of transactional
situations/ Predictable, familiar
topics related to daily
activities.
Understood, with some repetition,
by speakers accustomed to
dealing with non-native speakers.
Discrete
sentences
Novice
Communicate minimally with
formulaic and rote utterances,
list and phrases.
Most common informal
settings/ Most common aspects
of daily life.
May be difficult to understand,
even for speaker accustomed to
dealing with non-native speakers
Individual
words and
phrases
Superior
Advanced
The contexts expand as we
move up the scale.
the world
community
daily life
self
Superior
How do we define
accuracy/comprehensibility?


By the level of precision needed to convey the
message successfully
i.e. On a continuum of structural control
By the type of interlocutor who is able to
understand the speaker
i.e. On a continuum of required listener
empathy
Text Type

“Text” = “oral discourse organization”

What kind of text is required to
perform the function?
 Words and phrases
 Simple sentences
 Oral paragraphs
 Extended discourse
ACTFL Proficiency Level –
Intermediate
Speakers at the Intermediate level are distinguished
primarily by their ability to create with the language
when talking about familiar topics related to their daily
life. They are able to recombine learned material in
order to express personal meaning. Intermediate-level
speakers can ask simple questions and can handle a
straightforward survival situation.
They produce sentence-level language, ranging from
discrete sentences to strings of sentences, typically in
present time. Intermediate-level speakers are
understood by interlocutors who are accustomed to
dealing with non-native learners of the language.
ACTFL Proficiency Level –
Advanced
Speakers at the Advanced level engage in conversation in a
clearly participatory manner in order to communicate
information on autobiographical topics, as well as topics of
community, national, or international interest. The topics
are handled concretely by means of narration and
description in the major times frames of past, present, and
future. These speakers can also deal with a social situation
with an unexpected complication. The language of
Advanced-level speakers is abundant, the oral paragraph
being the measure of Advanced-level length and discourse.
Advanced-level speakers have sufficient control of basic
structures and generic vocabulary to be understood by
native speakers of the language, including those
unaccustomed to non-native speech.
ACTFL Proficiency Level –
Superior
Speakers at the Superior level are able to communicate with accuracy and
fluency in order to participate fully and effectively in conversations on a variety of
topics in formal and informal settings from both concrete and abstract
perspectives. They discuss their interests and special fields of competence,
explain complex matters in detail, and provide lengthy and coherent narrations, all
with ease, fluency, and accuracy. They present their opinions on a number of
issues of interest to them, such as social and political issues, and provide
structured argument to support these opinions. They are able to
construct and develop hypotheses to explore alternative possibilities.
When appropriate, these speakers use extended discourse without unnaturally
lengthy hesitation to make their point, even when engaged in abstract
elaborations. Such discourse, while coherent, may still be influenced by language
patterns other than those of the target language. Superior-level speakers employ
a variety of interactive and discourse strategies, such as turn-taking and
separating main ideas from supporting information through the use of syntactic,
lexical, and phonetic devices.
Speakers at the Superior level demonstrate no pattern of error in the use of
basic structures, although they may make sporadic errors, particularly in lowfrequency structures and in complex high-frequency structures. Such errors, if
they do occur, do not distract the native interlocutor or interfere with
communication.
Sublevels
The LOW sublevels =
a baseline performance for the level


sustained but
skeletal for the
level
“just hanging on”
The MID sublevels = solid
performance for the level


quantity and
quality for the
level
may have some
features of the
next level
The HIGH sublevels = performance
most of the time at the next major
level

functions
much of the
time at the
next higher
level

“fall” from the
next higher
level above
Evaluation Tools
 ACTFL
OPIc
 Internet-delivered version of the ACTFL Oral
Proficiency Interview (OPI)
 Fixed form
 All participants received the same prompts
 Intermediate through Superior-level tasks
 Same topics and role play situations
 Blindly double rated by certified ACTFL OPIc
raters
 Samples are digitally recorded and archived
 Participants received an official ACTFL OPIc
certificate and $25.00
Spanish Rater Site
Russian Rater Site
Evaluating the Sample

OPIc sample rated holistically

Performance of each task across the assessment
criteria for the level


Functions/global tasks, text type, accuracy
ACTFL OPI rating assigned
Evaluating the Sample
OPIc sample evaluated in terms of
performance at the next higher level

Sample evaluated in terms of specific factors
preventing speaker from being rated at the
next higher level

Functional breakdown
 Specific linguistic features that were inadequate for the criteria
of the next higher level

 Fluency (rate of delivery expected for the level), pronunciation,
vocabulary, accuracy (grammar and structure) pragmatic competence,
sociolinguistic competence, text organization
Intermediate Rater Review Form
Advanced Rater Review Form
Superior Rater Review Form
Spanish Data
Spanish Data
Country of Birth
Formally Studied Spanish in
Spanish Speaking Country
United States
3
Chile
1 111
1
15
1
90
80
70
60
50
40
30
20
10
0
Colombia
Dominican Republic
2
Ecuador
1
El Salvador
83
Mexico
Panama
Yes
Puerto Rico
No
Venezuela
Spain
Age of Immigration
Age
<2
1%
13%
2%
0%
84%
4%
18-25
26-35
36-45
46-55
56+
12%
4%
3%
1%
75%
1%
2-5
6-10
11-13
14-18
>18
Spanish Data
Language Used on the Job
Most Proficient Language
12%
0%
Spanish only
80
70
60
50
40
30
20
10
0
33%
English only
55%
A mix
No response/don't
work
Spanish
English
Equally
proficient in
both
languages
Language Spoken Most Often
Language Spoken at Home
5%
34%
47%
19%
Spanish only
English only
A mix
70%
Spanish
25%
English
A mix
Spanish Data
Self-Assessed Proficiency Level
ACTFL Proficiency Level
60
20
50
18
IL
16
40
IM
14
Intermediate
Advanced
30
Superior
Distinguished
20
IH
12
10
AL
8
6
AM
4
AH
2
S
10
0
IL
0
Intermediate
Advanced
Superior
Distinguished
IM
IH
AL
AM
AH
S
Spanish findings: self-assessment

General tendency was to over assess
proficiency level



Half of the Intermediate speakers selfassessed at Advanced
Half (52%) of the Advanced speakers selfassessed as Superior
Most Superior-level self-assessed correctly
(78%)

Only 11% self-assessed lower than Superior
Spanish findings: Intermediate level

When attempting tasks at the Advanced
level:




Most could initiate but not complete the tasks
Text type lacking in connectors and
organization
Lack of control over major time
Limited in ability to speak about topics beyond
the autobiographical
Spanish findings: Intermediate level

When attempting tasks at the Advanced
level:

Least successful was talking about current
event



Requires narration and description and the
vocabulary to move beyond the personal
More successful was dealing with a situation
with a complication
Most successful was past narration
Spanish findings: Advanced Low
and Mid

When attempting to discuss a topic from
an abstract perspective at the Superior
level:




86% do not deal with topic abstractly
55% initiate task but cannot complete
41% revert to examples of personal
experience
0% able to produce well organized extended
discourse
Spanish findings: Advanced Low
and Mid

When attempting a supported opinion at
the Superior level




50% do not address the task
27% resort to personal experience
64% lack precise vocabulary
91% unable to produce well organized
discourse
Spanish findings: Advanced Low
and Mid

When attempting to hypothesize at the
Superior level



0% were able to address the task
27% revert to describing a personal
experience
86% failed to produce well organized
extended discourse
Spanish findings: Advanced High

Primary reason for AH rating is functional, rather
than a structural breakdown


Limited ability to develop abstract ideas while
elaborating internally cohesive messages
When present, patterns of error are similar to
those made by L2 learners

Control over formulations that allow speakers to
speculate and elaborate on outcomes and
consequences (i.e. uses of subjunctive and other
complex grammatical structures)
Spanish findings: Advanced High

When attempting to discuss a topic from
an abstract perspective at the Superior
level




86% initiate response but cannot complete
57% revert to examples of personal
experience
72% lack extended discourse
14% lack precise vocabulary
Spanish findings: Advanced High

When attempting a supported opinion at
the Superior level




57% initiate but cannot complete the task
29% resort to personal experience
57% lack precise vocabulary
86% unable to produce well organized
discourse
Spanish findings: Advanced High

When attempting to hypothesize at the
Superior level:




0% were able to address the task
38% initiate but cannot complete
55% revert to describing a personal
experience
55% are unable to produce well organized
discourse
Russian Data
Russian Data
United States
Country of Birth
Russia
Studied Russian in a
Russian Speaking Country
Ukraine
Uzbekistan
1
4
3
1
1
3 2 1112 1
5 11
66
64
62
60
58
56
54
52
50
Kazakhstan
17
China
4
Moldova
15
54
Azerbaijan
United
Kingdom
Belarus
Yes
No
Latvia
Yes
No
Italy
Age
Age of Immigration
2%
10%
2%
<2
2-5
13%
73%
18-24
26-28
36-45
46-55
56+
15%
10%
22%
19%
20%
6-10
11-13
14-18
10%
4%
>18
Russian Data
Most Proficient Language
90
80
70
60
50
40
30
20
10
0
Languages Used on the Job
7, 7% 1%
Russian
Russian
English
29, 29%
64, 63%
Equally proficient
English
A mix
Russian
English
Equally
proficient
Language Spoken at Home
34%
16%
Languages Spoken Most Often
50%
English
A Mix
Russian
0%
Russian
47%
53%
English
A mix
Russian Data
Self-Assessed Proficiency Level
ACTFL Proficiency Rating
50
45
40
16
35
14
IL
IM
30
Intermediate
12
25
Advanced
10
20
Superior
8
15
Distinguished
6
10
5
IH
AL
AM
4
AH
2
S
0
0
Intermediate Advanced
Superior Distinguished
IL
IM
IH
AL
AM
AH
S
Russian findings: Self-assessment

General tendency was to over-assess proficiency
level





All of the Intermediate speakers self assessed at
Advanced
22% of the Advanced speakers self-assessed as
Superior
43% of Superior level self-assessed correctly
43% of Superior self-assessed lower at Advanced
14% of Superior self-assessed as Distinguished
Russian findings: Intermediate level

When asked to deal with Advanced-level
tasks:





Most do not address the task
Some initiate but are not able to complete
None maintain oral paragraph discourse
Most responses are marked by English
interference
Half are marked by lack of structural control
and lack of appropriate vocabulary
Russian findings: Intermediate level

When attempting tasks at Advanced level:

Talking about a current event is least
successful


Cohesive and organized text is the least controlled
feature
Most successful task is past narration

Russian past tense verbs are rather simple
Russian findings: Advanced Low
and Mid

When attempting to discuss a topic from an abstract
perspective at the Superior level


50% cannot deal with topic abstractly
88% initiate task but cannot complete




None demonstrated communicative strategies to address task at
Superior
83% revert to examples of personal experience
38% lack highly precise vocabulary
0% able to produce well-organized extended discourse


Most comfortable speaking in oral paragraphs
Vocabulary deficiencies alone do not keep these speakers from the
Superior level
Russian findings: Advanced Low
and Mid

When attempting a supported opinion at
the Superior level




94% do not address the task
73% resort to personal experience
77% lack precise vocabulary
88% unable to produce well-organized
discourse
Russian findings: Advanced Low
and Mid

When attempting to hypothesize at the Superior
level






0% address the task
38% initiate but cannot complete task
27% lack specific vocabulary
33% marked by English interference
55% revert to describing a personal experience
55% are unable to produce well-organized extended
discourse
Russian findings: Advanced High

When attempting to discuss a topic from an
abstract perspective at the Superior level





71%
51%
42%
57%
42%
initiate response but cannot complete
unable to deal with issue abstractly
revert to examples of personal experience
lack extended discourse
lack precise vocabulary
Russian findings: Advanced High

When attempting supported opinion at the
Superior level







86%
71%
57%
57%
71%
71%
71%
do not address the task
lack communication strategies
initiate but cannot complete the task
resort to personal experience
unable to deal with abstract
lack precise vocabulary
lack well-organized extended discourse
Russian findings: Advanced High

When attempting to hypothesize at the
Superior level:



85% refer primarily to American culture
42% do not address the task
42% lack communication strategies
Similarities across languages

For both language groups, talking about a
current event was the most challenging at
the Advanced level

These speakers tend to use language only in
familiar, informal environments that do not
move beyond the autobiographical
Similarities across languages

For both language groups, sustaining the
functions at the Superior level was the
most challenging

The ability to support opinion, deal abstractly,
and hypothesize in cohesive and internally
organized extended discourse
Similarities across languages

Proficiency levels increased with

More contact with heritage culture


Use of heritage language


In both cases, Advanced and Superior groups either lived in
a country where the heritage language is spoken or spent
significant time there
The higher the proficiency level, the greater the use of
heritage language or a mixture of heritage language and
English
Formal instruction in the heritage language at the
college level
Implications for instruction



Explicit and formal instruction in the language is critical
for heritage speakers to reach full professional
proficiency (Superior)
Strong connection between those who had formal
(college level) instruction in the language and those who
reached higher proficiency levels
Misperception that simply “speaking the language” at
home and with friends is sufficient for the workplace
likely linked to their tendency to over-assess their
abilities and conclude that formal instruction is not
necessary because they already speak the language
Implications for instruction

Instruction should focus on:

Awareness of what is defined as Superior level
language





Functions, contexts and content areas, the text type, and the
expectations for accuracy
Expansion of contexts and content areas beyond
personal and anecdotal
Expansion of the lexical base to include precise
(rather than generic) vocabulary
Producing coherent extended discourse that goes
beyond the single paragraph
Dealing with topics from abstract perspective (issues)
Implications for assessments

OPI remains a global functional assessment for
ALL speakers


Possible reporting of scores differently, special
feedback form
Purpose of assessment of heritage language
learners



Summative
Formative
Diagnostic
Implications for OPI testing

Heritage profiles differ from true L2 learner
profiles



Even at Intermediate, fluency and pronunciation may
sound native-like
Native-like pronunciation and lots of fluency do not
compensate for lack of sustained functional ability
Testers must keep the focus on the ability to address
and complete the functions
INSTITUTIONAL CONTEXT
AND SETTING REALISTIC
EXPECTATIONS
What is your institutional
context?



First and target languages
Student population
Program design and materials



Hours per day/week in-class, hw
Semesters/years
Class size
What can you reasonably expect as your
outcome?
Outcomes and Assessments

What Outcomes are reasonable given your
context?



Working across modes
What would you change to achieve more?
How will you design valid assessments for
measuring learner gain?
How long does it take to get
to…Advanced? Superior?

Many variables






First language; similarity to target language
Age and learning style
Type of Program
Aptitude
Motivation
…
There is no single study that answers this
question for us.
Case study: Foreign Service
Institute

Adult language learners






College graduates
Highly motivated
Previous language study
Class sizes of 6
Full-time language study for X weeks
Daily


5 hours in-class
4 independent learning (hw) daily
FSI data on required contact hours
(25 hours/week plus 3-4 hours of hw/day)
Language
Category I: Afrikaans, Danish, Dutch, French, Haitian
Creole Italian, Norwegian, Portuguese, Romanian,
Spanish, Swahili, Swedish, etc.
Class hours
to
Intermediat
e
Class hours to
Advanced
Class hours to
Superior
240
480
720
480
720
1320
480
1320
2400 - 2760
Category II: Bulgarian, Dari, Farsi, German; Greek,
Hindi, Indonesian, Malay, Urdu, etc.
Category III : Albanian, Amharic, Armenian, Azerbaijani,
Bengali, Bulgarian, Burmese, Czech, Estonian, Finnish,
Georgian, Hausa, Hebrew, Hungarian, Icelandic, Khmer,
Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali,
Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, SerboCroatian, Sinhala, Slovak, Slovenian,Tagalog, Tamil, Thai,
Turkish, Ukrainian, Uzbek,Vietnamese, Zulu, etc.
Category IV: Arabic, Chinese, Japanese, Korean, etc.
Instruction and Curricula for
Heritage Learners
Work across Modes





Interpersonal Speaking/Listening
Presentational Speaking
Presentational Writing
Interpretive Listening
Interpretive Reading
Interpersonal
Speaking/Listening



Likely their strongest area up to Advanced
functions
Need to practice discussing relevant issues in
the target culture
Maximize authentic listening/reading sources for
input


Builds vocabulary
Indications of those who read more improve their
speaking
Presentational Speaking
Presentational Writing




Not spontaneous, can be practiced, rehearsed
Maximize authentic listening/reading sources for
input
Learner-driven topics of interest
Can be “performed” at a level that is higher than
general proficiency
Interpretive Listening




Spontaneous; in real time
Formal and informal
Authentic sources for input from various
genres
Generally can work at a level above
productive skills
Interpretive Reading




Learner paced
Formal and informal
Authentic sources for input from various
genres
Generally can work at a level above
productive skills
Assessment Needs
 Diagnostic
 Formative
 Summative
Diagnostic





By skill or by mode?
Functions and Forms?
Linguistic features of productive skills
Self-assessment
Joint realistic goal setting (teacher and
learner) based on diagnostic assessments
Formative




Ongoing
Referenced to outcome goals
Portfolio to chart progress
Self-assessment reflections

Can-Do Statements
Summative



Referenced to outcome goals
Final Portfolio
Objective review or endorsement
Learner Autonomy

Assist learners in becoming reflective
about their language strengths and
weaknesses



Use of “discomfort” as a signal
Use of language journals
Use classroom/group time as efficiently as
possible to do what learners cannot do on
their own
Explicit Instruction for Heritage
Learners

Evidence that explicit instruction is
required in ALL languages (including one’s
native language) to reach professional
language use
Grammar Instruction:
Explicit vs. Implicit?


BOTH instead of “vs”
Think in terms of “functional grammar”




Why? (what communicative modes are
students working in?)
When? (when do students need those forms
as a result of “why”?)
What? (which forms are required?)
How? (how should you introduce/practice
those forms with heritage learners?)