LEVEL 4 IAW STANAG 6001
- A CONCEPTUAL MODEL Jana Vasilj-Begovic
BILC Secretary/Canada
Gerard Seinhorst
Head, Language Office, Defence Institute
Security & Intelligence, Netherlands
SCOPE OF PRESENTATION
Background
Methodology
Conceptual Framework
Implications for Testing
Way ahead
Background
BILC conference 2010 – Istanbul
Formation of level 4 WG
Members: Bulgaria, Canada, Denmark, Germany,
Italy, Netherlands, Shape, Sweden,
USA
Methodology
• Survey and compile literature
• Develop a conceptual framework
• Find illustrative samples
• Discuss testing implications
Provide guidelines for test development
Conceptual Framework
Aims to answer the questions:
o
how to interpret the STANAG 6001 descriptors for
Level 4?
o
what are the typical and shared (linguistic)
characteristics of Level 4 language users?
o
what are the implications for assessing Level 4
language ability?
Purpose: to enhance the common understanding
of Level 4 proficiency.
Level 4 Characteristics
• Linguistically complex
• Low redundancy
• Multiple meanings in socio-cultural context
• Rarely encountered in daily life
• Unfamiliar cultural concepts
• Highly individualized / sophisticated
interpretation of the layers of explicit and implicit
meaning
Types of Comprehension
• Literal
• Inferential
• Evaluative
 Understanding meaning “beyond the lines”
 Evaluating the significance of the message
The Level 4 Language User
STANAG 6001: “Expert”
This level of proficiency is typically, although not
necessarily and not exclusively, achieved by
individuals who use the foreign language
extensively to handle intellectually highly
demanding issues as part of their profession or
specialisation
The Level 4 Language User
• Very taxing academic and professional settings
• Deal with demanding topics at the highest levels
• Examples of military-related tasks:
 Spokesperson responsible for press releases and press
conferences requiring nuanced, culturally appropriate
communications
• Arbiter between warring factions
• Providing official interpretation services
• Analyzing the hidden communicative intent of diplomatic
pronouncements
The Level 4 Language User
• Adapts and tailors language to suit the purpose and
situation
• Understands/expresses subtleties and shades of meaning
• Consistently maintain high degree of grammatical and
lexical accuracy
• Uses metaphors, clichés and proverbs
spontaneously
• May have fossilized lapses in pronunciation and intonation
Significant Differences (1)
LEVEL 3
LEVEL 4
• Understands/Can produce formal
and informal language, for most
everyday social and work-related
situations.
• Understands/Can produce highly
sophisticated language appropriate
for almost all topics, and social as
well as professional situations.
• Understands/Can produce
language relating to abstract
concepts and hypotheses.
• Readily adapts language to any
situation. Shows a firm grasp of
various levels of style and can
understand/express subtle
nuances and shades of meaning.
• Uses the language with great
precision for all social and
professional purposes, including
persuasion, negotiation, or the
representation of an official
point of view.
• Understands/Can produce
extended discourse and conveys
meaning correctly and
effectively. Understands
arguments/Can argue for and
against different options.
Significant Differences (2)
LEVEL 3
LEVEL 4
• Can understand/express meaning
“between the lines”.
• Can understand/express meaning
“beyond the lines”.
• Understands/Demonstrates
humour and irony, but may not
fully understand some allusions,
as well as implications of nuance
and idioms.
• Good command of idiomatic
expressions, figures of speech
and colloquialisms.
• Rarely requests repetition. Has a
natural flow, without searching
for words.
• Consistently maintains a high
degree of grammatical and
lexical accuracy. Uses the
language effortlessly.
• Occasional errors rarely disturb
the native listener/reader.
• Errors are rare and without
pattern.
Implications for Testing
Testing receptive skills (listening and reading)
Tasks
Text type
Sample
Testing techniques
Text Type (listening)
Adaptation of Clifford’s Text Characteristics
Speaker Purpose
Typical Text Type
Listener Purpose
PROJECT lines of
thought beyond the
expected; connect
previously unrelated
ideas or concepts;
present complex ideas
with nuanced precision
and virtuosity.
Extended discourse that
is tailored for the
message and the
intended audience. To
achieve the desired tone
and precision of
thought, the speaker
will often skilfully use
low-frequency
vocabulary, cultural and
historical concepts, and
demonstrate
understanding of the
audience’s shared
experience and values.
Listen “beyond the
lines”, understand the
speaker’s sociolinguistic
and cultural references,
follow innovative turns
of thought, and
interpret the message in
view of its wider
cultural, societal, and
political setting.
Text Type (listening)
Adaptation of Clifford’s Text Characteristics
Speaker Purpose
Typical Text Type
Listener Purpose
PROJECT lines of
thought beyond the
expected; connect
previously unrelated
ideas or concepts;
present complex ideas
with nuanced precision
and virtuosity.
Extended discourse that
is tailored for the
message and the
intended audience. To
achieve the desired tone
and precision of
thought, the speaker
will often skilfully use
low-frequency
vocabulary, cultural and
historical concepts, and
demonstrate
understanding of the
audience’s shared
experience and values.
Listen “beyond the
lines”, understand the
speaker’s sociolinguistic
and cultural references,
follow innovative turns
of thought, and
interpret the message in
view of its wider
cultural, societal, and
political setting.
Text Type (listening)
Non-participatory Situations
Speaker Purpose
Typical Text Type
Listener Purpose
Analyse issues,
present complex or
novel ideas or
arguments on
professional, political,
economic, cultural or
societal issues
(through public media,
overheard
conversations).
Lectures, broadcast
interviews and
discussions, briefings,
conference
presentations.
For example: an
interview contrasting
critical thinking with
emotional intelligence.
Understand
unpredictable shifts of
subject matter and
tone; understands
highly sophisticated
language and
references embedded
in a wider sociocultural context.
Text Type (listening)
Participatory Situations
Speaker Purpose
Elaborate on various
highly abstract topics
Typical Text Type
Negotiations
Debates
Professional
Counsel/persuade or meetings.
influence using
For example:
nuanced language
influencing others to
specifically tailored to change a policy
the listener(s).
position
Listener Purpose
Understand highly
complex discourse
and all styles of
speech (highly formal
and abstract or highly
informal/colloquial).
Respond and
contribute
satisfactorily, although
not always in the
projective mode.
Text Type (reading)
Clifford’s Overview of Text Characteristics
PROJECT lines of
thought beyond the
expected; connect
previously unrelated
ideas or concepts, or
present complex ideas
with nuanced precision
and virtuosity.
Extended discourse that
is tailored for the
message being sent and
the intended audience.
To achieve the desired
tone and precision of
thought, the author will
often demonstrate the
skilful use of lowfrequency vocabulary,
cultural and historical
concepts, and an
understanding of the
audience.
Read “beyond the lines”,
understand the author’s
sociolinguistic and
cultural references,
follow innovative turns
of thought, and
interpret the text in
view of its wider
cultural, societal, and
political setting.
SAMPLE LISTENING TEXT
Extracted from the Charlie Rose Show
Topic: Science and Health
http://www.charlierose.com/view/interview/12
276
SAMPLE READING TEXT
AN AMORAL MANIFESTO, Joel Marks (Philosophy
Now)
Hold onto your hats, folks. Although it is perhaps fitting
that the actual day on which I sit here at my computer
writing this column is April 1st, let me assure you that I do
not intend this as a joke. For the last couple of years I
have been reflecting on and experimenting with a new
ethics, and as a result I have thrown over my previous
commitment to Kantianism. In fact, I have given up
morality altogether! This has certainly come as a shock to
me (and also a disappointment, to put it mildly). I think the
time has come, therefore, to reveal it to the world, and in
particular to you, Dear Reader, who have patiently
considered my defences of a particular sort of moral
theory for the last ten years. In a word, this philosopher
has long been labouring under an unexamined
assumption, namely, that there is such a thing as right and
wrong. I now believe there isn’t.
Testing Techniques
Alignment of text and task level
Constructed response vs. multiple choice
What line of argumentation does the author present for
his relinquishment of morality?
One skill vs. combined skill testing
Rating decisions
Implications for Testing
Testing productive skills (speaking and
writing)
Tasks
Sample
Testing techniques
What L4 Speakers/Writers Can Do
• Tailor speech to a specific audience
• Counsel
• Persuade
• Advocate a position at length
• Elaborate on abstract concepts
• Express nuances and shades of meaning
Sample of Speaking Proficiency
• Task: discuss abstract topic
• Testing/elicitation technique (OPI): descriptive
prelude
• Level of task aligned with level tested
Sample of Writing Proficiency
• Task: analyse abstract issue (NATO related topic)
• Testing technique/writing prompt: descriptive
prelude
• Level of task aligned with level tested
SAMPLE WRITING PROMPT
One of the most significant trends in recent decades in the
arena of international military and political affairs has been
the perceived change in the scope and purpose of NATO
commitments and engagements. In the post-Soviet Era,
the mission of NATO has apparently shifted from
“defending the free world” to “saving the innocent from
ethnic cleansing or genocide”—whether at the hands of
their own rulers or of neighboring nations—through
intervention in an armed struggle on the side of a party
chosen on the grounds of moral righteousness and justice.
The policy of intervening to prevent genocide or ethnic
cleansing is seen as the moral equivalent of humanitarian
aid to save innocent human lives.
Sample of Writing Proficiency
In the year 2000, the UN published the Brahimi report
on peacekeeping and one of its probative findings
was that peacekeeping was significantly more
effective, when there was regional ownership of the
mission. There are several reasons for this, but one of
the major contributing factors is that few conflicts can
be adequately solved, unless there is a solid regional
backing for the peace process. Additionally,
peacekeeping soldiers from the same region and
culture are generally much more attuned to the
cultural particularities of any given conflict zone and
as such they may be better able to understand and
confront the underlying cleavages in a society torn by
war and atrocities.
Implications for Testing
• Finding appropriate texts
• Selecting appropriate testing techniques
• Determining proficiency/performance ratio
• Sampling widely to elicit a representative/ratable
sample of proficiency
• Reconciling requirements for wide sampling with
those of practicality
• Validating (target population)
• Training testers/raters
WG Status
• Paper finalized and disseminated (available on
BILC Website www.natobilc.org)
• Format and protocol for level 4 testing under
consideration
• Pilot project of level 4 testing protocol planned
QUESTIONS
THANK YOU!