AN INTRODUCTION TO TESTING
AND
ASSESSMENT
SAN DIEGO STATE UNIVERSITY
LANGUAGE ACQUISITION RESOURCE CENTER
June 30, 2011
Alan Svidal
Office of Language Acquisition,
San Diego Unified School District
1
An Introduction to Test and Assessment
AGENDA
• Gathering background information
• Using backward design
• Some important definitions
• Test and assessment protocols
• Writing tests with practice
• Writing assessments with practice
• Practical considerations about tests and assessments
• Taking a test: level 1 French
• Overview of scoring rubrics
• Writing rubrics f0r reading activity
2
Gathering background information
 Recall a time you took a test that was an unpleasant experience.
 Recall a time it was a pleasant experience.
 What made the difference between these experiences?
3
 When is it necessary to test students?
 When is it not necessary to test them?
 What do teaches hope to learn from tests?
 What do students learn from tests?
4
Backward or Reverse Lesson Design
identify target standards. (Which ones will guide the lesson development?)
First, I ___________________________________________
select functions and knowledge. (What will students need to do and know?)
Then, I ____________________________________________
develop the scoring rubric. (How I will measure student performance?)
Next, I ____________________________________________
develop the assessment. (How well can students use the language.?)
Then, I ____________________________________________
select activities that allow students to succeed. (What and how will they practice?)
Next, I _____________________________________________
teach. (What will I do to provide frequent, meaningful practice?)
Finally, I ___________________________________________
5
DEFINITIONS
Define the word
test
" ______________";
assessment
Define the word " ________________";
6
DEFINITIONS
test
Define the word " ______________";
Tests measure academic knowledge,
facts, formulas, processes, rules, etc.
Tests measure what a student should
know in comparison to other students at
the same grade level or in the same
course of study.
Tests are summative measures that reveal
mastery of language at specific points in
time.
7
Which statement about Napoléon II is NOT true:
A. He was the son of Napoléon Bonaparte.
B. He never ruled France.
C. He was the older brother of Napoléon III.
D. He was the king of Rome.
Fill in the blanks with the correct form of the verb “to swim”.
The form may only be used once.
1. I like __________.
2. Yesterday, I ________ for two hours.
3. By the end of the week, I will have _____ for 20 hours.
3. Is _________ you favorite activity?
Match the word in column A to its opposite in column B.
1.
2.
3.
4.
Original
Superfluous
Raucous
Salacious
a. necessary
b. quiet
c. redundant
d. praiseworthy
e. bodacious
8
DEFINITIONS
assessment
Define the word " ________________";
Assessments are ‘performance based’ evaluations
that examine behavioral outcomes and the means
students use to produce those outcomes.
Assessments do not measure a given body of
knowledge but rather how that knowledge is
applied to performance.
Assessments are formative measures that
highlight the functional use and on-going
development of language over time.
9
•You are planning a trip to Paris with friends.
•On a map of Paris, share your itinerary with the class.
• Point out what you will see,
• where the sites are located,
• how you will get there on the metro.
10
Fill in the blank with a single word that fits the context of this paragraph.
Martin lived near Paris with __1___ parents whom he loved very
____2__. They were kind ___3______loving but ____4___ poor. Martin loved
school, ____5___. Every ___6____ , when his mother ____7____ him up,
Martin was _____8____ to get out of ____9_____, put ___10___ his clothes,
eat his ___11_____ and __12___ hurry __13___ to school. This morning,
however, Martin looked____14___ the widow but did ___15___ smile. “What’s
the _____16____, Martin?”, his mother ____17___.“Well, ”Martin ____18___,
“it’s the __19__ day of school and I am very ____20____.”
Choose the word that best completes the sentence.
Martin lived near Paris with __1___ parents whom he loved
very ____2__. They were kind ___3______loving but ____4___ poor.
1.
2.
3.
4.
a. many
a. true
a. so
a. well
b. his
b. good
b. but
b. they
c. whose
c. much
c. and
c. however
d.
d.
d.
d.
one
restfully
however
terribly
11
12
What is a “rubric”?
My definition:
Our group’s definition:
What the experts say: “A rubric is…
(1) a scoring guide
(2) that lists key indicators of a performance
(3) in which numeric values are applied
(4) to descriptions of different achievement levels
(5) allowing scorers to differentiate among
performance outcomes.
13
“A rubric is…
(1) a scoring guide
(2) that lists significant indicators of a performance
(3) in which numeric values are applied
(4) to descriptions of different achievement levels
(5) allowing scorers to differentiate among performance outcomes.
1. Writing Rubric
Ann Jones
Score: 7 / 9
Outstanding
3
Satisfactory
2
Poor
1
Punctuation
Punctuation
almost always
correct
X
Some errors
throughout
Careless; numerous
errors
Effort
More than
required
Meets
requirements
Some items missing,
work appears hastily
assembled
X
Creativity
Creative,
original
descriptions;
realistic
characters;
well
illustrated;
neat
Some
creativity;
simple
descriptions;
mostly neat
X
Show no creativity or
planning;
incomplete
descriptions;
unrealistic
characters,
haphazard
illustrations or no
illustrations
14
AN OVERVIEW OF
TEST AND ASSESSMENT PROTOCOLS
15
Assessment Protocols: From Linear to Non-linear
We define “linear” assessments as those that produce predictable,
non-spontaneous responses.
Why are standardized tests and computer-based language programs
largely “linear”?
16
Assessment Protocols: From Linear to Non-linear
We define “non-linear” assessments as those that allow for
unpredictable, spontaneous responses.
• What are the advantages of “non-linear” assessments?
• What are the disadvantages of “non-linear” assessments?
17
1. As you examine the following assessment protocols, identify those that are:
Linear
Bridging to non-linear
Non-linear
2. When you have looked at all the assessment options:
• Write an example of a “linear” assessment question.
• Write an example of a “non-linear” assessment question.
18
TEST AND ASSESSMENT
PROTOCOLS
1.DISCRETE POINTS
Focus on language details and recall of
facts:





True/false
Matching
Multiple choice
Fill in blanks
Vocabulary/grammar drills
19
ASSESSMENT PROTOCOLS
2. INTEGRATIVE
Focus on student control of the language to
expand mastery through combining prior and
current learning via listening, reading,
speaking, writing, and viewing:
 Cloze
 Dictation
 Writing
 Interviews/polls
 Summaries
20
ASSESSMENT PROTOCOLS
3. PROMPTED
Focus on the use of manipulatives, visuals, and realia to
stimulate language production:





COCI (Classroom Oral Competency Interview)
CWCA (Classroom Writing CompetencyAssessment)
A/B pictures
Story boards/posters
Flash cards
21
ASSESSMENT PROTOCOLS
4. PERFORMANCE
Focus on demonstration, negotiation, interaction, and
coping with uncertainties:





Problem solving “games” or info gap act.
Group discussions
Classroom presentations
Reaching consensus
Role-playing
22
ASSESSMENT PROTOCOLS
5. CREATIVE
Focus on individual or group product,
reflection, and exposition:





Cultural Expeditions and presentations
Projects (radio, broadcasts, podcasts)
Skits
Puppet shows
Videos
23
ASSESSMENT PROTOCOLS
6. SUMMATIVE
Focus on language acquisition over time:




Flosem
Student Portfolios
Dairies/journals
Student books
24
ASSESSMENT PROTOCOLS
7. OPEN-ENDED
Focus on non-specified responses, varied
contexts and contents, broad range of
appropriate responses:





Computer based research
Student constructed tests
“Instruction” by students
Descriptions using pictures
Situation role play
25
10 Ways Language Learners
can
Demonstrate Learning
1.
Portfolios
2.
Scrapbooks
3.
Interviews
4.
Phone call to teacher
5.
Skits/charades
26
10 Ways Language Learners
can
Demonstrate Learning
6.
Dialogues/conversations
7.
Projects (videos, shows, books)
8.
Writing samples
9.
Storyboards
10.
Tests/quizzes
27
PROCEDURES FOR CONSTRUCTING A READING OR LISTENING TEST
1. Establish goal for summative testing.
• achievement test: to measure short-term mastery of knowledge
quizzes
• prochievement test: to measure mastery at the end of unit of study
unit test, mid-term exam, final exam
• proficiency assessment: to measure language mastery as well as skill
placement exam, end of course exam
2. Decide what type of test is most appropriate for the purpose.
• teacher-made test
• textbook test
• standardized test
• modified combination
3. Select reading or listening passage to be tested.
• authentic or simulated
• appropriate length
• appropriate level of difficulty
• relevant/interesting
4. Provide a context or setting, especially for listening.
• You are about to hear a weather forecast.
• You will read about all the things Dan, an American student, did
during the course of one day in Rome.
28
5. Develop items that test the most significant elements of the content
• be sure students are tested on what they have learned and practiced.
• keep the language of test questions at the students’ proficiency level.
• do not ask too many questions per reading or listening passage.
• write directions in the English.
• make sure the choice of answers is not ambiguous
• do not ask questions about minor details when testing listening.
6. As you write test questions, consider advantages and disadvantages of
test formats
• multiple choice
• true or false
• matching
• etc
29
Key Components for Oral and Written Assessment
Comprehensibility:
How well are students understood by others?
Comprehension
How well do students understand others?
Language Control:
How accurate is their language?
Vocabulary Use:
How extensive and appropriate is their vocabulary?
Communication Strategies:
How do they maintain communication?
Cultural Awareness:
How is their cultural understanding reflected in their
communication with others?
30
Nicolas and the Bicycle
Linear, bridging, and non-linear assessment
31
ASSESSING ABILTY TO SPEAK OR WRITE IN A SENTENCE
If you want to measure a students ability to speak in the dreaded
“COMPLETE SENTENCE”, try this technique that gives students a
logical and natural way to write or speak in complete sentences.
In natural speech, statements and answers are often single words or
phrases. Questions, however, are almost always framed as complete
sentences.
So, give students an ANSWER and ask them to say or write a logical
question.
HERE ARE A FEW ANSWERS. WRITE AN APPROPRIATE
QUESTION FOR EACH





“15 minutes.”
“No, I don’t think so.”
“Because I said so!”
“Friday, before school.”
“$35.00 plus tax.”
32
PRACTICAL CONSIDERATIONS ABOUT TESTS
AND ASSESSMENTS
•
Give advance notice.
•
Avoid surprise tests and quizzes
•
Plan ahead for possible conflicts.
•
Have an alternative test form ready.
•
Know what you will do about cheating and tell
the class.
•
Be aware of the amount of time to take the test
and to correct it.
•
Give clear directions, in English if necessary.
33
 Allow no distractions on the desks or in the room.
 Be sure the test will be ready on test day.
 Keep all the test materials in a secure place.
 If using equipment, be sure it works.
 If using rubrics, give them to students early on.
 Do not let standardized tests leave the room
34
Using the Framework: Stage I
Under each bullet, provide two ways in which your students might demonstrate that they can
fulfill the Stage I functions, one as a test, the other as an assessment.
• express likes and dislikes:
Paul loves sports. Circle the activities in this list that fit his interest.
Make a power point slide showing foods you would like to see served in the
school cafeteria and tell the class what these foods are.
• make requests:
Match the 5 common classroom requests in column A to appropriate pictures in
column B.
Write a note to your parents listing 3 reasons why you would like a bigger
allowance.
• obtain information:
Read the 5 sentences below about the time each student goes to bed and then
drawn in that time on each student’s clock.
Interview 10 students about how much time they spend watching TV and share
the results with your study group.
35
Using the Framework: Stage II
Under each bullet, provide two ways in which your students might demonstrate
they can fulfill Stage II functions, one as a test, the other as an assessment.
• express their needs:
Match Paul’s needs listed in column A with appropriate solutions in column B.
You are spending the winter break in Alaska. Make a list of things you will need to
pack and share this list with your study group.
• make requests:
Read the statements below. Then, fill in the blank with the appropriate question
word you would use to ask a question about the statement.
You will role play (as the customer) a phone conversation with your partner (as the
travel agent) in which you discuss hotels and sights to see in Paris.
• compare / contrast:
Look carefully at pictures A and B. They are similar but not identical. List five
differences you see.
Your cousin from Egypt is coming to stay with your family. Write a letter in which
36
you describe the differences he/she will encounter in San Diego.
UNIQUE 1
Vocabulaire
.
interrogation
quiz
37
UNIQUE
• u• ni• que
•u-ni-que
•UNIQUE
38
Unique?
• Voilà Notre Dame
• unique?
• Oui? Non?
Oui !
•
__
Notre Dame est unique.
39
Unique?
• Voilà la Tour Eiffel
• unique?
• Oui? Non?
Oui
•
___!
La Tour Eiffel est unique.
40
Unique?
• Voilà le drapeau de la France.
• unique?
• Oui? Non?
Oui !
•
____
•
Le drapeau de la France est unique.
41
Interrogation 1a
#A. Notre Dame
#B. le drapeau de la France
#C. la Tour Eiffel
1. # ________
2. # ______
la lettre correcte
3. # _______
42
Interrogation 1a
#A. Notre Dame
#B. le drapeau de la France
#C. la Tour Eiffel
B
1. # ______
C
2. # ______
A
3. # _______
43
Interrogation 1b
Complétez la phrase.
1.
La Tour Eiffel ______ unique.
2.
Notre Dame ______ unique.
3.
Le drapeau de la France ________ unique.
44
Interrogation 1b
Complétez la phrase.
1.
est
La Tour Eiffel ______ unique.
2.
Notre Dame ______ unique.
3.
Le drapeau de la France ________ unique.
est
est
45
Interrogation 1c
46
Interrogation 1c
Notre Dame est unique.
La Tour Eiffel est unique.
Le drapeau de la France est unique.
47
Meaning from context
Directions:
•Read ALL of the following entries from Madame Videau’s
planning agenda.
•You will be asked to respond to some questions after you read.
• You may make notes if necessary.
48
Les dates
Le calendrier de Madame Videau
27 Place Édith Cavell
H2Y 1H3 Montréal, Québec
le 5 mai : oncle Bernard arrive de Rome (à l’aéroport)
le 9 mai : la famille dîne au restaurant Le lion bleu.
le 10 juin : le 18ième anniversaire de Julie
le 26 juin : le mariage de Fabien et d’Alice, église de Ste.-Thérèse
le 1ier juillet: le Jour du Canada, célébration publique (Île Ste-Hélène)
le 19 juillet : oncle Bernard retourne en Italie.
le 30 juillet : la famille fait du camping aux États-Unis.
49
What do you know ?
(Answer as much in French as you can.)
1. List five facts you learned about Bernard ?
2. Name two things that Madame Videau has planned for the family.
3. What two things do you know about Julie?
4. On which date do you suppose there will be a fireworks display in
town? Why?
5. How are dates written differently in French than in English?
(You should be able to see two differences.)
6. Madame Videau’s birthday is May 25th. Write that date in French
and wish her happy birthday.
50
What do you know ?
(Answer as much in French as you can.)
1. List five facts you learned about Bernard ?
•
•
•
•
•
•
•
•
•
•
Bernard is an uncle.
He is Madame Videau’s uncle.
He lives in Italy.
He is Italian.
He speaks French.
He is arriving from Rome.
He is arriving on May 5th.
He is arriving at the airport.
He is returning to Italy.
He is returning on July 19th.
51
What do you know ?
(Answer as much in French as you can.)
What two things has Madame Videau planned for the family?
•
•
•
•
A dinner on May 9th at the restaurant Le lion bleu.
The wedding of Fabien and Alice.
To go to the Canada Day celebration.
A camping trip to the U.S. on July 30th.
52
What do you know ?
(Answer as much in French as you can.)
What do you know about Julie?
•
•
•
•
•
•
Julie’s birthday is June 10th.
Julie will be 18.
Julie is a Gemini.
Julie knows Madame Videau.
Julie is Canadian/ from Quebec.
Julie is a girl.
53
What do you know ?
(Answer as much in French as you can.)
When do you suppose there will be a fireworks display in town?
Why?
•
•
•
•
On July 1st.
It is Canada Day
It is a public celebration and a holiday.
It celebrates the confederation of Upper and Lower
Canada.
54
What do you know ?
(Answer as much in French as you can.)
How are dates written differently in French than in English?
(You should see two differences.)
•
•
•
•
•
•
•
•
In French, the date (numeral) comes in front of the month.
The names of the months are not capitalized.
The word “le” precedes the numeral (date).
In English, we add “-st” to the numeral “1”.
In French, we add “-ier” to the numeral “1”.
In English, we use cardinal numbers with dates.
In French, we use ordinal numbers with dates.
In English, we add “of” but not in French.
55
What do you know ?
(Answer as much in French as you can.)
Madame Videau’s birthday is May 25th. Write that date in
French and wish her “Happy Birthday”.
• le 25 mai.
• Bon anniversaire, Madame!
56
TAKING A LOOK AT
SCORING RUBRICS
57
“A rubric is…
(1) a scoring guide
(2) that lists significant indicators of a performance
(3) in which numeric values are applied
(4) to descriptions of different achievement levels
(5) allowing scorers to differentiate among performance outcomes.
1. Writing Rubric
Ann Jones
Score: 7 / 9
Outstanding
3
Satisfactory
2
Poor
1
Punctuation
Punctuation
almost always
correct
X
Some errors
throughout
Careless; numerous
errors
Effort
More than
required
Meets
requirements
Some items missing,
work appears hastily
assembled
X
Creativity
Creative,
original
descriptions;
realistic
characters;
well
illustrated;
neat
Some
creativity;
simple
descriptions;
mostly neat
X
Show no creativity or
planning;
incomplete
descriptions;
unrealistic
characters,
haphazard
illustrations or no
illustrations
58
Points to remember in Developing Rubrics
Select indicators that identify significant components of the
instructional activity:
 Fluency
 Pronunciation
 Vocabulary
Use numerical values to define different achievement levels within
each element:
 4 - 3 - 2 - 1 (Scales larger than 5 may be difficult to describe.)
Describe each achievement level in specific terms that recognize
potential differences among learners and are written in student-friendly
language:
 4 = speech is sustained: fillers maintain comprehension
 3 = speech is hesitant: fillers interrupt comprehension
 2 = speech is awkward: fillers interfere with comprehension
 1 = speech is broken: fillers prevent comprehension
59
Points to remember in Developing Rubrics
Summary words may be added to qualify numeric values:
 (4) Excellent / (3) Good/ (2) Acceptable/ (1) Unacceptable
 (4) Exemplary / (3) Proficient / (2) Basic / (1) Below basic
With preexisting rubrics, clarify each description by sharing some
examples of student work.
Be prepared to edit and revise rubrics that miss the mark you set.
AND MOST IMPORTANTLY:
GIVE STUDENTS THE RUBRIC BEFORE
THEY BEGIN THE WORK!
60
Critique of Existing Rubrics
Generic Rubrics for Collaborative Work
4
3
2
1
Workload
equality
Workload
shared
equally
Workload
somewhat
unequal
Workload
unequaldone mostly
by one or two
students
Workload
unequal-one
student has done
all the work
On task
All the time
Most of the
time
Sometimes
Little involvement;
rarely on task.
Interaction
Much
discussion;
shows
respect for
others
Some
discussion;
respectful of
others
Little
discussion;
easily
distracted,
somewhat
disrespectful
of others
Shows little
interest;
disrespectful of
others
61
Generic Rubric for Oral Presentations
Yes
No
Accurate
pronunciation
Accurate
Grammar
62
Generic Rubric for Oral Presentation
4
3
2
1
Pronunciation
Accurate
throughout,
near native
Understandable
with very few
errors
Some errors but
still
understandable
Poor
pronunciation,
very anglicized
Fluency
Smooth delivery
Fairly smooth
Unnatural
pauses
Halting,
hesitant, long
gaps
Comprehensibility
Easily
understood
Understood
Difficult to
understand
Incomprehensible
Vocabulary
Extensive use of
targeted
vocabulary
Some use of
targeted
vocabulary
Minimal use of
targeted
vocabulary
Fails to use
targeted
vocabulary
Credibility (shows
knowledge of other
culture)
Credible role
play; reflects the
culture
Credible role
play; somewhat
reflects the
culture
Limited
credibility; little
connection to
target culture
Not credible; no
connection to
target culture
visible
Performance
Lively,
enthusiastic;
good eye contact
General
enthusiasm;
some eye
contact
Low energy;
limited eye
contact
Reads from
cards,
monotonous; no
eye contact
63
Generic Rubrics for Written Material #1
4
3
2
1
Exceeds
Expectations
Grammar
Perfect
Uses well what
is being
studied
Some errors
with what is
being studied
Doesn’t seem
to understand
what is being
studied
Vocabulary
Creative use of
vocabulary
Vocabulary at
present level of
study
Some use of
current
vocabulary;
key words
missing
Minimal use of
targeted
vocabulary at
present level of
study; words
used
incorrectly
Spelling
Perfect
Very few errors
in spelling and
accent marks
Some errors in
spelling and
accent marks
Many errors in
spelling and
accent marks
64
La biographie de ma famille
5-6 points
o
o
o
o
o
o
la tache est complétée avec toutes les corrections
il y a des détails supplémentaire
le brouillon est dans le dossier
la composition est très bien organisée et présentée
tout est fini à l’heure
on a suit toutes les directions
2-4 points
o
o
o
o
o
il n’y a pas de détailles supplémentaires
le brouillon est dans le dossier
le projet est peut-être en retard
l’organisation et la présentation sont acceptables
on a suit toutes les directions
1 point
o
on a fait un projet mais pas acceptable parce qu’il ne s’adresse pas aux directions
0 points
o
pas de projet
5pts – extraordinaire
4pts – excellent
3pts – bon
2pts – passable
1pts – pas acceptable
0pts –pas acceptable
65
Foreign Language Department
Vocabulary
A=
Rich and extensive vocabulary; very accurate usage
B=
Occasionally lacks basic words; generally accurate usage
C=
Often lacks needed words; somewhat inaccurate usage
D=
Lacks basic words; inadequate usage
Verbs/Structure
A=
Utterances almost always correct, with some minor errors
B=
Many correct utterances, but with some structural errors
C=
Some utterances rendered correctly, many structural errors
D=
very few utterances structurally correct
Pronunciation/Comprehensibility
A=
Entirely comprehensible to native speaker; only an occasional word not comprehensible
B=
Some errors of pronunciation, but still mostly comprehensible
C=
Many errors about half incomprehensible
D=
Mostly incomprehensible; occasional phrases comprehensible
Fluency
A=
Speech natural and continuous; no unnatural pauses
B=
Generally natural and continuous; only slight stumbling or unnatural pauses
C=
Some definite stumbling and hesitation; sentences may be left uncompleted
D=
Speech halting and fragmentary; long, unnatural pauses
A= 4 points
B= 3 points
C= 2 points
D= 1 point
16 points are possible
66
Creating a Scoring Rubric
You will receive a level one benchmark assessment used
by students in San Diego Unified.
Create a rubric that would adequately evaluate student
performance on this benchmark.
Afterward, we will examine the district’s rubric
as well as a student scoring guide.
67
TECHNIQUES THAT BEST PROMOTE STUDENT
REPAIRS
Elicitation
Metalinguistic Feedback
Clarification request
Repetition
ALLOW YOUR STUDENTS TO SELF-REPAIR
If you allow time and provide students with the
proper cues, they will SELF-REPAIR
The least effective technique to correct a
mistake is to give the students the answer!
68
Metalinguistic Clues
Without providing the correct form,
teacher questions student (for
example, “Is it feminine?”, “Do we say
it like that?”, etc.
Elicitation
Questions that require more than a
yes/no answer.”What is the I form of
the verb?”
Repetition
Teacher repeats the error with high
intonation to draw student’s attention
to it.
69
Explicit Correction
Clearly pointing to the mistake and
correcting it.
Recast
Teacher repeats student’s utterance
but without the mistake.
Clarification Request
Using phrases like “Excuse me”, “I
don’t understand”, the teacher
indicates that the message was not
understood
70
WHEN CORRECTING PLEASE…
understand that they will probably
not internalize the correction.
remember that clarifying meaning is
not necessarily error correction.
remember that students what to
make a point rather than to make a
point correctly.
71