About K–12 Assessments
ETS develops various English language assessments for K–12 students whose native
language is not English. These assessments are often used in conjunction with a state or
country’s large-scale program to provide students, schools, and the public with
information about students’ performance in English. ETS assessment specialists write
and develop test material that is aligned to curriculum standards or language-learning
objectives. Test material is developed for all grade levels—from elementary through
secondary school. Learning to identify levels of difficulty as well as the depth of
knowledge required to respond successfully to test items is a large part of K–12 test
development.
Summer interns will be working to develop material for listening and reading
comprehension assessments.
For listening assessments, test takers listen to a recorded selection, which may be a
monologue or a conversation, and then respond to either a single test question or a set of
questions. The questions may require test takers to recall specific information mentioned
during the selection or to make a prediction or draw an inference based on what is clearly
implied in the selection. Interns will be trained in how to develop selections and test
questions that are aligned with learning standards and aimed at the intended level of
difficulty. The challenge in crafting such items is in striking a balance between
maintaining the intended level of difficulty and ensuring that test questions assess only
the test taker’s listening ability.
For reading comprehension assessments, test takers read a passage and respond to a set of
questions that pertain to the passage. The questions require the test taker to identify the
main idea of the passage as well as both stated and implied information contained in the
passage. The challenge in developing reading comprehension assessments for English
language learners is to come up with multiple ways of presenting written information
without going beyond a test taker’s vocabulary knowledge.
Below, you will find information on submitting a work sample.
Format for submitting your work sample
1. Complete the work sample by using the spaces provided on pages 3, 4, 8, and 9.
2. Delete all of the instructional material on pages 1–2 and 5–7. Your document should
consist of 4 pages that contain only the writing tasks.
3. Rename the document by using the convention
K-12_your last name_your first name.doc.
For example, K-12_Listening_Doe_Jane.doc.
K–12 Assessments Work Sample/p. 2
General Information
In the following pages, you will be asked to provide a work sample made up of several
tasks for writing test questions (called items). Each item-writing task is preceded by
specific directions.
Item-Writing Terminology
Stem
The part of an item that describes the task
The stem may be a question, an incomplete statement, or a complete
statement that requires interpretation.
Key
The correct answer
Options
All of the choices in a multiple-choice item
Most K–12 test items have four options.
Distracters
The incorrect options (answers) in an item
They should be attractive; that is, they should have some appeal to the test
taker.
Stimulus
A spoken or written text, such as a reading passage or a short
conversation, that sets up an item and options
Illustration
Stem
Because the equipment is very delicate, it must be handled with --------.
Key
Distracters
(A)
(B)
(C)
(D)
care
caring
careful
carefully
Options
General guidelines for writing good items
• Avoid using inflammatory, controversial, or unpleasant topics. Test material should not
have a suggestive subtext. Language should be free of racist, sexist, or otherwise
potentially upsetting content.
• There should be only one key for each item. Check that distracters cannot reasonably be
interpreted as correct answers. Distracters should be grammatically correct and plausible
within certain contexts, but they should not be true for the existing stimulus.
K–12 Assessments Work Sample/p. 3
Item-Writing Tasks: Three Parts
PART I—Reading a Report
Directions: Complete the article below, adding between 250 and 300 words. Then
follow the directions for writing three questions about the article. Instructions for writing
each question appear after the article.
New Designs Mark Textile Firm’s Twenty-fifth Anniversary
HUNTSVILLE (June 19)—Franort Textiles, one of the area’s largest employers, unveiled new
collections of upholsteries and draperies yesterday during a reception held to mark the company’s twentyfifth anniversary. “These two collections—Oceans Alive and Earthbound—capture the essence of Franort:
sophisticated designs that offer high performance and low environmental impact,” said Franort’s general
manger, Selma Ortega. “The new ocean and forest themes fit right in with the designs for our existing
collections.”
Question #1: Write a key (correct answer) for the following stem:
What is this article mainly about?
Question #2: Write a stem (a question) about an explicitly stated detail in the article.
Question #3: Write a key (correct answer) for the following stem:
What can be inferred about the themes for Franort’s existing designs?
(An inference question requires test takers to identify information that is implied but not
explicitly stated in the article.)
K–12 Assessments Work Sample/p. 4
PART II—Reading a Narrative
Directions: Create TWO stems and keys (but not distracters) for the following passage.
Note the general level of vocabulary used in the passage.
Joe came home from school one day very excited. He and his friend Mike were going
to finish building a model airplane. He went into his room to get the plane, but the plane
was not there. That was strange. He couldn’t find it in his room, and his mother said she
had not seen it anywhere else.
Then he remembered where it was. He had left it at the park. He and Mike had
stopped working on it for a while to play on the field. He had not had picked it up when
they went home.
Joe was afraid that somebody took it. The model airplane was a present from his
grandfather, and he didn’t want to lose it.
Joe ran to the park, but he didn’t find the airplane. He was even more worried now.
Then, he ran to Mike’s house. When he arrived at the house, Joe saw Mike’s brother.
Mike’s brother gave the model airplane to Joe.
“You forgot this,” Mike’s brother said. “Mike brought it home because he didn’t want
the other children to take it!”
K–12 Assessments Work Sample/p. 5
PART III—Listening to Dialogues
 A dialogue item should test common speech functions such as:
o
o
o
o
o
o
o
explaining something
asking for or providing information
accepting or declining an invitation
requesting (or offering) something
suggesting a course of action
giving and taking advice
describing someone or something
 The dialogue stimulus should be a conversation between two speakers and it should
have no more than 125 words and no more than six exchanges, meaning that each
speaker should speak no more than three times in a conversation.
 The questions can be divided into three major categories:
o Paraphrasing: understanding what is said in other words
 What does the girl mean?
 What does the boy say about the class?
 What does the teacher offer to do?
 What does the woman think the girl should do?
o Summarizing: understanding the entirety of what is said
 What are the speakers talking about?
 What is the purpose of the announcement?
 What point is the speaker making?
o Making an inference: understanding what is not explicitly stated
 What will the boy probably do next?
 What does the woman suggest the girl do?
 What is probably true about the man?
K–12 Assessments Work Sample/p. 6
 The question following a dialogue should always refer to what is spoken by the last
speaker. Consider the sample below.
(Narrator):
Listen to a conversation between a boy and his cousin.
(Boy):
Can I borrow your calculator to do my homework?
Mine’s not working.
(Girl):
SorryI need mine to do my homework.
(Narrator):
What does the girl mean?
The question “What does the girl mean?” refers to what the final speakerthe
girlsays at the end of the dialogue.
Note that the intended answer (“She can’t lend the boy her calculator.”) can be
predicted without seeing the options. In a well-crafted item, the listener should not
have to read the options to determine the correct answer.
Note also that answering this question requires the listener to understand both
exchanges in the dialogue. That is, if you only comprehended the second line
(“SorryI need mine to do my homework.”) you could not answer the question
because you would not know what “mine” refers to (the calculator, in this case).
Consider the case below:
(Boy):
Can I borrow your calculator to do my homework? Mine’s
not working.
(Girl):
Yes, you may borrow my calculator.
(Narrator):
What does the girl mean? (Answer: “The boy may borrow
her calculator.”)
In this case, the listener does not need to comprehend the entire exchange, only
the girl’s line, to answer the question correctly. While this doesn’t necessarily
make the item bad, it does make it much easier.
K–12 Assessments Work Sample/p. 7
 To create the options that are not correct (the distracters), use information that is
presented in the dialogue to write options that answer the question but are not true
based on what is spoken in the dialogue. Consider the dialogue below.
(Narrator):
Listen to a conversation between a boy and his aunt.
(Boy):
Do you know anyone who is looking for a stereo? I’m
getting a new one, and I want to sell my old one.
(Woman):
I don’t know of anyone offhand. But if I were you, I would
put an advertisement in the newspaper.
(Narrator):
What does the boy’s aunt think the boy should do?
The intended answer is “advertise his stereo for sale in the newspaper.”
Using information presented in the dialogue, wrong answers could be:
 give her his old stereo
 read an article about stereos in the newspaper
 buy a stereo that is advertised in the newspaper
All of the options contain language that is found in the dialogue, and all provide a
grammatical answer to the question “What does the boy’s aunt think the boy
should do?”but none answer the question based on what is actually said in the
dialogue.
K–12 Assessments Work Sample/p. 8
Directions (two exercises):
1. Write a key and three distracters for the question (stem) that follows the dialogue
below. The underlined words in the script are words that will receive emphasis when
spoken in a recording.
(Narrator): Listen to part of a TV interview with an author.
(Man): With me in the studio is Mary Contini, whose latest story for children, “The
Ring and the Wind,” was just released last week. Ms. Contini, is it true that, for the
first time ever, someone persuaded you to rewrite part of a story?
(Woman): Yes, that’s right. I usually don’t change a word, even at the request of my
publisher. But this time I made an exception. The original ending of the story was
quite sad. My daughter didn’t like it, and she insisted I do something about it. She’s a
pretty sensible girl, so I thought she might have a point. I listened to her suggestion
for a new ending, and I decided that she was right!
Stem: What does the woman explain?
Key:
Distracter #1:
Distracter #2:
Distracter #3:
K–12 Assessments Work Sample/p. 9
2. Write a dialogue followed by one question (containing a stem, a key, and three
distracters) that is appropriate for students ages 9 to 12.
(Hint: After writing your dialogue and question, read it aloud to a friend. A proficient
English speaker should be able to give you the right answer without reading or hearing
the options.)