Prep for Training
• Check to see if questions on screen are easy to read
• Check Internet connection
• Hook up laptop speakers
• Cue videos
Creating Text-Dependent
Constructed Response Questions
~ Training of Trainers ~
Penny Plavala, School Improvement Specialist
Multnomah ESD
Please access:
http://guest.portaportal.com/pplavala
Training of Trainer Goals
Provide training resources to support two one-hour
sessions:
1. Creating Text-Dependent CR Questions
2. Assessing Text-Dependent CR Questions
 Resources are available on the PortaPortal:
http://guest.portaportal.com/pplavala
Note
These are ideas I have used in trainings for K-5 teachers.
Use / modify activities. Create new strategies.
 Training of Trainers:
Time to stop along the way and process how
you might use a piece with your staff.
Let’s learn together!
If you think of a way to modify an activity,
please share your idea.
Please Note
Penny’s PortaPortal
• The Power Point will be removed today after our session.
• The Power Point should not be shared outside your
district. It is for your use as a trainer. Please keep my
name as creator.
• All of the other resources & links from this training will
remain on the PortaPortal, but may move down as
other events are showcased.
Session One Goals
• Refresher: Smarter Balanced Assessment Components
• Focus on Constructed Response Questions
• Create Questions for a Classroom Formative Assessment
 Instructional design:
direct instruction, work independently,
with a partner, and in small groups
Please locate a partner
Warm-Up
Tell your partner what you know –
or have heard about Constructed
Response or Text Dependent
questions.
Why Are We Here?
This spring, students will answer Constructed Response
questions on the Smarter Balanced Assessment.
This is a new item type for Oregon students.
By integrating CR questions into our classroom
assessments, students will be familiar with this type of
item.
We will focus today on Text-Dependent
Constructed Response questions.
The SBAC Assessment System
English Language Arts and Mathematics, Grades 3 – 8 and High School
11th Grade: Last 6 weeks
Last 12 weeks of year*
DIGITAL CLEARINGHOUSE of formative tools, processes and exemplars; released items and tasks;
model curriculum units; educator training; professional development tools and resources; an
interactive reporting system; scorer training modules; and teacher collaboration tools.
INTERIM ASSESSMENT
Computer Adaptive
Assessment and
Performance Tasks
INTERIM ASSESSMENT
Computer Adaptive
Assessment and
Performance Tasks
PERFORMANCE
TASKS
• Reading
• Writing
• Math
COMPUTER
ADAPTIVE
ASSESSMENT
CR Questions on Both Parts of Test
Optional Interim
assessment system —
no stakes
Summative assessment
for accountability
What Will Be Tested?
Claim 1
Reading
Claim 2
Writing
Claim 3
Speaking & Listening
Claim 4
Research
• Students can read closely and analytically to
comprehend a range of increasingly complex
literary and informational texts.
Place text
here
• Students can produce effective and well-grounded
writing for a range of purposes and audiences.
• Students can employ effective speaking and
listening skills for a range of purposes and
audiences.
• Students can engage in research/inquiry to
investigate topics, and to analyze, integrate, and
present information.
Assessment Item Types
Selected Response (SR)
Variety of multiple choice
Technology Enhanced (TE)
Technology embedded into
items
Constructed Response (CR)
Short answer using textual
evidence
Performance Tasks (PT)
Use higher level thinking skills;
integrate reading, writing, and
listening
Our Focus Today
•Require the student to generate a response as
opposed to selecting a response.
•Include both short and extended responses.
•Allow students to demonstrate their use of
complex thinking skills consistent with the
expectations for college and career readiness.
The following items are from the SBAC Practice Test 2013.
Components of a Constructed Response Item
The Shepherd’s Boy and the Wolf
STIMULUS
A Shepherd's Boy was tending his flock near a village, and thought it would be
great fun to trick the villagers by pretending that a Wolf was attacking the sheep:
so he shouted out, "Wolf! Wolf!" and when the people came running up he
laughed at them because they believed him. He did this more than once, and
every time the villagers found they had been tricked, for there was no Wolf at all.
At last a Wolf really did come, and the Boy cried, "Wolf! Wolf!" as loud as he could:
but the people were so used to hearing him call that they took no notice of his
cries for help. And so no one came to help the boy, and the Wolf attacked the
(Reading Passage)
In
few
sentences,
explain
what
lesson
the
reader
In aafew
sentences,
explain what
lesson the
reader
can learn
from
the shepherd’s
boy. Use
details
from the
to support your
response.
can
learn
from
thestory
shepherd’s
boy.
Use details from
the story to support your response.
STEM
SPACE FOR
ANSWER
Constructed Response
Read the sentence and the directions that follow.
The hermit crabs in the ocean have learned to adapt to the changing
housing situation.
Using details from the text, define the word adapt and explain how the
crabs have adapted.
Type your answer in the space provided.
5th grade
Constructed Response
Explain why the author is interested in dragonflies.
Use details from the passage to support your explanation.
Type your answer in the space provided.
3rd grade
Constructed Response
Explain what archeologists think about how the Grand Canyon
was discovered.
Use at least two details from the passage to support your answer.
Type your answer in the space provided.
4th grade
Constructed Response
Describe two ways that sleeping on the International Space
Station can be difficult for astronauts.
Use details from the presentation to support your answer.
Type your answer in the space provided.
5th grade
Scoring a Constructed Response Item
2-point Use Evidence Rubric
2
The response gives sufficient evidence of the ability to cite
evidence to support arguments and/or ideas.
1
The response gives limited evidence of the ability to cite
evidence to support arguments and/or ideas.
0
The response gets no credit if it provides no evidence of the
ability to cite evidence to support arguments and/or ideas.
Conventions are not
scored on Constructed
Response questions
Smarter Balanced Assessment Consortium
Turn and Talk
• What skills do students need to answer Constructed
Response questions?
• When do students complete short answer questions
in your classroom?
Trainer Check-In
• Considering the prior knowledge of your staff…
• What elements of the SBAC and CR overview might
you use in the training introduction?
• What may you revise?
After students read a passage in your
classroom, how do you check for
understanding?
• Answer questions during whole-class discussion
• Quick write – summarize key ideas (exit slip)
• Thumbs up, sideways, down
• Answer text-dependent constructed response
questions
Text-Dependent
Constructed Response Questions
• Definition
• Examples
CR = Constructed
Response
• Components
• Practice creating questions for your classroom
Text-Dependent CR Questions:
• are open ended, short answer questions that measure
application-level skills as well as content knowledge.
• can assess higher level thinking:
- comparisons, contrasts, causes, effects, changes
- identify patterns or conflicting points of view
- categorize or summarize information
- state a generalization, conclusion, explanation or
prediction……using evidence from the text.
Text-Dependent CR Questions:
• use a range of primary and secondary stimuli and
authentic "real world" examples including short
reading passages, time lines, maps, graphs, cartoons,
charts.
• are graded against specific criterion using a rubric.
Text-Dependent CR Question
Example:
“Some animals would not survive if they were unable
to use tools.” What two pieces of evidence from the
article would you use to support this statement?
TextBased
Answers
TextDependent
Questions
Teachers write questions that keep students in
the text. Students shouldn’t have to go outside
the text for answers.
“Stay within four corners of text”.
Quality Text-Dependent Questions
Can be answered by careful close reading
Do not depend on outside sources
Require more than recalling facts
Often require students to infer
Require time for students to process
Constructed Response Questions:
Please take a moment with your partner to look over
the examples from the Smarter Balanced Assessment
Consortium.
These questions were taken from the Grade 3-5
Practice Performance Tasks: 2011- 2013.
What do you notice?
• Look closely at the verbs. Circle them!
• Locate verbs on Bloom’s list
support
use details
describe
select
list
analyze
Idea:
Type on
screen the
words as
teachers
report out
Where do we see these
verbs on Bloom’s list?
Training Option
What are Text Dependent Questions (2 min.)
https://www.youtube.com/watch?v=HzRls2cc8lc
Outstanding New Resource
Text-Dependent Questions: Pathways to
Close and Critical Reading, Grades K-5
• Creating Effective Close Reading Lessons
- What does the text say?
- How does the text work?
- What does the text mean?
• Sample passages with questions
• Classroom videos
Types of Text-Dependent Questions
Examples
• Follow along on pink sheet
• Examples increase in difficulty
• On screen: Kindergarten examples
Note: All questions would include a second sentence
asking students to support their answer with evidence
from the text.
Build Understanding of the Meaning
• Sequence of information
• Determine importance of ideas
• Find details that support main
ideas
• Answers who, what, when,
where, why, how much, or how
many.
• Retell the story in order using the words
beginning, middle, and end.
• How long did it take to go from a hatched egg
to a butterfly?
• What is one food that gave him a stomach
ache? What is one food that did not him a
stomach ache?
Vocabulary and Text Structure
• Bridges literal and inferential
meanings
• Denotation
• Connotation
• Shades of meaning
• Figurative language
• How organization contributes
to meaning
• How does the author help us to understand
what cocoon means?
Author’s Purpose
• Genre: Entertain? Explain? Inform? Persuade?
• Who tells the story?
• Point of view: First-person, third-person
limited, omniscient, unreliable narrator
• Who tells the story—the narrator or the
caterpillar? How do you know?
Inferences
• Use details and examples in the text when
drawing inferences.
• Details in text + what you know = inference
• The title of the book is The Very Hungry
Caterpillar. How do we know he is
hungry?
Opinions and Intertextual Connections
• State opinions and support with evidence
from the text
• Make connections to other texts
Narrative
Informational
•In your opinion, is this a
happy story or a sad one?
How do you know?
•In your opinion, how are
these two books similar?
How are they different?
Are These Text-Dependent
CR Questions?
• Look up at screen to view the sample questions.
• Read the question.
• Talk to partner to determine if this is an example of a
text-dependent question.
• Discuss why or why not?
From poem “Paul Revere’s Ride” by Henry W. Longfellow
• How does the author express a rising call for action in
the final four lines of the poem?
• How does the author portray Revere as a protector of
his community? Cite examples from the text.
• Do you think it is fair for people to protest against the
government?
•
Many historians argue the accuracy of this poem. Do
you think students should be expected to read this
poem if it contains inaccurate details?
From other texts:
• In “Letter from Birmingham Jail”, Dr. King discusses
nonviolent protest. Discuss a time you wanted to fight
against something that you felt was unfair.
• After reading Dr. King’s letter, what can you infer about
the content of the letter he received? Use details from
the text to support your answer.
• In the poem “Casey at the Bat”, how does the author
make Casey’s experiences realistic in stanza four?
•
In the poem, Casey strikes out. Describe a time you
failed at something.
From: Alice’s Adventures in Wonderland
• In the story, what kind of books does Alice find useful?
• How would you react if you saw a talking rabbit?
• Do you think books without pictures are useful?
Why or why not?
•
What details about the rabbit catch Alice’s eye in the
third paragraph?
•
How did Alice react when she saw the talking rabbit?
Trainer Check-In
• Look back at the last few slides reviewing:
– Levels of questions
– Sample questions (from SBAC and on screen)
• What elements might you use in a teacher training?
• What might you revise?
How do we create
Text-Dependent
Constructed Response questions?
Stop!
The next slides and activities are for this training of
trainers only.
We will get back to teacher training in a minute.
Visual Options
Powtoon (3 min.) Writing Text-Dependent Questions
• https://www.youtube.com/watch?v=ixDBlejFOLE
Powtoon (2:26 min.) Creating Text-Dependent Questions:
Five Easy Steps
• https://www.youtube.com/watch?v=dzzVRxwu1g8
Practice Activity
• Engage NY Project:
- reading passages
- sample questions
- scored student work
• 3rd grade passage:
- Excerpt from: David and the Phoenix
- Word Count 572
- Lexile 870
SBAC Word Counts and Lexile Targets
Grade
Word Count Range
(short text)
Word Count Range
(long text)
3
200 – 487
488 – 650
4
450 – 562
563 – 750
5
450 – 562
563 – 750
6
650 – 712
713 – 950
7
650 – 712
713 – 950
8
650 – 712
713 – 950
11
800 – 825
826 – 1100
Grade
CCSS Lexile Band
3
420 – 820
4–5
740 – 1010
6–8
925 – 1185
9 – 10
1050 – 1335
11-12
1185 – 1385
Individual & Partner Activity
1. On your own, read David and the Phoenix passage.
2. As you read, think of possible CR questions you
might write for students.
3. With your partner, write 2 T-D CR questions.
Resources:
- SBAC sample questions
- Pink sheet sample questions
- Prompts for TDQ’s
- Bloom’s verbs
Back to the Teacher Session
• We are moving into a 10-15 min work session where
teachers will create questions using the text they
brought from their classroom.
• They will give this text and questions to students
before Session Two.
How do we create
Text-Dependent
Constructed Response questions?
A Constructed Response Question:
- from handout • Clearly tells students what they are going to do.
• Uses simple, but authentic vocabulary and good sentence
structure.
• Identifies the information or materials that students
should use when preparing their response. (article, video, chart, etc.)
• Clearly indicates the process that should be demonstrated.
(explain, describe, analyze, evaluate, etc.)
• Identifies key words by using bold text as needed.
• Add: Requires students to use evidence from the text to support
their answers.
Text-Dependent CR Question
Sample Question:
Draw two examples of where the seed travels in the
story.
Kindergarten Teachers
David Douglas S.D.
Sample Questions:
From the story, name two ways Kitten takes care of
Patches. (Britton Finds a Kitten)
Give two details from the article about
what people do on Thanksgiving in the United States.
Grade 1 Teachers
David Douglas S.D.
Sample Question:
Describe two problems that Paul faced as a child. Use
details from the story to support your answer.
Grade 2 Teachers
David Douglas S.D.
Sample Question:
After reading the folktale, “Catching the Sun”, name at
least four reasons why the people and the animals,
who were living in the night, were sad.
Grade 3 Teachers
David Douglas S.D.
Sample Question:
How is a drought similar to other disasters? How is it
different?
Using information from the article, describe how a
drought is similar to other disasters. How is it different?
Article: “Where’s the Water?”
Sandy Duty, Grade 4
Gresham-Barlow S.D.
Sample Question:
Explain how the scientific method has changed
theories about our solar system. Use at least two
examples from the video.
Video: Changing Theories - The Scientific Method in Action
Grade 5 Teachers
David Douglas S.D.
Facilitated Work Session
• Work alone or with a partner who has the same
reading passage.
• Work quietly to not disturb others.
• Leave today with an assessment to use immediately in
your classroom.
Directions for Creating CR Questions
1. Review the reading passage independently to get a good
understanding of the content.
2. Create two text-dependent Constructed Response
questions for this reading passage, and write them on
notebook paper/device.
Resources
• SBAC sample questions
• Levels of questions
• Prompts
• Bloom’s verbs
Time to Share
• Get together with another person or pair and share the
questions you created.
• Discuss the questions according to the criteria on your
handout.
Trainers:
Report Out!
Or, teachers can do
the activity on the
next slide...
depending on
remaining time and
background
knowledge.
Evaluating and Refining Your Questions
Does the question have value and is it worthy of
students’ time?
Is the question text dependent, requiring understanding
of the text and the use of textual evidence?
Is the question aligned to and reflective of the rigor of the
CCSS?
Review questions according to the three
qualities on your handout.
Make needed revisions.
Next Steps: Baseline Assessment
• Prepare the passage and questions for student use.
• Give the reading passage and questions to students
by _______.
• Bring a set of student answers to our session on _______.
• We will create rubrics and score the answers.
Capturing Our Work
• Please place all materials in your folder.
• Bring this folder to our next session
Please thank your partner and
small group members
for their good work.
Trainer Check-In
• Look back at the last few slides reviewing:
– Creating questions for reading passage
– Evaluating and refining questions
• What elements might you use in a training?
• What might you revise?
Trainer: Partner Talk
• Share the passage you brought today.
• Would this passage render two TDQ’s?
Brainstorm:
What are the characteristics of a passage teachers
should bring to the first session?
Report Out!
Session Two Goals
• Refresher: Text-Dependent Constructed Response
Questions
• Create CR Rubric and Expected Answer
• Score Student Work
Please locate a partner
Smarter Balanced Assessment
•Require the student to generate a response as
opposed to selecting a response.
•Include both short and extended responses.
•Allow students to demonstrate their use of complex
thinking skills consistent with the expectations for
college and career readiness.
CR questions are are Text-Dependent when students
have to support their answer with evidence from the
reading passage.
Trainers: After the first session, ask teachers to
provide you with sample questions you can share at
this second session.
Analyze / Integrate Information Rubric
2
1
The response gives sufficient evidence of the ability to locate,
select, interpret and integrate information within and among
multiple sources.
The response gives limited evidence of the ability to locate,
select, interpret and integrate information within and among
multiple sources.
• Only uses one source as support
0
The response gets no credit if it provides no evidence of the
ability to locate, select, interpret and integrate information
within and among multiple sources.
Sample Rubrics: SBAC Sample Question packet from Session One
Steps to Creating a Rubric
• Choose one of the questions you wrote.
• On the template, create a rubric for the skill being
assessed. (describe, explain, compare, etc.)
• Make notes on what would separate score points.
Look back at Sample
Questions
Sample Question:
Describe two problems that Paul faced as a child. Use
details from the story to support your answer.
Grade 2 Teachers
David Douglas S.D.
Rubric for A Bed for Paul Question
2-Point Rubric
2
The response gives sufficient evidence of the ability to describe
problems Paul faced as a child
• Gives two supporting examples.
1
The response gives limited evidence of the ability to describe
problems Paul faced as a child.
• Gives only one supporting example.
0
The response gets no credit if it provides no evidence of the
ability to describe problems Paul faced as a child.
Make notes on differences between score points.
Partner Talk
How would you create a rubric for these questions?
(Look at your template.)
From the story, name two ways Kitten takes care of
Patches.
Give two details from the article describing what people
do on Thanksgiving in the United States.
Explain how the scientific method has changed theories
about our solar system. Use at least two examples from
the video.
Step 1: Create a Rubric
• Choose one of the questions you wrote.
• On the template, create a rubric for the skill being
assessed. (describe, explain, compare, etc.)
• Make notes on what would separate score points.
Step 2: Create the Expected Answer
• What is the expected answer for a score of 2?
• Write down on template:
- the elements of the 2 score answer – or –
- a sample answer your student might write
Examples
Step 2: Create the Expected Answer
for the Score of 2
Two problems Paul faced as a child:
- Not having a big enough bed
- Not having a big enough blanket
Step 2: Create the Expected Answer
for the Score of 2
Where the seed traveled:
-
Sun
Icy mountain
Ocean
Desert
Ground
Step 2: Create the Expected Answer
• What is the expected answer for a score of 2?
• Write down on template:
- the elements of the 2 score answer – or –
- a sample answer your student might write
Stop!
The next slides and activities are for this training of
trainers only.
We will get back to teacher training in a minute.
From David and the Phoenix
Question 1:
How does David feel when he sees the mountain for
the first time? Use two details from the story to
support your response.
Better prompt: Explain how David feels….
Question 2:
What do paragraphs 4-7 show about David? Use two
details from the story to support your response.
Better prompt? _______________
Partner Work
1. On the template, create 2 rubrics for the skills being
assessed. (describe, explain, compare, etc.)
2. Create the expected answer for the score of 2 for
both questions.
Note
I would only have the teachers create one rubric during
the session – not repeat for a second question – or do
this during another training.
The idea is to have them see what the differences are
between the score point. They would not necessarily
have to create a rubric for every question they wrote –
but recording the expected score of 2 might make sense.
How could the “rubric writing” activity be improved?
Back to “real time” session.
Scoring Student Work
• Take out the student work you brought today.
• Use the rubric and expected answer as resources
when you score the CR question.
• Write the score of 2, 1, or 0 on the student’s paper or
on post-it notes.
Scoring Student Work
• After you score 5-8 papers, start jotting down the
trends you are seeing in the student work: (notebook
paper)
* What are students doing well?
* What skills need strengthening?
• Continue scoring and tracking trends.
Time to Share
• Get together with another person or pair and share the
trends you noticed.
• What instruction is needed to boost the skills needed to
answer CR questions?
Report Out!
Trainers: Mock Scoring Session
• Ivory handout: David and the Phoenix • Student Work
• Imagine these are ½ sheets with student answers to
the question we reviewed.
• On your own, score all of the questions according to
the rubrics and expected answers you created.
When you see your partner is finished scoring,
quietly start discussing scores.
Trainers: Review Scores
Look at scores from Engage NY project.
During your training session, teachers with the same
reading passage could exchange a few papers and crossscore, then discuss student work.
Teachers could reflect on Trends notes – and discuss how
they might teach the key skills students need for success.
Baseline Formative Assessment
• Keep the scores of your student work as a baseline.
• Use results to determine the skills you need to teach
students.
• Create mini lessons with your team using text from your
curriculum.
• Create the next round of CR questions.
• Score student work and look for
growth from the baseline assessment.
Trainer Check-In
• Look back at the last few slides reviewing:
– Creating rubrics and expected answer
– Scoring student work
• What elements might you use in a training?
• What might you revise?
Trainers: Next Steps
Session 3 Ideas
• Teachers brainstorm skills needed to answer TDQ:
- putting part of question in answer
- supporting answer with evidence from the text
- analyzing, synthesizing, describing, explaining, etc.
• Create mini lessons to teach key skills (modeling).
• Use template?
Facilitated Planning Time
• Review the information and resources shared today.
• Create a training outline – depending on the number
of sessions available and amount of time in each
session.
• Use Penny as a resource in your planning.
Please thank your partner and
small group members
for their good work.