Professional Development to Practice
Common Formative Assessment
Sound Assessment Design
The contents of this presentation were developed under a grant from the
US Department of Education to the Missouri Department of Elementary and
Secondary Education (#H323A120018). However, these contents do not
necessarily represent the policy of the US Department of Education, and
you should not assume endorsement by the Federal Government.
Professional Development to Practice
Welcome and Introductions
Please take a moment to introduce (or
reintroduce) yourself to the group, by telling your
name, district, and position.
Our trainers for the day are….
Professional Development to Practice
Introduction to Missouri Collaborative Work
Use Getting Started Guide to determine starting point and scope of learning
Getting Started
Focus Areas
Collaborative Work Training
Wrap Up Activity
Follow-up to Training
Collaborative Data
Teams (CDT)
Overview and
Purpose
Collaborative Teams
Effective Teaching/
Learning Practices (EP)
Common Formative
Assessment (CFA)
Overview and
Purpose of CFA
Overview and
Purpose of EP
Agendas
Communication
Norms
Roles
Advanced
Processes
Assessment
Assessment
Capable
Capable
Learners
Learners
Spaced
Spaced
versus
versus
Massed
Massed
Reciprocal
Teaching
Feedback
Data Team Process
Steps Sequence
and Examples
Quality
Assessment
Design
Selected
Response
Items
Consensus
Collaborative Skills
Protocols
Activity: Wrap
Up/Overview of
Next Steps
Overview and
Purpose of DBDM
Developing
Meaningful
Learning Targets
Foundational
Processes
Activity: Wrap
Up/Overview of
Next Steps
Constructed
Response
Items
Performance
Events
Activity: Wrap
Up/Overview of
Next Steps
Follow-Up Based on Data: Coaching and Revisiting PD
September 2013
The contents of this presentation were
developed under a grant from the US
Department of Education to the
Missouri Department of Elementary
and
Secondary
Education
(#H323A120018). However, these
contents do not necessarily represent
Data-Based Decision
Making (DBDM)
1. Collect and Chart
Data
2. Analyze and
Prioritize
3. SMART Goal
4. Instructional
Decision Making
5. Determine Results
Indicators
6. Ongoing
Monitoring
Activity: Wrap
Up/Overview of
Next Steps
School-Based
Implementation Coaching
Overview and
Purpose of
Coaching for
supporting
school-wide
implementation
Critical skills of
coaching
Coaching in
Practice
Activity: Wrap
Up/Overview of
Next Steps
Follow-Up Based on Data:
Coaching and Revisiting PD
Professional Development to Practice
Common Formative
Assessment (CFA)
Overview and Purpose
of CFA
Developing
Meaningful Learning
Targets
Quality
Assessment
Design
Quality Assessment
Design
Selected
Response
Items
Constructed
Response
Items
Performance
Events
Professional Development to Practice
Learner Objectives of the
Formative Assessment Series
Understand the clear purposes of assessment by clarifying
Why they are assessing
Who will use the results of assessment data
What they will do with the assessment data
Develop clear and meaningful learning targets to guide
instruction and student learning.
Construct quality assessment instruments which are of
sound design and measure pre-determined learning
targets.
Professional Development to Practice
Outcomes for the Day
As a result of todays training you will…
develop a better understanding of the components
and characteristics of a quality formative assessment.
develop a better understanding of various types of
assessment items and the pros and cons of each type.
continue using a backwards design approach and a
template form to write a formative assessment.
evaluate your formative assessment for quality.
Professional Development to Practice
Module 3: Quality Assessment
Design
Essential Questions:
1. What decisions drive the type of assessment
items to use in common formative assessments?
2. What are the essential components needed to
create a quality formative assessment?
3. What are the characteristics of quality
selected-response, constructed-response and
performance tasks?
Professional Development to Practice
Session at a Glance
 Introductions/Objectives/Outcomes/ Norms
 Brief Review of Assessment Principles and Reflection
 Selected Response Items
 Constructed Response Items
 Performance Tasks
 Continue with Assessment Development Process
 Evaluate Your Test
 Using Data to Inform Test Writing Skills
 Implementation Steps, Roadblocks, and Supports
 Additional Learning
 Closure
Professional Development to Practice
Norms
Begin and end on time
Be an engaged participant
Be an active listener – open to new ideas
Use notes for side bar conversations
Use electronics respectfully
Professional Development to Practice
Accurate Assessment
ASSESS WHAT?
What are the learning
targets?
Are they clear?
Are they good?
WHY ASSESS?
What’s the purpose?
Who will use the
results?
ASSESS HOW?
What method?
Written well?
Sampled how?
Avoid bias how?
Be sure students
understand targets too!
Students are users too!
Students track progress and
communication,!
COMMUNICATE
HOW?
How is information
managed?
reported?
Students can participate
in the process too!
Effectively Used
Source: Adapted from Classroom Assessment for Student Learning: Doing it Right-Using
it Well., by R.J. Stiggins, J.Arter, J.Chappuis, & S. Chappuis, 2004, Portland, OR.
Professional Development to Practice
Accurate Assessment
ASSESS HOW?
What method?
Written well?
Sampled how?
Avoid bias how?
Source: Adapted from Classroom Assessment for Student Learning: Doing it Right-Using
it Well., by R.J. Stiggins, J.Arter, J.Chappuis, & S. Chappuis, 2004, Portland, OR.
Professional Development to Practice
Read and Reflect with a shoulder partner…
Every educator must understand the principles of
sound assessment and must be able to apply those
principles as a matter of routine in doing their work.
Accurate assessment is not possible unless and until
educators are given the opportunity to become
assessment literate. (They) must understand student
achievement expectations and how to transform
those expectations into accurate assessment exercises
and scoring procedures. (NEA, 2003)
Common Formative Assessments, Larry Ainsworth
& Donald Viegut, 2006, Corwin Press, pg 53
Professional Development to Practice
What is Assessment Literacy?
“The ability to understand the different purposes
and types of assessment in order to select the
most appropriate type of assessment to meet a
specific purpose.” (Larry Ainsworth, 2006)
Professional Development to Practice
What evidence do we need that students have
met our stated purpose(s)?
“Fruitful assessment often poses the question
‘what is an array of ways I can offer students to
demonstrate their understanding and skills?’ In
this way, assessment becomes a part of teaching
for success and a way to extend rather than
merely measure learning.”
Quote by Carol Ann Tomlinson, 1995, taken from
Common Formative Assessment by Ainsworth and Viegut
Professional Development to Practice
Video Clip
http://www.youtube.com/watch?v=_CqgnZhb--Q
Professional Development to Practice
Assessment of Missouri Learning
Standards
The knowledge, skills and processes specified in
Missouri’s Learning Standards (MLS) for Mathematics
and English and Language Arts will be measured by
Smarter Balanced Assessment Consortium (SBAC) using
a variety of test item types…..selected response,
constructed response and performance tasks.
Sample SBAC items may be viewed on the website:
http://www.smarterbalanced.org/sample-items-andperformance-tasks
Professional Development to Practice
Let’s define…
Selected-response assessments
Constructed-response assessments
Performance Assessments
Pull out this template and
pair up with someone.
Using your current level
of understanding, create
a definition, identify the
benefits and drawbacks
of each type of
assessment.
Professional Development to Practice
Selected-Response Assessments…
Require students to select one response from a
provided list
Types include multiple-choice; true-false; matching;
short answer/fill-in-the-blank
Also include short answer/fill-in-the-blank (with a
listing of answer choices provided)
Assess the student’s knowledge of factual information,
main concepts, and basic skills
Professional Development to Practice
Benefits of Selected-Response Items
Can be scored quickly
Can be scored objectively as correct or
incorrect
Covers a wide range of content
Professional Development to Practice
Drawbacks to Selected-Response Items
Tends to promote memorization of factual
information rather than higher-level
understanding
Inappropriate for some purposes (performance,
writing, and creative thinking)
Lack of student writing in most cases, unless
part of assessment design (Haladyna, 1997,
pp.65-66)
Professional Development to Practice
Five Roadblocks to Effective
Item Writing
Unclear directions
Ambiguous statements
Unintentional clues
Complex phrasing
Difficult vocabulary
Popham, 2003b, p.64
Professional Development to Practice
Key Points for Writing SR Items
Choose a selected-response format(s) that aligns
to the standard being measured
Make sure the items will produce the needed
evidence to determine mastery of the standard
Include the vocabulary of the standard selected
for assessment (as appropriate)
Make sure the test question(s) require the same
level of rigor as that of the standard
Write each stem first, then write distractors
Professional Development to Practice
Example of a Selected Response Item
Many experts will tell you that television is bad for you. Yet this is an
exaggeration. Many television programs today are specifically geared towards
improving physical fitness, making people smarter, or teaching them important
things about the world. The days of limited programming with little interaction
are gone. Public television and other stations have shows about science, history,
and technical topics.
Which sentence should be added to the paragraph to state the author’s main claim?
A. Watching television makes a person healthy.
B. Watching television can be a sign of intelligence.
C. Television can be a positive influence on people.
D. Television has more varied programs than ever before.
Professional Development to Practice
Example of another Selected
Response Item
Smarter-Balanced
Assessment
Consortium (SBAC)
has multiple-choice
items that cue
students to select
more than one
answer.
Professional Development to Practice
A third example of a Selected Response Item
Use the illustration and your knowledge of social studies to answer the following question.
5. What colonial claim about the Boston Massacre is supported by this illustration?
a. Most American colonists in Boston were killed.
b. British soldiers fired on unarmed colonists.
c. There were more soldiers than civilians at the Boston Massacre.
d. Colonists were better equipped for war than British soldiers were.
Professional Development to Practice
Constructed-Response Items…
Require students to organize and use knowledge and
skills to answer a question or complete a task
Types include short-answer; open response; extended
response; essays
More likely to reveal whether or not students
understand and can apply what they are learning.
May utilize performance criteria (rubrics) to evaluate
degree of student proficiency.
Professional Development to Practice
Benefits of Constructed-Response Items
Responses will contribute to valid inferences about
student understanding better than those derived from
selected-response items.
Measure higher-levels of cognitive processes.
Allow for diversity in student responses or solution
processes.
Provide a better picture of students’ reasoning
processes.
Promote the use of evidence to support claims and
ideas.
Professional Development to Practice
Drawbacks of Constructed-Response
Take longer to score
Can have errors in design
Dependent on student writing proficiency
A challenge to score consistently and
objectively
Must have clear rubrics for scoring criteria so
scoring is not subjective
Professional Development to Practice
Video Clip
http://www.youtube.com/watch?v=gp7W6wV-obs
Professional Development to Practice
Key Points to Writing
Constructed Response Items
 Items should be open-ended and require students to
create a response
 Students must demonstrate an integrated
understanding of the “unwrapped” concepts and skills
 Items must match the level of rigor of the
“unwrapped” standards
 A proficient answer reflects the understanding of
higher-order instructional objectives
 Constructed-response items MUST be accompanied by
scoring guides.
Professional Development to Practice
Example of Constructed
Response Item
The table shows the price of different quantities of medium-sized
apples at Tom’s Corner Grocery Store. What is the least amount of
money needed to buy exactly 20 medium-sized apples if the bags
must be sold intact and there is no tax charged? Be sure to show
all of your work.
Number of
Apples
Total Price
Bag of 1
Bag of 6
Bag of 12
$.30
$1.20
$2.10
Professional Development to Practice
Another example of Constructed
Response Item
Scenario: Your friend is using the computer to type his
one-page report for history. It is just two lines over one
page and he doesn’t know how to make it fit on one
page.
Question: Using the proper computer terminology,
describe to your friend two valid solutions for making his
report fit on one page without deleting any of the
content.
Professional Development to Practice
A third example of Constructed
Response Item
Stimulus: Information of Sally’s Experimental
Design
Evaluate Sally’s experimental design. Identify
two things Sally could have done differently to
make her results more valid. Give reasoning for
each one of your suggestions.
Professional Development to Practice
Performance Tasks…
Require students to construct a response, create a
product, or perform a demonstration.
Are open-ended and usually allow for diversity in
responses or solution processes
Are evaluated using scoring criteria given to students
in advance of performance
Highly engaging for students
Promotes critical thinking and/or problem solving
Promotes peer and self assessment
Professional Development to Practice
Benefits of Performance Tasks
Have the ability to assess multiple learning
targets and application of knowledge
Highly engaging for students
Promotes critical thinking and/or problem
solving
Promotes peer and self-assessment
Offers multiple opportunities for students to
revise work using scoring guide feedback.
Professional Development to Practice
Drawbacks of Performance Tasks
Rubrics are more involved and take longer to
develop
Performances take longer to score
Can have error in evaluative design
Success is often dependent on factors other than
those targeted for assessment (i.e. writing
ability, verbal skills, physical abilities, etc.)
A challenge to score objectively and consistently
Professional Development to Practice
Key Points to Writing a Performance Task
 Student performance should be recorded on a checklist or scoring
rubric.
 Contains a written prompt that cues the student to perform some type
of task that requires a demonstration, presentation or product
creation.
 Shows connections by measuring learning targets across strands or
content areas.
 Model what application of learning looks like in life beyond the
classroom.
 Should measure mastery of multiple learning targets and higherlevel cognitive processes.
 May be completed on one or more sittings or over time.
Professional Development to Practice
Example of Performance Task
Part I: During the U.S. Civil War, quilts became a popular item for women to
make. You will write an informative essay summarizing the history and purposes
of civil war quilts.
To gain the information needed to write your essay, you will watch a video and
read two articles about quilts that were made during the Civil War. Take notes
because you may want to refer back to your notes while writing your essay.
Part II: Your class is planning a field trip to a history museum. To help prepare
for what you will see, write an informative essay about Civil War quilts. In your
essay, discuss the history of the quilts, including the reasons people made these
quilts during the Civil War, and explain how the quilts were made. Include
evidence from the sources in Part I to help support the information you include
in your essay. The rubric is provided showing how your essay will be scored.
Professional Development to Practice
Another example of Performance Task
Professional Development to Practice
A third example of Performance Task
Professional Development to Practice
Reflection Time
Think about the content, skills and processes you
teach in your classroom and answer the three
questions below.
What content information
and simple skills do you
teach in your classroom
that would lend itself well
to being assessed by using
SR items?
What concepts, principles
and processes do you
teach in your classroom
that would lend itself well
to being assessed by the
use of CR items.
When might you ask
students to do a PT to
show their application
of multiple skills and
processes?
Professional Development to Practice
Module #3 is a continuation of
Module #2!
Module 2
Module 3
Professional Development to Practice
Now let’s practice!
Using either the
sample CFA
Development
Template, or
personal work
created in steps
1 through 5
previously,
complete Steps 6
through 9 by
selecting the
appropriate
types of
assessments,
matching up the
test items with the
learning target.
Professional Development to Practice
Step 10: Define Achievement Levels
7. Selected Response: Write Test Items:
8. Constructed Response: Write Test Items:
9. Performance or Personal Communication
Write Test Items:
Give correct answers.
Create Scoring Rubrics
Create Scoring Rubrics.
10. Define Achievement Levels: Describe how information from the scoring guides can be used collectively to determine achievement levels for
students. These levels will be used in the Data Team Process. (In example below, students complete a 7 question formative assessment.
Questions 1-5 are selected response, and questions #6 and #7 are constructed response items with either 3 pt or 4 pt rubrics for scoring)
Correct answers on all 5 SR items, at least 2 out of 3 on CR item #6, and at least 3 out of 4 on CR item #7
Proficient & Higher
Correct answers on at least 3-4 SR items, at least 2 out of 3 on CR item #6, and at least 2-3 out of 4 on CR item #7
Close to Proficient
Correct answers on 1-2 of 5 SR items, at least 1 out of 3 on CR item #6, at least 1 out of 4 on CR item #7
Far to Go
Correct answers on 0-1 SR items, OR 0-1 out of 3 on CR item #6, OR 0-1 on CR item #7
Intervention
Proficient & Higher
Close to Proficient
Far to Go
Intervention
11. Review and Revise……Exchange tests with another group. Evaluate the overall quality of the assessment as well as the individual items within
the test. Make suggestions and return test to writers for them to make suggested revisions.
NEXT STEPS:
12. Give the Pre-Assessment to students and collaboratively score—begin the DT process by charting the results for each teacher and for subpopulations.
13. Evaluate the students understanding of the BIG ideas as you go along with the unit of study by using the Essential questions…an indicator of
what’s happening as you continue with the unit of study.
14. Give the Post-Assessment to students and collaboratively score----chart post test results. Compare Pre-Test Results with Post-Test Results.
Determine next steps.
(Adapted from Larry Ainsworth’s resources—Formative Assessment and Leadership and Learning CFA Resources))
When items are
written,
complete step
10 by
describing how
information from
the scoring
guides can be
used collectively
to determine
achievement
levels for
students. These
levels will be
used later in the
Data Team
Process.
Professional Development to Practice
Step 11: Review and Revise
7. Selected Response: Write Test Items:
8. Constructed Response: Write Test Items:
9. Performance or Personal Communication
Write Test Items:
Give correct answers.
Create Scoring Rubrics
Create Scoring Rubrics.
10. Define Achievement Levels: Describe how information from the scoring guides can be used collectively to determine achievement levels for
students. These levels will be used in the Data Team Process. (In example below, students complete a 7 question formative assessment.
Questions 1-5 are selected response, and questions #6 and #7 are constructed response items with either 3 pt or 4 pt rubrics for scoring)
Correct answers on all 5 SR items, at least 2 out of 3 on CR item #6, and at least 3 out of 4 on CR item #7
Proficient & Higher
Correct answers on at least 3-4 SR items, at least 2 out of 3 on CR item #6, and at least 2-3 out of 4 on CR item #7
Close to Proficient
Correct answers on 1-2 of 5 SR items, at least 1 out of 3 on CR item #6, at least 1 out of 4 on CR item #7
Far to Go
Correct answers on 0-1 SR items, OR 0-1 out of 3 on CR item #6, OR 0-1 on CR item #7
Intervention
Proficient & Higher
Close to Proficient
Far to Go
Intervention
11. Review and Revise……Exchange tests with another group. Evaluate the overall quality of the assessment as well as the individual items within
the test. Make suggestions and return test to writers for them to make suggested revisions.
NEXT STEPS:
12. Give the Pre-Assessment to students and collaboratively score—begin the DT process by charting the results for each teacher and for subpopulations.
13. Evaluate the students understanding of the BIG ideas as you go along with the unit of study by using the Essential questions…an indicator of
what’s happening as you continue with the unit of study.
14. Give the Post-Assessment to students and collaboratively score----chart post test results. Compare Pre-Test Results with Post-Test Results.
Determine next steps.
(Adapted from Larry Ainsworth’s resources—Formative Assessment and Leadership and Learning CFA Resources))
Review test items
collaboratively to
affirm the quality
and
appropriateness of
the test items.
Professional Development to Practice
Evaluate Your Test
 Collectively, do the assessment items produce the necessary evidence to
determine whether or not the student has mastered the standard(s) targeted
for assessment?
 Are the assessment items the most appropriate type to use to measure the
targeted standard?
 Do the assessment items require students to demonstrate the same level of
rigor as specified by the targeted standard?
 Are the items worded clearly and concisely?
 Are the directions clear so students clearly understand what they are to do?
 Are the items free of any type of bias?
Professional Development to Practice
Using Data to Inform Test Writing Skills
Use of data from a careful item analysis can
help a teacher improve his/her test writing
skills.
Additionally, looking at student results can give
the teacher ideas as to improvements that need
to be made in instruction and/or curriculum.
See next slide.
Written by Jana Scott, MAP Instructional Facilitator, University of MO-Columbia, 2007.
Professional Development to Practice
Possible Causes for Faulty
Items/Low Scores on Items
1. Basic content or skills have not been addressed or taught.
2. Students are unfamiliar with the process needed to solve the problem/answer the
question (i.e. problem solving, deductive/inductive thinking, making an inference, etc.)
3. Students are unfamiliar with the format needed to answer the question. (i.e. political
cartoon, letter, graph, etc.)
4. Students are unfamiliar with the meaning of the language used in the test item. (i.e.
compare and contrast, paraphrase, illustrate, evaluate, etc.)
5. Lack of reading ability. Vocabulary used in item stem or stimulus is too difficult.
6. Wording of the item is unclear or confusing.
7. The rubric does not align with the test item. The rubric holds students accountable for
something that was not cued in the item stem.
8. The rubric holds students to a standard that is not grade-level appropriate.
9. The item is asking the impossible or improbable (i.e. Asking for two similarities and two
differences when there are not that many. Asking for three details when there are not
that many.)
10. The stimulus material used as a basis for item development is at fault.
Written by Jana Scott, MAP Instructional Facilitator, University of MO-Columbia, 2007.
Professional Development to Practice
Reflection
Based on what you have learned today,
What steps might you take in order to become a
“top notch” writer of formative assessments and of
the various types of test items?
 What potential challenges do you foresee? How
might these be overcome?
What tools and/or resources might you use to
ensure the assessments and assessment items you
write are top quality?
Professional Development to Practice
Practice Profile
Missouri Collaborative Work Practice Profile
Foundations present in the implementation of each essential function: Commitment to the success of all students and to improving the quality of instruction.
Co m m o n Fo r m a t iv e As s ess m en t
Close to
Far from
Proficient
Proficient
(Skill is
Exemplary Proficiency
(Follow-up
emerging, but
Essential Functions
Ideal Implementation
Proficient
professional
Evidence
not yet to ideal
development
proficiency.
and coaching
Coaching is
is critical.)
recommended.)
All of the following criteria are met.
At least 6 of At least 4 of
Less than 4 of
Common
the criteria are
the criteria are Formative
 Learning goal is clearly connected to a big idea/essential the criteria
are met.
met.
met.
Assessment
learning in the domain
Development &
 Learning goal develops deep understanding of
Implementation
underlying concepts and/or acquisition of skills
Educators develop
Template.
clear and meaningful
 Learning goal clearly engages higher order thinking
1 learning goals to guide
processes
instruction and
 Learning goal is clearly manageable and can be
student learning.
accomplished in the course of a lesson or unit (may be
several periods)
 Learning target is clearly explained to students
 Connections between current learning goal and prior
learning are clearly made
All of the following criteria are met.
At least 3 of At least 2 of
Less than 2 of
Common
the
criteria
the
criteria
are
the
criteria
are
Formative
 Success criteria are clearly and effectively aligned to
are
met.
met.
met.
Assessment
learning goals
Development &
 Success criteria clearly and effectively relate to what
Educators establish
Implementation
students will say, do, make or write to show evidence of
clear and measureable
Template.
learning
student success
2
 Success criteria clearly and effectively reflect ways for
Professional Development to Practice
Implementation Fidelity
Yes
1. Common formative assessment is linked to selected
learning standards.
2. Learning goal engages higher order thinking
processes.
3. Learning goal can be accomplished in the course of
a unit.
4. Learning target is written in language that students
can clearly understand.
5. Learning target is clearly explained to students.
6. Success criteria are written in language that
students can clearly understand in a rubric or
checklist.
7. Students receive feedback based on learning goal
and their assessment results.
8. The quality of assessment items for measuring
mastery is reviewed and items revised as needed.
Total
Partially No
If partially or no, please
explain.
Professional Development to Practice
For Additional Learning
 In-depth additional training from RPDC staff members on how to
write quality selected response items, constructed response items
and performance events.
 Books contained in Bibliography on the next slide.
 Websites and Videos about Formative Assessment
 http://www.youtube.com/watch?v=2K8qbI_FzGE
 http://www.amle.org/Publications/WebExclusive/Assessment/tabid/1120
/Default.aspx
 http://www.ncpublicschools.org/docs/accountability/educators/fastresear
chresources.pdf
 http://www.ncpublicschools.org/accountability/educators/vision/formative
Professional Development to Practice
Bibliography
Common Formative Assessments: How to Connect
Standards-Based Instruction and Assessment; Larry
Ainsworth, Donald Viegut; Corwin Press;2006.
Common Formative Assessment Training Manual;
Second Edition; The Leadership and Learning Center;
Houghton Mifflin Harcourt;2011.