DEVELOPING QUALITY
ASSESSMENTS
Presented by Ellen Vorenkamp for the MI Assessment Consortium
at Lewis Cass ISD Best of Formative Assessment Series Session 5
Overview

The need for Quality Assessments


Accurate Data
Use of Data



The Development Process



MAC Modules
Planning for your Assessment (CASL)
Key Components—Common Core Connections



Student Growth
Educator Evaluation
Test Blue Prints
Quality Items
Conclusion
Good Assessment applies to all
Learners




Regardless of student ability or communication
level, our assessments must have a clear purpose
and stretch our students to higher levels of doing
and thinking
Good assessment design is fundamental to all
Each student is unique
As ever – take the best and adapt to our kids!
Essential Questions to Dialogue
Around…

As you develop quality assessments, keep the
following questions in mind…
 What
is the purpose of the assessment?
 Who will use the information?
 How will it be used?
 Is the use formative or summative?
A Formative View

As you develop quality assessments, keep the
following questions in mind…
 What
is the purpose of the assessment?
 To
provide teachers immediate information on student
learning
 Who
will use the information?
 Teachers
 How
 To
 Is
and students in the classroom
will it be used?
inform next steps in the learning process
the use formative or summative?
 Formative
A Summative View

As you develop quality assessments, keep the
following questions in mind…
 What
is the purpose of the assessment?
 Educator
 Who
will use the information?
 Teachers
 How
 To
 Is
Evaluation / Accountability
and Administrators
will it be used?
certify the learning process
the use formative or summative?
 Summative
Think…Pair…Share…
What elements are necessary to ensure quality
assessments?
 List
these qualities
 Discuss why these are important
MAC’s Thinking…

24 Modules that cover the entire development
process…
 Planning
 Modules
1-7
 Developing
 Modules
8-20
 Implementing
 Module
21
 Reflecting
 Modules
21-24
Quality Assessments…

are Reliable and yield Valid data.

In order for these two requirements to be met assessment
developers must pay special attention to the following:
Standard/Item Alignment
 Balance of Representation
 Target-Method Match
 Quality Items


The best way to ensure your assessment is reliable and valid
is to create a test blueprint and follow the blueprint while
developing the assessment.
Test Blueprints


Requires a bit of planning and forethought as you
begin this process…
Several steps should be completed that will help
make developing your test blueprint easier
Standard Alignment




Are the assessment items tightly aligned with the
standards?
Do the assessment items address the targeted
standards sufficiently and at all levels of cognitive
demand?
Are there an equal number of assessment items per
standard? If not, is there “rationale”?
Are there enough assessment items per standard to
determine mastery?
Balance of Representation


Items are balanced and should reflect what was
taught and how it was taught within the classroom
environment
The assessment as a whole should reflect the
emphasis of the lesson, unit or other educational
experience on which you are assessing.
 For
example, a social studies assessment might cover 4
related standards or learning targets…
Balance of Representation
Lesson Instruction
34%
33%
2-E1.0.3
2-E1.0.1
2-E1.0.2
33%
Balance of Representation
Lesson Assessment
34%
33%
2-E1.0.3
2-E1.0.1
2-E1.0.2
33%
Target-Method Match
1)
2)
3)
Identify the Depth of Knowledge (DOK) for the
standard
Match the cognitive demand of the standard with
an appropriate item type
Determine which item types will be included in the
assessment and in what proportion.
16
Depth Of Knowledge
BLOOM’S TAXONOMY
KNOWLEDGE
“The recall of specifics and universals, involving little
more than bringing to mind the appropriate material”
COMPREHENSION
“Ability to process knowledge on a low level such that
the knowledge can be reproduced or communicated
without a verbatim repetition.”
APPLICATION
“The use of abstractions in concrete situations.”
ANALYSIS
“The breakdown of a situation into its component
parts.”
SYNTHESIS AND EVALUATION
“Putting together elements & parts to form a whole,
then making value judgments about the method.”
WEBB’S DOK
RECALL
Recall of a fact, information, or procedure (e.g., What
are 3 critical skill cues for the overhand throw?)
SKILL/CONCEPT
Use of information, conceptual knowledge, procedures,
two or more steps, etc.
STRATEGIC THINKING
Requires reasoning, developing a plan or sequence of
steps; has some complexity; more than one possible
answer
EXTENDED THINKING
Requires an investigation; time to think and process
multiple conditions of the problem or task.
Wyoming School Health and Physical Education Network (2001). Standards, Assessment, and Beyond. Retrieved
May 25, 2006, from http://www.uwyo.edu/wyhpenet
Prepared and published by the Educational
Policy Improvement Center
Prepared and published by the Educational
Policy Improvement Center
Reflection…

What are the implications for classroom instruction?

What are the implications for classroom assessment?
Types/Levels of Standards




Recall and Reproduction– (Knowledge) Facts and concepts
we want students to know (Usually DOK Level 1 or 2)
Skills & Concepts – (Skills) Students use their knowledge and
reasoning to act skillfully (Usually DOK Levels 2; Can go
higher based on context)
Strategic Thinking– (Reasoning) Using what they know to
reason and solve problems (Usually DOK Levels 2-4)
Extended Thinking– (Application/Products) use knowledge,
reasoning, and skills to create a concrete product (Usually
DOK 3-4)
Methods of Assessments
Selected
Response
•Multiple
Choice
•True-False
•Matching
Constructed
Response
•Diagram
•Fill-in-the-blank
(words, phrases)
•Essay
•Short answer
(sentences,
paragraphs)
•Web
•Concept Map
•Flowchart
•Graph
•Table
•Matrix
•Illustration
Adapted from the work of Dr. Robert Marzano
Performance
Assessment
Observations/
Conversations
•Presentation
•Movement
•Science lab
•Athletic skill
•Dramatization
•Enactment
•Project
•Debate
•Model
•Exhibition
•Recital
•Performance
Task
•Oral questioning
•Observation
•Interview
•Conference
•Process
description
•Checklist
•Rating scale
•Journal sharing
•Thinking aloud
a process
•Student selfassessment
•Peer review 23
Target-Method Match
How well does your method of assessment match your target?
Assessment Method
Target to be
Assessed
Selected
Response/
Short-Response
Extended Response
Performance
Assessment
Observation/
Conversations
Knowledge/
Recall
Good match
Good match
Not a good
match
Partial match
Reasoning/
Strategic
Thinking
Partial match
Good match
Good match
Good match
Performance
Skills
Not a good
match
Not a good
match
Good match
Partial match
Application/
Products
Not a good
match
Partial match
Good match
Not a good
match
Test Blueprint
Putting it all together:
 Standard
 Specific Learning Target derived from Standard
 Target DOK Level
 Assessment Method
 Number of Items/Points
Performance Tasks

Performance tasks challenge students to apply
their knowledge and skills to respond to
complex real-world problems. They can best be
described as collections of questions and activities
that are coherently connected to a single theme or
scenario. These activities are meant to measure
capacities such as depth of understanding, writing
and research skills, and complex analysis, which
cannot be adequately assessed with traditional
assessment questions. The performance tasks may
take one to two class periods to complete.
Components of a Performance
Assessment

Performance Assessments:


call upon the examinee to demonstrate skills and competencies
(application)
are comprised of 4 basic components:

A reason for the assessment


A specific performance to be evaluated


Construct the prompt or item and determine the criteria for success
Exercises to elicit that performance, and


Identify the content and/or skill standard(s) to be assessed
Describe the way you will collect evidence, when you will collect the
evidence and how much evidence you will collect.
Systemic rating procedures

Determine the score for proficiency or mastery; determine who will rate the
performance, scoring method (rubric, checklist, anecdotal record, portfolio)
SBAC Performance Tasks (PT)

Performance tasks, the most complex of all items, include the
following elements:
Integrate knowledge and skills across multiple claims.
Measure capacities such as depth of understanding, research
skills, and/or complex analysis with relevant evidence.
Require student-initiated planning, management of
information/data and ideas, and/or interaction with other materials.
Reflect a real-world task and/or scenario-based problem
Allow for multiple approaches.
Represent content that is relevant and meaningful to students.
Allow for demonstration of important knowledge and skills.
Require scoring that focuses on the essence of the Claim(s) for
which the task was written.
Seem feasible for the school/classroom environment.
PT Sample: Prior to Test Date
PT Sample: Stimuli and Tasks
PT Sample: Stimuli and Tasks
PT Sample: Stimuli and Tasks
PT Sample: Stimuli and Tasks
PT Sample: Stimuli and Tasks
th
4
Grade ELA Sample
th
4
Grade ELA Sample
th
4
Grade ELA Sample
Additional Sample Items

http://www.dese.mo.gov/divimprove/assess/docum
ents/asmt-sbac-math-hs-sample-items.pdf
 Scroll
down to page 3
Implications for the Classroom



Assessments will evolve to be more rigorous
and real world relevant, must match our
teaching to this same standard
Assessment data must be used in the moment
to inform “next steps” in the learning
process
Schools must have a balanced assessment
system in place within their classrooms
Not the end of the journey…


MAC Modules
Much left to do to ensure quality assessments
A Final Thought…

“Remember, formative assessment works,” says
Popham. “When it is used, students learn better. By
using this assessment-rooted instructional process,
teachers can increase the test-based achievement of
their students… ‘Student growth’ will be demonstrated
on the tests because, in fact, student growth will have
occurred.”
 “Formative
Assessment’s ‘Advocatable Moment’” by James
Popham in Education Week, Jan. 9, 2013 (Vol. 32, #15, p.
29)
Questions and Discussion