Building Effective Assessments
Agenda
 Brief overview of
Assess2Know content development
 Assessment building pre-planning
 Cognitive factors
 Building the assessment
Building Effective
Assessments Using
Formative Assessment Content
Assess2Know
 Reading grades 2 – 11
 Mathematics grades 2 – 11
 Science grades 3 – 11
NEW
 Social Studies grades 9 – 11
Formative Assessment Content
High Quality Content
 Rigorous development process
 Tight alignment with Michigan State Standards
 Highly trained and experienced assessment
development staff
Formative Assessment Content
Rigorous Requirements
 Clearly and simply written; no “trick questions”
 Appropriate vocabulary for the grade level being tested
 Functions independently; does not provide information
that may be used to answer other items
 Requires students to read the passage in order to
answer the question
 Does not use “all of the above” or “none of the above”
 Presents answer options in a logical order
Formative Assessment Content
Reading
 Passages are commissioned works by wellknown authors
 Lexiles and Flesch-Kincaid
 Genres
 Bloom’s Taxonomy, Cognitive Difficulty Levels
 Maps, Charts, Graphs, Images, and Primary
Sources
Flesch-Kincaid Formula
FKRS = (0.39 x ASL) + (11.8 x ASW) 15.59
• FKRS: Flesch-Kincaid Readability Score
• ASL: average sentence length in words or
average number of words in sentence (number
of words divided by the number of sentences)
• ASW :average number of syllables per word
(the number of syllables divided by the number
of words)
Assess2Know Lexiles
• Lexile measures are considered to be
the most accurate measure of
readability because they are the only
formula which applies a psychometric
formula to the assignation for
establishing readability.
• Since 2005, Metametrics assigns each
A2K passage a Certified Lexile
Measure.
Assess2Know Reading
Items
• To determine grade level placement of passages,
Riverside uses readability guidelines to place
passages at each grade level.
• In addition, passage structure, interest level, topic,
and word count are among some of the things used
to evaluate where a passage should be placed for
grade level benchmarking.
• District level committees review passages and items
and if they had felt a passage was not appropriate,
Riverside replaces it and the items at the review.
– Advantage to onsite committee reviews because educators
in the field are affirming what is appropriate for each
benchmark.
Assess2Know Readability Limits
Grade
Flesh-Kincaid
Lexile
minimum
maximum
minimum
maximum
3
2.0
4.0
300L
850L
4
3.0
5.0
500L
950L
5
4.0
6.0
650L
1050L
6
5.0
7.0
750L
1075L
7
6.0
8.0
850L
1100L
8
7.0
9.0
950L
1150L
9
8.0
10.0
1000L
1200L
10
9.0
11.0
1050L
1300L
11
10.0
12.0
1100L
1300L
12
11.0
13.0
1100L
1300L
Formative Assessment Content
 Mathematics, Science, and Social Studies
 Real world context
 Unique
 Scenario Based Science Items
 Bloom’s Taxonomy, Difficulty levels
 Use of Tools
 Maps, Charts, Graphs, Images, and Primary
Sources
Research Efforts
• The Research and Measurement Services
Department analyzes data from multiple
users of the Assess2Know Benchmark item
bank from across the country to monitor the
performance of the items in the bank.
• Goal is to ensure that the item bank provides
users with strong quality items with a range of
difficulty.
• Results look at average ranges for statistics
such as p-values, discrimination statistics,
and point-biserials for each item.
• This information is used to plan for refresh
and future development of our item banks.
High Quality
Assessment
Why do we give assessments?
Importance of Designing High
Quality Assessments
• Assessments measure what is being
taught in the classroom
• Align to the district curriculum
– Curriculum Maps
– Scope and Sequence
• Results of the assessment are used to
report on student achievement
– Used to make decisions about student
growth, interventions and program
placement
Characteristics of High Quality
Assessments
• Design curriculum based assessments that
measure student achievement to the state
standards
• Report assessment results at the individual
student, class, school and district level
• Provide achievement information that can be
compared from year to year
• Construct assessments that are valid and
reliable
Characteristics of High Quality
Assessments
• Assessment design allows for a variety
of assessment types: selected
response, open ended items,
constructed response and rubrics
• Delivered in a timely manner
• Easy to administer
Purpose of Assessment
• Evaluate students overall
achievement and growth in a
content domain
• Diagnose students strengths and
weaknesses in and across content
domain
• Plan educational interventions and
to design individual instruction
Purpose of Assessment
• Place students in appropriate
educational program
• Select applicants into programs with
limited enrollment
• Certify individual achievement or
qualifications from Standards for
Educational and Psychological Testing
What Do We Ask Ourselves Before
Building an Assessment?
• What do we want to measure?
• What information do we want to
gather?
• How long do we want the
assessment to take?
• How will we determine which
standards we want to assess?
• How will we use the results?
Guiding Principles to Keep in Mind
When Building Formative
Assessments
• A minimum of three items for every
standard you want to measure.
• Spread items measuring an individual
standard across the assessment.
• Range of overall assessment difficulty
and for each standard measured.
• Begin assessment with easier items,
build to more difficult and conclude with
easier items.
Guiding Principles to Keep in Mind
When Building Formative
Assessments
• How many passages will be used for
reading assessments?
• What item types will be used?
• How many items per passage?
• Total number of items? Can this be
completed in the allotted time?
Guiding Principles to Keep in Mind
When Building Formative
Assessments
• Spread items with graphics across test.
• Readability levels for reading passages;
genres needed.
• A minimum of five items per passage.
• Even spread of answer choices across
assessment when possible; spread them out
across the form.
• Check items for appropriateness to use
online, paper, or both.
• Check for clueing
Formative Assessment Content
Cognitive Difficulty Levels
Level 1 Basic Skills
 recall of information such as fact, definition, term, or simple
one-step procedure.
Level 2 Conceptual Understanding
 includes the engagement of some cognitive processing
beyond recalling or reproducing a response. A conceptual
understanding item requires students to make some
decisions as to how to approach the problem or activity and
may imply more than a single step.
Level 3 Extended Reasoning
 requires problem solving, planning, and/or using evidence.
Items require students to develop a strategy to connect and
relate ideas in order to solve the problem while using
multiple steps and drawing upon a variety of skills.
Importance of Content
Complexity
•
•
•
•
•
•
Vastness of Content
Alignment
Validity
Clarity
Teacher Guidance
Truth in Advertising
Level One: Basic Skills
Recall of information such as fact, definition,
term, or simple one-step procedure.
•
•
•
•
•
Support ideas by reference to details in text
Use dictionary to find meaning
Identify figurative language in passage
Solve a one step word problem
Perform a specified procedure
Level 2: Conceptual
Understanding
Includes the engagement of some cognitive
processing beyond recalling or reproducing a
response. A conceptual understanding item
requires students to make some decisions as
to how to approach the problem or activity
and may imply more than a single step.
• Predict logical outcome
• Identify and summarize main points
• Represent a situation mathematically in more
than one way.
• Interpret a visual representation
Level 3: Extended Reasoning
Requires problem solving, planning, and/or
using evidence. Items require students to
develop a strategy to connect and relate ideas
in order to solve the problem while using
multiple steps and drawing upon a variety of
skills.
• Determine effect of author’s purpose on text
elements
• Summarize information from multiple sources
• Provide a mathematical justification
• Describe, compare and contrast solution
methods
Building Effective
Assessments
Questions