A Guide to Standard Setting
Large scale assessments such as the Oregon Statewide Assessments and the English
Language Proficiency Assessments are designed to provide information to educators,
stakeholders, and policymakers about what students know and can do in a particular
content area. Scores are reported using two conventions: scale scores and achievement
levels. Scale scores allow students to be compared to one another. Achievement levels
allow students to be compared against the expectations that Oregon has for student
achievement in regard to state standards.
To establish the achievement expectations Oregon has for students on statewide
assessments, Oregon educators and stakeholders participate in a standard setting. During
the standard setting, the participants recommend to the state the minimum scale score a
student will need in each grade and content area to be classified in each of Oregon’s
achievement levels. The minimum scale score needed to be classified in a achievement
levels is called a cut score. Cut scores demarcate one achievement level from another as
demonstrated in Figure 1. Before recommending cut scores to the state, Oregon standard
setting participants meet for three days and engage in structured conversations that
include consideration of the following: Oregon standards, the achievement levels, the
difficulty of the content as represented by the items on the test, and knowledge of Oregon
students.
Does Not Meet
Meets
Nearly Meets
Exceeds
Students
Students
Students
Students
Nearly Meets
Meets
Exceeds
Cut score
Cut score
Cut score
Figure 1. Oregon achievement levels separated by cut scores
Standard setting participants study the content on the test to make decisions about how
much content knowledge a student should know to enter a particular achievement level
using the Bookmark Standard Setting Procedure. The Bookmark Standard Setting
Procedure has been used to set achievement standards in 25 states. One outcome of the
standard setting process is the achievement level descriptor. Achievement level
descriptors describe what students know and can do based on findings from the test
content. These may be used by educators to target instruction. Using the established
achievement levels, the state may define an accountability model or demonstrate
academic progress over time.
Oregon Department of Education
Office of Assessment and Information Systems
January 2007
J/ASMT/Perf Standard Setting/Non-Technical Description