Update on Development of the
Smarter Balanced Common Assessment
February 14, 2012
• Welcome, the webinar will begin at 2:30 Pacific.
• Please mute your audio so that your personal
conversation is not heard by others.
• Please use “Chat” to ask that a question be
addressed.
2
•
•
•
•
Introduce Oregon staff engaged with SBAC
Welcome participants and guest, Tony Alpert
Quick Background (Kathleen Vanderwall)
Recent Questions impacted by state policy
(Derek Brown)
• Timeline, Evidence Centered Design, Process
for Answering Questions (Tony Alpert)
• Open Questions
3
1
Accessibility and Accommodations
2
Formative Assessment Practices and
Professional Learning
3
Item Development
4
Performance Tasks
5
Reporting
6
Technology Approach
7
Test Administration
8
Test Design
9
Transition to Common Core State
Standards
10
Validation and Psychometrics
Oregon Workgroup
Members Advising
SMARTER Balanced
4
• Dianna Carrizzales-Engelmann
Accommodations and Accessibility Co-Chair
• Mark Freed
Reporting Co-Chair
• Steve Slater
Validation and Psychometrics Co-Chair
• Kathleen Vanderwall
Item Development Member
• Rachel Aazzerah
Test Administration Member
• Jim Leigh
Transition to the Common Core Member
5
Participants
Tony Alpert
Chief Operating Officer
SMARTER Balanced Assessment Consortium (SBAC)
6
Quick Background
Kathleen Vanderwall
Manager, Test Design
Oregon Adopted CCSS
Superintendent of Public
Instruction Susan Castillo
“By joining other states in
the Common Core
movement, we better
position our students to
compete nationally and
internationally with their
peers for jobs in the 21st
century workplace.”
Oregon Board of
Education adopted
Mathematics and
ELA & Literacy
CCSS in October 2010
Operational Assessment
begins 2014-2015
www.ode.state.or.us/go/commoncore
Oregon is a governing member of SBAC
One Aspect of the
Summative Assessment
The SMARTER Balanced
Mandatory comprehensive
Assessment Consortium
accountability measures
(SBAC) is one of two
that include computer
multistate consortia
adaptive assessments and
awarded funding from the
performance tasks,
U.S. Department of
administered in the last 12
Education to develop an
weeks of the school year in
assessment system based
grades 3–8 and high school
on the new Common Core
for English Language Arts
State Standards (CCSS).
(ELA) and mathematics
http://www.k12.wa.us/SMARTER/
CCSS Implementation
Stewardship Team
CCSS implementation efforts
require a group of committed
individuals that represent all
sectors of education in order to
achieve the overarching Common
Core goal: every Oregon student
College and Career-ready. Called
the Stewardship Team, this
group consists of 39
stakeholders, identified through a
statewide nomination process,
and appointed by the
Superintendent of Public
Instruction.
Membership includes
• K-12 and postsecondary
educators in English language
arts, mathematics, science,
social science, and career and
technical education
• Early childhood educators,
• ESD staff
• Faculty from colleges of
education (teacher
preparation)
• Special education and English
Language Learner directors
• Representatives from
business, Oregon PTA, and
professional educational
organizations
Recent Questions
Impacted by State Policy
Derek Brown
Manager, Assessment of Essential Skills
• Does the question pertain to assessment?
SBAC is in the questioning period itself,
procuring contracts for test design,
communication etc.
• Does the policy question pertain to Oregon
specific requirement (such as the Oregon
Diploma)? ODE’s process includes: ODE Staff
Recommendation with supporting data, Review
by appropriate stakeholder group (i.e. AESRP),
Recommendation to State Board of Education
12
Q: Will SBAC replace OAKS?
A:Yes, we believe so. ODE will institute the SBAC
assessment to replace OAKS in reading, writing,
and math starting 2014-15.
Q: Does SBAC assessment become the ticket to HS
diploma?
A: Once ODE staff have the fundamental
information needed about this assessment then
AESRP will evaluate and recommend an
appropriate achievement level to SBE.
13
• What about 11th graders who take OAKS in
2013-2014 and don’t pass….will they or won’t
they have opportunity on SBAC as seniors
towards their diploma? We do not have an
answer at this time, we don’t yet know about
SBAC eligibility. Anticipate that ODE will
give equitable opportunity.
14
• What about 9th or 10th graders who already
passed OAKS in high school prior to taking
SBAC as 11th graders in 2014-15? No firm
policy yet, but we will bring this up with
stakeholder groups like AESRP and
Accountability Advisory
• Will their OAKS tests count towards diploma?
No policy yet, ODE will engage stakeholders.
We anticipate feedback that would
encourage ODE to write such policy.
15
Development Timeline
Tony Alpert
Chief Operating Officer, SBAC
www.smarterbalanced.org
17
Grades
3
9
11
12
Summative
Interim
(Optional)
Formative Tools and
Professional Learning
(Optional)
8
✔
✔
✔
10
1-2 Performance
Tasks as Required
to Cover CCSS
✔
✔
EOC and
Comprehensive
✔
✔
✔
EOC and
Comprehensive
✔
Optional
EOC and
Comprehensive
18
✔
SBAC is committed to using evidence-based
design in its development of the common
assessment:
• Claims are the broad statements of the
assessment system’s learning outcomes
• Each claim requires evidence that will support
interpretations of competence towards
achievement of all claims
• Relevant and sufficient evidence needs to be
collected in order to support each claim
19
Evidence-Centered Design
(from SBAC Item Spec
showcase #1)
20
Evidence-Centered Design
How to view Evidence-Centered Design
(ECD)?
 ECD is as a logical, systematic approach to test design
and development based on the principles of evidentiary
reasoning.
 ECD is not used as a rigid set of procedures.
 ECD gives us tools for linking all of the steps of test
development to the collection of the evidence required
to support the claims to be made about test takers.
21
Evidence-Centered Design
What does ECD do?
 ECD does not make the traditional test development
steps obsolete.
 ECD offers tools to help get the steps done.
 ECD helps clarify, document, and link the steps in test
development, but the steps still need to be done
whether ECD is used or not .
22
Evidence-Centered Design
Steps in ECD
• ECD starts with claims about the KSAs of test
takers—what SBAC wants to say about students on
the basis of assessment results.
• SBAC has provided high-level claims for
summative tests in ELA and Mathematics.
• ECD requires that evidence be provided to
support each claim.
• SBAC has provided descriptions of the evidence
needed to support the claims.
• The SBAC Content Specifications provide the
claims and descriptions of evidence needed as the
basis of ECD.
23
Evidence-Centered Design
Steps in ECD
 Each item/task (SR, CR, PT, TE) has to elicit evidence
for one or more claim(s).
 Item Specifications will set specifications for the tasks,
and rules for their accessibility and fairness. Each task
will be linked to evidence for a claim or claims.
24
Evidence-Centered Design
Interconnectedness of ECD Steps
 Earlier decisions affect all later phases.
 Any later phase could require changes in earlier
phases.
 Phases provide information for each other, both forward
and backward.
 Iteration and recycling throughout the phases is
expected and helpful.
25
Evidence-Centered Design
Benefits of ECD
 ECD strengthens validity argument.
 ECD aligns all parts of the test design and development
process.
 Every step of test development from setting
specifications to reporting scores is logically linked to
previous and following steps.
26
Theory Of Action
and CCSS
Item Piloting
Large Scale Item
Writing
Content
Specifications
Initial Item
Writing
Alignment and
Bias Review
Large Scale
Alignment and
Bias Review
Item and Test
Specifications
Rapid
Prototyping
Field Testing and
Scaling
27
...the SMARTER Balanced Assessment
Consortium launches a new website:
www.smarterbalanced.org
• Visitors will be able to:
• Download resources and new fact sheets
• Sign up for a monthly e-newsletter
• Explore an interactive timeline of activities by school year
• Provides frequent updates on activities, milestones, events
• Showcases the innovative work of the Consortium
28
• Submit questions to holly.carter@state.or.us
• Answering through review of
• SBAC’s RTTT Proposal to US DOE
• Contract
• Technical Advisory Committee
• Governing State Votes
and confirmation through SBAC.
29
Derek Brown, Manager of Assessment of Essential Skills
Kathleen Vanderwall, Manager of Test Design