Smarter Balanced Assessment Consortium
Judy Hickman
Director of Mathematics
February 2015
Topics
•
•
•
•
2014 Field Test
What’s New?
– Updated Equation Editor tool/keypad
– Hand-scoring Guide
– Interim assessment packages released
Achievement Level Setting
Questions
1
Field Test 2014
2
What Do You Mean by “Accessibility Features”?
Example Stacked Translation
6
3
American Sign Language
Choice of Supports – Want to Explore?
•
•
•
•
Practice test
Training test
http://sbac.portal.airast.org/practicetest/
Updated periodically with new
features
http://www.smarterbalanced.org/wordpress/wpcontent/uploads/2014/08/SmarterBalanced_Guidelines.pdf
4
Common Embedded Support
Some of the Activities Since Field Testing…
•
Hand-scoring of 7,331,948 student responses
and 1,850 items/prompts
•
•
•
Rubric validation, item calibration, data review
•
Release of Interim Comprehensive
Assessments (ICA) and Interim Assessment
Blocks (IAB)
•
Digital Library available to members
Achievement Level Setting
Review and Revision of Test Item and
Performance Task Specifications (ongoing)
5
What’s New?
Improved Equation/Numeric Keypad
http://www.smarterbalanced.org/practice-test/
6
Improved Functionality
•
At a meeting of the Smarter Balanced
Mathematics Reasoning Project, CEO of
Desmos Inc. presented the usability research
that informed the improvements to its online
graphing calculator (www.desmos.com)
•
Smarter Balanced has used this research to
collaborate with American Institutes for
Research and Desmos to improve the user
experience for students for operational testing
in 2015
Nice Tutorial Available
7
New Publication – Hand-scoring Guide
http://www.smarterbalanced.org/wordpress/wp‐content/uploads/2014/10/Smarter‐
Balanced‐Scoring‐Guide‐for‐Selected‐Short‐Text‐Mathematics‐Items.pdf
New Publication – Hand-scoring Guide
8
New Publication – Hand-scoring Guide
Smarter Balanced Interim Assessments
•
Interim Items - indistinguishable from
summative items
– all items were developed together using the
same specifications and protocols for ensuring
quality
– all assess Common Core State Standards
– use Universal Design principles
– provide same accessibility resources
– provide evidence to support claims in
mathematics and English language arts/literacy
18
9
Two Types of Interim Assessments
•
Interim Comprehensive Assessment (ICA)
─ similar to the Summative Assessment in scope and
format. They can be used to identify areas of strength or
need based on the Common Core State Standards, and
provides teachers with information regarding student
readiness.
•
Interim Assessment Blocks (IAB)
─ focused sets of items that measure several assessment
targets. Results provide teachers with information about
a student’s understanding of the topics or standards
measured in the IAB.
Summative versus Interim
•
Summative Assessments are implemented on
standardized and secure conditions.
•
Interim assessments may be implemented
with much greater flexibility.
•
Interim Assessments are not secure and they
are not public.
10
Achievement Level Setting
Approach
•
•
•
•
•
Collaborative and transparent
Inclusive and consensus-driven process
Scientifically rigorous design
Using best practices
Multiple reviews, technical certification
https://www.youtube.com/watch?v=bW_yGf4BB1E
11
Major Activities
•
Online Panel
– Large-scale and inclusive – thousands of
participants
– Recommendations broken down by
subgroup (teachers, higher education, etc.)
•
In-Person Panel
– 500 participants in grade/subject panels
– Nominated by states
– Bookmark procedure
•
Cross-Grade Review
– 60 panelists from In-Person Panel
– Review Online Panel and In-Person Panel
results
– Recommend changes
Achievement Level Setting Timeline
April 30
August 18‐21
October 6-17
October 13-19
• States Approve Achievement Level Setting
Plan
•
Pilot of ALS at Measurement Incorporated
• Online Panel open to thousands of educators
and other constituents
• In-Person Panel
October 20
• Cross-Grade Review: Subgroup of In-Person
October 30
• Technical Advisory Committee Review
November 12
• State Chiefs Vote to Endorse Achievement
Level Recommendations
12
Additional Quality Controls
•
•
•
•
Engagement of TAC
– TAC reviewed ALS procedures throughout approval
process by Smarter Balanced
Independent Audit
– MI engaged services of an audit team to observe and
document proceedings, reported results to TAC
State Observers
– States invited to send an observer to In-Person and
Cross-Grade events
Revisiting of Cut Scores in 2015
– Measurement Incorporated contract includes
recommending a process to revisit cut scores in 2015
Did you know over 1000 math items can be
found in the 2/4/14 Math Specifications?
http://www.smarterbalanced.org/smarter-balanced-assessments
•
Supporting Document: Scoring Reporting and Estimated Testing
Times
•
•
•
•
•
•
•
Mathematics Content Specifications
Mathematics Grades 3-5 (ZIP) (Update 2/4/14)
Mathematics Grades 6-8 (ZIP) (Update 2/4/14)
Mathematics High School (ZIP) (Update 2/4/14)
Mathematics PT Item Specs All Grades (PDF) (5/27/14)
Scoring Guide for Selected Short-Text Mathematics Items
http://sbac.portal.airast.org/practice-test/
13
Questions?
14