PLYMOUTH STATE UNIVERSITY
SMARTER BALANCED ASSESSMENT
Scott J Mantie, Ph.D.
Bureau of Accountability & Assessment
New Hampshire Department of Education
December 4, 2014
The Context and Perspective
Why do we assess?
 For
many reasons, but the most important are:
1. Improve teaching and learning
2. Evaluate and improve programs
3. Support accountability uses
 In
serving all three of these purposes, assessments
should reflect intended learning targets, but can
also serve as an important signal to what should be
learned
Assessing for Deep Understanding



We want students to develop deep understanding of
important knowledge and skills and know if/how students
are acquiring those skills/knowledge.
A major component of Next Generation Learning is the
ability to solve novel problems—this requires deep
understanding!
Assessment conveys what’s important to learn (a signal)
as well as providing an opportunity to check on students’
understanding and evaluate achievement
Previous Assessment Status
With other, earlier assessments…
“The Remediation Gap”
High School
was here…
College/Career
was here…
Next-Generation Assessment Needs

Rigorous assessments of progress toward “college and
career readiness”

Common cut scores across all Consortium states

Information about grade-level achievement and growth


Valid, reliable, and fair for all students (students with
“significant cognitive disabilities" will be assessed using
another valid, reliable and fair assessment)
Administered online, using multiple measures (interim)
How we got here…

SBAC Underlying Principles:
SBAC learning and assessment system grounded in:
Theory of Action
Research-supported instructional practice (blueprints)
 Technology-supported tools
 Innovative assessments
 State-of-the-art classroom support mechanisms
 Have the broadest access for the greatest number of students

How we got here…

SBAC is Based on Research:

Demonstrating evidence – does the assessment collects the
information desired? This is an ongoing process

Using Educational-Assessment Best Practices & Materials:


Standards for Educational and Psychological Testing (AERA, 1999)

U.S. Department of Education’s Standards and Assessment Peer Review Guidance (2009b)

Program Evaluation Standards (Yarbrough, Shulha, Hopson & Caruthers, 2011)

Guiding Principles for Evaluators (American Evaluation Association, 2004)
Evidence-based to determine what can be done to improve
learning – this evolves operationally
How we got here…



SBAC is both Reliable & Valid

It was essential to have comprehensive and detailed
evidence to support the validity and reliability of SBAC
assessment.

Test development process seeks to make explicit the claims
that test users can make when interpreting student scores.
Detailed & Rigorous Framework
Constant Test Development & Test Evaluation processes
Adaptive Assessment
 Adjusts to student’s ability by basing difficulty of future questions on
previous answers
 More accurate measurement of student achievement – particularly for high
and low performing students
 Creates an individually tailored set of questions
 Tested (mathematically) in the Field-Test

CAT implemented 2001, Supporting Research:

A Framework for the Development of Computerized Adaptive Tests, Nathan A.
Thompson, Assessment Systems Corporation, and David J. Weiss, University
of Minnesota

The Road Ahead for State Assessments, Rennie Center for Education
Research & Policy, Policy Analysis for California Education (PACE)
Here is how the
Smarter Balanced system
fits into the school year
Summative Assessments
Performance
Tasks
Optional
interim
assessments
Optional
interim
assessments
Scope, sequence and
timing of interim
assessments locally
determined
Computer-adaptive
accurate, aligned to
Common Core and built to
promote great instruction
Computer
Adaptive
Assessment
Student results
Fast, detailed
and digital with
year-on-year
measurement.
Teacher Development / NH Networks (Formative)
Instruction
School year
10
Assessments in New Hampshire
 Assessment in New Hampshire





SBAC: state assessment for mathematics and English language arts
NECAP Science: state assessment for science
NH ALPs Science: alternate assessment for science
ELA/Math Alternate Assessment: DLM (Dynamic Learning Maps)
Schools piloting 11th grade SAT: USED – no approval yet
 Data Privacy:
 NO student identifiable information collected!
NH Privacy & Security of Student Data FAQs\faq_privacy.pdf
 NO non-academic questions are asked!
 On-line, Adaptive Assessment vs. Paper/Pencil
 We want to take advantage of the engaging aspects that adaptive, on-line tests can
have for students
Emphasis: Removing the test as being the focus of the assessment
Online vs. Paper/Pencil
• Adaptive (online) adjusts to student vs. non-adaptive (p/p) fixed
• Online – greater security (p/p lost/misplaced paper, changes numerous hands,
etc.)
• Online – faster results (p/p must wait until all test results are in)
• Online – Performance Tasks (p/p no performance tasks)
• Online – more usability, accessibility and accommodations (p/p fewer)
• Language supports (glossaries in 10 languages & dialects, full Spanish
translations of math assessment)
• Deaf or hard of hearing receive tests in American Sign Language, signed
by recorded human interpreters
• Refreshable Braille keyboards and real-time embossers allow blind
students to receive online tests in Braille
• Accommodations uploaded (IEP & 504)
Online – will assess a broader range of students because of the many adaptive and
accessibility features
NH Assessment: Important Points
 SBAC Operational Vendor Selected






In conjunction with NEAC States (CT & VT)
Approval through G&C in December
Training, documentation and portion of reporting
Interim Assessments (2-12) & Digital Library ARE included
Digital Library operational December, 2014
Transition plan from previous to new SBAC accommodations
 Practice & Training Tests, Scoring Guides, Documentation
See: http://sbac.portal.airast.org/
Emphasis: Removing the test as being the focus of the assessment
NH Assessment: Important Points, II
 SBAC Usability, Accessibility and Accommodations (UAA)






NECAP to SBAC Crosswalk Draft completed (…last night)
“Transitioning from current usability/accommodations to SBAC
usability/accommodations” Planning document
UAA Implementation Guide – baseline
UAA “Frequently Asked Questions” – baseline
Alt assessment (DLM) & SBAC score integration – vendor
Alt Assessment Stakeholders meeting – SBAC or DLM decision tree
(12/4/14 meeting date)
UAA Documents\SmarterBalanced_Guidelines_091113[1].pdf
UAA Documents\Smarter Balanced Usability Accessibility and
Accommodations FAQs[1].pdf
NH Assessment: Important Points, III
 SBAC Digital Library Rollout




Week of 12/1/14: Send/acquire SBAC contact for each district
Week of 12/8/14: Provide access to SBAC Digital Library to district
contacts and direct them to SBAC-provided Digital Library
manuals/information
Week of 12/15/14: Provide documentation/information to district SBAC
district contacts related to how to sign-up SBAC Digital Library users (i.e.
educators/teachers)
https://www.smarterbalancedlibrary.org/digital-library-resources
https://www.smarterbalancedlibrary.org/digital-library-resources
NH SBAC Testing Window
 Window can begin:
 66% instructional days: Grades 3-8
 80% instructional days: Grade 11
 Where other States are at regarding 12 Week Test Window: Variable
 Testing Window for NH:





Grades 3&4, weeks 1-6
Grades 5&6, weeks 3-8
Grades 7,8,11, weeks 5-12
Makeups during same grade-band weeks
Idea: Create a “testing schedule for your school?
Smarter Balanced: Interim Details

Interim to Provide valid, reliable and fair information about:

May be used for K through 12 grade levels (aligned to CCSS 3-8 & 11)

Student progress toward mastery of skills measured by summative assessment

Performance at content cluster level so teachers and administrators can track student
progress throughout the year and adjust instruction accordingly

Individual/group (e.g., school/district) performance at the claim level to determine
whether teaching and learning are on target

Teacher-moderated scoring of performance events as a professional development
vehicle to enhance teacher capacity to evaluate student work aligned to standards

Student progress toward the mastery of skills across all students and subgroups of
students
Smarter Balanced: Digital Library Details

Digital Library (AKA: Formative Assessment Resources) to Provide
measurement tools and resources to:

Improving teaching and learning

Monitor student progress throughout the school year

Help teachers and other educators align instruction, curricula and assessment

Help teachers and other educators use the Summative, Performance and Interim
assessments to improve instruction at the individual and classroom levels

Illustrate how teachers and other educators can use assessment data to engage students
in monitoring their own learning
This will be made available to all NH teachers
SBAC Checklist:
Logical, best-practices design…?
 Asking non-academic questions…?
 Student privacy…?
 Validity & Reliability of assessment…?
 Accessibility to more students…?
 Timeliness of scores…?

The Test: What We’ve Learned

Access SBAC Practice & Training Tests:



GOTO: http://sbac.portal.airast.org/
http://sbac.portal.airast.org/
THEN, GOTO: “Resources & Documentation”
 “Practice and Training Test User Guide”
 “Classroom Activity Administration Guidelines”
 “Scoring Guides”
 “Performance Task Writing Rubrics”
OR…
GOTO: “Student Interface Practice and Training Tests”
Problem??? : Older technology or software (like my State computer…)
“Download Secure Browser”
PRACTICE/TRAIN USING THE PRACTICE/TRAINING TESTS!
The Test: What We’ve Learned, II

One district’s plan…
High Leverage Strategies
1.
Develop student stamina to persevere when presented with reading, writing
and problem-solving
2.
Teach skills of synthesis with multiple sources (short stories, books, poems,
online articles, Scholastic News articles, etc.)
3.
Regular practice on technology devices (Netbooks, laptops, etc.) in order to
explicitly teach typing and technology manipulation skills
4.
Teach students how to effectively respond to DOK 3 and 4 questions with
DOK 3 and 4 answers
5.
Increase opportunities for students to improve typing skills
Communication Planning –
Assessment Transition


Stakeholder Advisory Group
 Assisting with the development of a communications plan around
standards, competencies and assessments
Preparing communication resource documents



FAQs; talking points; parent resources
Organizing and redesigning public information on the NH DOE
website
Collecting and redesigning educator resources on the NH Network
Visit: SmarterBalanced.org
for current information
23
For more information…
603-271-3844
Please contact Dr. Scott Mantie if you have any questions!
Scott.Mantie@doe.nh.gov
603-271-3844
Information & Practice/Training Test
Resources:
http://sbac.portal.airast.org/