The Smarter Balanced Assessment Consortium
Training 2 - Day 1
Portland, Oregon
December 4 & 5, 2013
The Digital Library Review Process Using the Quality Criteria

•
Introductions by State
•
Share one success from Training 1
•
Share one lesson learned from Training 1
Slide 2

Formative Assessment
Advisory Panel Developed
Quality Criteria
State Leadership Teams and
State Networks of Educators
Established, Involving Around
2,000 K-16 Educators Across the Country
SLT Training 1 and SNE
Training 1 Completed
Over 450 Resources
Submitted to the Digital
Library
Blueprints for Exemplar
Instructional Modules and
Assessment Literacy
Modules Blueprints Drafted
Matrices for the Exemplar
Instructional Modules for
ELA and Math and for the
Assessment Literacy
Modules Drafted
Digital Library Software
Versions 1.0 and 1.1
Released
Slide 3

Since Training 1 …
SLT Webpage
Updates
• Updated Cover
Profile Guide
• Tips for
Identifying
Copyrights
SLT Features in the Digital Library
• What can I view as an SLT member?
• The Individual
SNE Report
SNE Payment
Schedule
• Provides additional time for SNE Trainings
• Provides sufficient time for
SNE members to submit and review
Slide 4

• www.amplify.com/smarter-balanced-slt
•
Updated Copyright Functionality in the
Digital Library
•
Copyright Clearance Form
•
Updated Copyright Information in Cover
Profile Guide
•
Tips for Identifying the License Presentation
Slide 5

SLT Training Cycles
SNE Payments
SLT Training Dates
PCG Sends “SNE
Member Payment
Spreadsheet” to
SLT Lead
SLT Returns
Spreadsheet to PCG
SNE Payment
December 16, 2013 December 23, 2013 January 24, 2014
August –
September 2013
November -
December 2013
Training 3
March 2014
Training 4
May 2014
Training 5
August 2014
February 24, 2014
May 19, 2014
July 21, 2014
October 20, 2014
March 4, 2014
May 27, 2014
July 29, 2014
October 28, 2014
March 28, 2014
June 20, 2014
August 22, 2014
November 21, 2014
Slide 6

Short-Term
• SLT Training 2 in November and December 2013
• Software Version 1.15 in February 2014
Long-Term
Milestone
• SLT Trainings 3 (March 2014), 4 (May 2014), and 5 (August
2014)
• Software Version 1.2 in April 2014
• First set of 23 Exemplar Instructional Modules and
Assessment Literacy Modules in March 2014
• Second set of Modules in May 2014 and third set of
Modules by September 2014
• Digital Library Open to Consortium Members in
April 2014
Slide 7

Slide 8

Step 1:
Resource
Submitted
Resource Posting Work Flow
Step 2:
Gatekeeping
Criteria
Applied
Step 3:
Quality
Criteria
Applied
Cover
Profile
SNE
1
SNE
1
SNE
2
SNE
3
Slide 9

Digital Library Resources
Assessment
Literacy Modules
Exemplar
Instructional
Modules
Education
Resources
• Commissioned Professional Learning Modules
• Resources for educators, students and families
• Frame Formative Assessment within a Balanced Assessment System
• Articulate the Formative Assessment Process
• Highlight Formative Assessment Practices and Tools
• Commissioned Professional Learning Modules
• Instructional coaching for educators
• Instructional materials for students
• Demonstrate/support effective implementation of the formative assessment process
• Focus on key content and practice from the Common Core State
Standards for Mathematics and English Language Arts
• High-quality vetted instructional resources and tools for educators
• High-quality vetted resources and tools for students and families
• Reflect and support the formative assessment process
• Reflect and support the Common Core State Standards for Mathematics and English Language Arts
• Create Professional Learning Communities
Slide 10

Learning
Goal
• SLT members will understand the Quality
Criteria ratings and how to apply them consistently.
Success
Criterion
• SLT members will provide evidence of inter-rater reliability among the state SLT members and with other state SLT
members in cross-state teams.
Slide 11

Formative Assessment Advisory Panel
Convened the Formative Assessment Advisory Panel to develop the Quality Criteria during three two-day meetings
Meeting 1
April 17 – 18
• Brainstormed initial
Quality Criteria
Synthesized feedback
Meeting 2
May 8 – 9
Developed first draft of
Quality
Criteria
• Determined structure of
Quality Criteria
• Developed comprehensive list of potential criteria
• Discussed merits of checklist vs. rubric-based approach
Meeting 3
May 22 – 23
• Tested criteria using sample resources
• Refined criteria
Developed 2 nd draft; received feedback from panelists; developed present draft
Slide 12

Members of the Formative Assessment
Advisory Panel
Twelve experts in the CCSS for ELA, the CCSS for Mathematics, the formative assessment process, adult learning, online professional learning, diverse learners, and urban and rural education comprised the Formative
Assessment Advisory Panel.
1) Dr. Lynne Anderson-Inman (University of Oregon)
2) Dr. Robert Calfee (Stanford University, UC Riverside)
3) Dr. Bridget Dalton (University of Colorado)
4) Dr. Diane Heacox (St. Catherine University)
5) Dr. Joan Herman (UCLA – CRESST)
6) Dr. John Hill (Purdue University)
7) Dr. Yvette Jackson (National Urban Alliance for Effective Education)
8) Dr. Henry Kepner (University of Wisconsin - Milwaukee)
9) Dr. Katherine McKnight (National Louis University)
10) Valerie L. Mills (National Council of Supervisors of Mathematics)
11) Dr. James Popham (UCLA)
12) Dr. Lucinda Soltero-Gonzalez (University of Colorado-Boulder)
Slide 13

Components of the Quality Criteria
Cover Profile
Gatekeeping Criteria
Quality Criteriafor Professional Learning Resources
Quality Criteria for Instructional Resources
Governance Criteria
Slide 14

Purposes of the Quality Criteria
Ensure that the resources in the
Digital Library reflect the intent of the CCSS
Ensure that all resources reflect the Smarter
Balanced vision of effective formative assessment practices
Ensure that all resources in the Digital
Library are of the highest quality, regardless of source
Provide consistency in the review process for all resources
Slide 15

Step 1:
Resource
Submitted
Resource Posting Work Flow
Step 2:
Gatekeeping
Criteria
Applied
Step 3:
Quality
Criteria
Applied
Step 4:
Decision
Posted
Cover
Profile
SNE
1
SNE
1
Returned to Submitter
SNE
2
Sent to SLT
SNE
3
Slide 16

•
Individually Read and Annotate the Quality Criteria
– Pages 2-5: Quality Criteria for Professional Learning Resources
– Pages 6-9: Quality Criteria for Instructional Resources
•
Think About, Then Discuss with Your Table Group:
– What are the differences between the two sets of criteria?
– How might these criteria help focus the review process?
– Where will your state need more clarity and information for Training 2?
•
Whole Group Discussion
Slide 17

–
Quality Criteria FAQ
–
Univeral Design for Learning
–
Professional Learning Standards
Slide 18

Slide 19

Accessing the Digital Library in the Training Environment
1. Open your Internet browser
(Google Chrome typically works best, though Safari, Firefox, and Internet Explorer 9+ are also options).
2. Type the web address: http://training.smarterbalancedlibrary.org
3. Type your assigned email address: slt#@smarterbalancedlibrary.org
4. Type the password: password
5. Click “Log into the Digital Library”
Slide 20

•
15 minutes
– Read the Cover Profile
– Read the PDF Materials
•
Watch the Video (whole group)
•
Apply Gatekeeping Criteria as a State Team
– 15 minutes
•
Individually Rate the Resource using the
Quality Criteria – 20 minutes
•
30 minutes
– Discuss Ratings with Your State Team
– Determine Ratings (with Reasons), Comments,
Posting Recommendation, Rationale as a Team
Slide 21

•
On your poster paper:
– Resource Title
– Criteria Ratings
– Posting Rec
– Rationale
•
Discuss with the whole group:
– Talk us through your team’s process when discussing and deciding on the ratings.
Slide 22

1.
2.
3.
Slide 23

The Smarter Balanced Assessment Consortium
Training 2 - Day 2
Portland, Oregon
December 4 & 5, 2013
The Digital Library Review Process Using the Quality Criteria

Follow Up
Tickets Out the Door
•
SLT Training Site
•
Who do I call?
– Help Desk
– SLT Lead
– Chrys
•
Will submitter be identified with his/her resource?
•
Radio
•
Meaning of 1-5 in software
Slide 25

SLT Training
Dates
SNE Payments
PCG Sends “SNE
Member
Payment
Spreadsheet” to
SLT Lead
SLT Returns
Spreadsheet to
PCG
SNE Payment
August –
September
2013
November -
December
2013
December 16,
2013
February 24,
2014
Training 3
March 2014 May 19, 2014
Training 4
May 2014
Training 5
August 2014
July 21, 2014
October 20,
2014
December 23,
2013
March 4, 2014
October 28,
2014
January 24,
2014
March 28,
2014
May 27, 2014 June 20, 2014
July 29, 2014
August 22,
2014
November 21,
2014
SNE
Responsibilities
Submit 2 resources
-Post 1 resource
-Review 3 resources
-Post 1 resource
-Review 3 resources
-Post 1 resource
-Review 3 resources
-Post 1 resource
-Review 3 resources

How can we help SNE members feel more connected and maintain morale?
•
Send Thank You Cards
•
Highlight Successes
•
Recognizing SNEs with Principals/Providing
Certificates
•
Reassurance of completion and payment
•
Recognizing SNE First Approved Resource
•
Badges
Slide 27

SLT Training Dates
August – September
2013
November -December
2013
Training 3
March 2014
Training 4
May 2014
Training 5
August 2014
SLT Training Cycles
SNE Payments
SNE Payment SNE Responsibilities
January 24, 2014
March 28, 2014
June 20, 2014
August 22, 2014
November 21, 2014
Submit 2 resources
Post 1 resource
Review 3 resources
Post 1 resource
Review 3 resources
Post 1 resource
Review 3 resources
Post 1 resource
Review 3 resources
Slide 28

Accessing the Digital Library in the Training Environment
1. Open your Internet browser
(Google Chrome typically works best, though Safari, Firefox, and
Internet Explorer 9+ are also options).
2. Type the web address: http://training.smarterbalancedlibrary.org
3. Type your assigned email address: slt#@smarterbalancedlibrary.org
4. Type the password: password
5. Click “Log into the Digital Library”
6. Select (Combo Expl) Citing Evidence from
Informational and Literary Text in Discussion
Slide 29

Citing Evidence from Informational and
•
5 minutes
Literary Text in Discussion
– Review About this Resource
•
Watch video (whole group)
•
30 minutes
– Review pdf files
– Individually vet the resource using the Professional
Learning Quality Criteria
•
30 minutes
– Discuss and compare answers with your team
– Create your poster
Slide 30

•
On your poster paper:
– Resource Title
– Criteria Ratings and Rationale
– Posting Recommendation
– Rationale
•
Discuss with the whole group: What were the notable differences in reviewing an
Instructional Resource versus a Professional
Learning Resource?
Slide 31

Slide 32

SNE Recommendation Options

Do not recommend



Slide 33

20 SNE Recommendation Combinations
Please note: The decision depends on the combination of recommendations; the order of the recommendations does not matter.
Decision SNE Recommendation SNE Recommendation
Post with
Distinction
Post
SLT Member
Judgment
Recommend with Distinction Recommend with Distinction
Recommend with Distinction Recommend with Distinction
Recommend with Distinction Recommend
Recommend Recommend
Recommend with Distinction Recommend with Distinction
Recommend with Distinction Recommend with Distinction
Recommend with Distinction Recommend
Recommend with Distinction Recommend
Recommend
Recommend
Recommend
Recommend
Recommend with Distinction Recommend with Revisions
Recommend with Distinction Recommend with Revisions
Recommend with Distinction Do Not Recommend
Recommend Recommend with Revisions
Not Post; Return to submitter
Recommend
Recommend
Recommend with Revisions
Do Not Recommend
Recommend with Revisions
Recommend with Revisions
Recommend with Revisions
Do Not Recommend
Recommend with Revisions
Recommend with Revisions
Do Not Recommend
Do Not Recommend
SNE Recommendation
Software or
SLT
Decision?
Recommend with Distinction Software
Recommend
Recommend
Software
Software
Recommend
Recommend with Revisions
Do Not Recommend
Recommend with Revisions
Do Not Recommend
Recommend with Revisions
Do Not Recommend
Software
SLT
SLT
SLT
SLT
SLT
SLT
Recommend with Revisions
Do Not Recommend
Do Not Recommend
Recommend with Revisions
Do Not Recommend
Do Not Recommend
Recommend with Revisions
Do Not Recommend
Do Not Recommend
Do Not Recommend
Software
Software
Software
Software
Software
Software
Software
Software
Software
Software
Slide 34

Recommendation Combinations-
SLT Member Decision
An SLT member will make a posting decision given the following recommendation combinations:
SNE Recommendation SNE Recommendation SNE Recommendation
Recommend with
Distinction
Recommend with
Distinction
Recommend with
Distinction
Recommend with
Distinction
Recommend with
Distinction
Recommend with
Distinction
Recommend
Recommend
Recommend with
Revisions
Do Not Recommend
Recommend with
Revisions
Do Not Recommend
Recommend
Recommend
Recommend
Recommend
Recommend with
Revisions
Do Not Recommend
Slide 35

SLT Decision Scenario
SNE Recommendation SNE Recommendation SNE Recommendation
Recommend with
Distinction
Recommend with
Distinction
Recommend with
Distinction
Recommend with
Distinction
Recommend with
Distinction
Recommend with
Distinction
Recommend
Recommend
Recommend with
Revisions
Do Not Recommend
Recommend with
Revisions
Do Not Recommend
Recommend
Recommend
Recommend
Recommend
Recommend with
Revisions
Do Not Recommend
Slide 36

•
Log into the “real” Digital Library
•
Software Demo
•
Resource Review tab
Slide 37

•
Resource
•
Cover Profile
•
Three SNE Quality Criteria Reviews
– Recommend with Distinction
– Recommend
– Recommend with Revisions
•
SLT Arbitration Form
Slide 38

•
Find someone you haven’t spoken to from a different table
•
Share your table’s recommendation and rationale.
•
Whole Group Share
– What did you hear that was interesting?
– Any new thoughts?
Slide 39

Slide 40

https://www.surveymonkey.com/s/SLTTraining
2
Slide 41