The Smarter Balanced Assessment
Consortium
Using the Individual Contributor and
Reviewer Report
Agenda
•
•
•
Log into the Digital Library and navigate
to the report.
Select an SNE member and run a report.
Understand the results displayed in each
section of the report.
Page 2
www.smarterbalancedlibrary.org
Page 3
Logging into the Digital Library
4
Navigating to the Individual Contributor
and Reviewer Report
5
Running a Report – Default Settings
6
Running a Report – Choose a Time
Period
7
Running a Report for SNE Gabe Kotter
8
Three Sections – SNE Profile Information
9
Three Sections – Contributed Resources
2
0
Mathematics of Money
Formative Assessment in Mathematics
10
Three Sections – Resource Reviews
11
Gabe’s SNE Profile
12
Contributed Resources
2
0
Mathematics
of1Money
Test Resource
Formative Assessment in Mathematics
Page 13
Understanding Contribution Information
2
0
The Resources Contributed metric displays the number of resources that Gabe
has submitted for review.
Page 14
Understanding Contribution Information
2
0
The Contributed Resources Posted metric displays the number of resources that
have passed the Quality Criteria review process and are currently posted to the
Digital Library.
Page 15
Understanding Contribution Information
2
0
The Contributed Resources Posted with Distinction metric displays the number of
resources that have passed the Quality Criteria review process and are currently
posted to the Digital Library with distinction.
Page 16
Understanding Contribution Information
2
0
Mathematics of Money
Formative Assessment in Mathematics
Page 17
Post Status
Submitted
The resource is awaiting review.
In Review
The resource is currently being reviewed.
Posted
The resource has been successfully posted to
the Digital Library.
Posted with Distinction
The resource has been successfully posted to
the Digital Library with distinction.
Returned
The resource has been returned to the SNE
for revisions.
Page 18
Understanding Contribution Information
2
0
Mathematics of Money
Formative Assessment in Mathematics
Page 19
Resource Reviews Tables
1 (25%)
3 (75%)
3
44%
Page 20
Understanding Review Information
3 (75%)
1 (25%)
Page 21
Understanding Review Information
3 (75%)
1 (25%)
The Accepted Reviews column displays the number of resources that Gabe passed
through the Gate-Keeping review, indicating they met all these criteria.
Page 22
Understanding Review Information
3 (75%)
1 (25%)
The Returned Reviews column displays the number of resources that Gabe did not
pass through the Gate-Keeping review, indicating they did not meet all these
criteria.
Page 23
Understanding Review Information
3 (75%)
1 (25%)
The Total Reviews column displays a sum of the Accepted Reviews and Returned
Reviews.
Page 24
Understanding Review Information
3 (75%)
1 (25%)
Page 25
Understanding Review Information
Accepted Reviews
Returned Reviews
Total Reviews
Increases by 1
Increases by 1
• SNE completes
Gate-Keeping
review
• Returns the
resource to the
contributor
• SNE completes
Gate-Keeping
review
• Accepts the
resource and
moves it
forward
Increases by 1
Increases by 1
Page 26
Understanding Review Information
3 (75%)
1 (25%)
Page 27
Understanding Reviewer Information
1 (25%)
3 (75%)
3
44%
Page 28
Understanding Reviewer Information
3
44%
The Reviews in Progress column displays the number of resources for which
Gabe is currently conducting Quality Criteria Reviews.
Page 29
Understanding Reviewer Information
3
44%
The Completed Reviews column displays the number of resources for which
Gabe has completed Quality Criteria Reviews.
Page 30
Understanding Reviewer Information
3
44%
The Average Consistency Rate for All Completed Reviews column shows the
average recommendation consistency rate for all resources reviewed in the
selected time period.
Page 31
Understanding Reviewer Information
Reviews in Progress
SNE starts a new Quality
Criteria review
Increases by 1
SNE cancels that same
Quality Criteria review
Decreases by 1
SNE completes that
same Quality Criteria
review and makes a
recommendation
Decreases by 1
Page 32
Completed Reviews
Increases by 1
Understanding Reviewer Information
3
44%
Page 33
Understanding Reviewer Information
Page 34
Understanding Reviewer Information
R+ (Recommend with Distinction)
R (Recommend)
R- (Recommend with Revisions)
NR (Do Not Recommend)
Page 35
Understanding Reviewer Information
Page 36
Consistency Rate Calculation Example
Tony
Maria
Gabe
Do Not Recommend
(NR)
Recommend with
Revisions (R-)
Recommend (R)
Recommend with
Distinction (R+)
Consistency Rate Calculation Example
Tony
Maria
Gabe
Do Not Recommend
(NR)
Recommend with
Revisions (R-)
Recommend (R)
Recommend with
Distinction (R+)
NR = 1
R- = 2
R=3
R+=4
Numerical Values
Do Not Recommend (NR) = 1
Recommend with Revisions (R-) = 2
Recommend (R) = 3
Recommend with Distinction (R+) = 4
Consistency Rate Calculation Example
Difference of 2
Difference of 1
Tony
Maria
Gabe
Do Not Recommend
(NR)
Recommend with
Revisions (R-)
Recommend (R)
Recommend with
Distinction (R+)
NR = 1
R- = 2
R=3
R+=4
Numerical Values
Do Not Recommend (NR) = 1
Recommend with Revisions (R-) = 2
Recommend (R) = 3
Recommend with Distinction (R+) = 4
Consistency Rate Calculation Example
Difference of 2
Difference of 1
Tony
Maria
Gabe
1
2
3
4
Do Not Recommend
(NR)
Recommend with
Revisions (R-)
Recommend (R)
Recommend with
Distinction (R+)
Reviewer
Rating
Divergence from
Gabe
Divergence from
Tony
Divergence from
Maria
Total Divergence from
Other Reviewers
R=3
0
2
1
3
Reviewer B
NR = 1
2
0
1
3
Reviewer C
R- = 2
1
1
0
2
Gabe
Consistency Rate Calculation
100% –
(
Total Divergence from Other Reviewers
Maximum Possible Divergence from Other Reviewers
)
Consistency Rate Calculation
Reviewer
Rating
Divergence from
Gabe
Divergence from
Reviewer B
Divergence from
Reviewer C
Total Divergence from
Other Reviewers
R (3)
0
2
1
3
Reviewer B
NR (1)
2
0
1
3
Reviewer C
R- (2)
1
1
0
2
Gabe
100% –
(
Total Divergence from Other Reviewers = 3
Maximum Possible Divergence from Other Reviewers = 6
)
= 50%
Consistency Rate
Consistency
Explanation
Rate
100%
The review is highly aligned with the other
two SNE reviewers.
83%
The review is somewhat aligned with the
67%
other two SNE reviewers.
50%
The review is not very aligned with the other
33%
two SNE reviewers.
17%
The review is not at all aligned with the
0%
other two SNE reviewers.
Page 43
Understanding Reviewer Information
Page 44
Understanding Reviewer Information
Page 45
Understanding Reviewer Information
Page 46
Understanding Reviewer Information
Page 47
Looking at SNE Reviewers’ Comments
Page 48
Looking at SNE Reviewers’ Comments
Page 49
Looking at SNE Reviewers’ Comments
Page 50
Looking at SNE Reviewers’ Comments
Page 51
Reflecting on the Learning
•
•
•
Log into the Digital Library and navigate
to the report.
Select an SNE member and run a report.
Understand the results displayed in each
section of the report.
Page 52
Congratulations!
You have completed the tutorial for
using the Individual Contributor and
Reviewer Reports.
Please contact the Help Desk, if you have any questions.
Monday-Friday
7:00am-7:00pm (EST)
855-657-8587 (toll free)
SmarterBalancedLibrary@amplify.com
Page 53