California Assessment of Student
Performance and Progress (CAASPP)
Introduction to the
2015 CAASPP Online Reporting
System (ORS)
1 – 2 p.m.
May 4, 2015
Purpose
•
To provide an introduction to the Online Reporting
System (ORS), which is the reporting system local
educational agencies (LEA) will use to view preliminary
student results on the following CAASPP assessments:
–
–
–
–
–
Smarter Balanced Summative Assessments in English
language arts/literacy (ELA) and mathematics
California Standards Test (CST) for Science
California Modified Assessment (CMA) for Science
California Alternate Performance Assessment (CAPA) for
Science
Standards-based Tests for Spanish (STS)
Introduction to the 2015 CAASPP Online Reporting System
2
Agenda
•
•
•
•
•
•
•
Overview
Accessing the ORS
Understanding the ORS Interface
Viewing Score Reports in the ORS
Live Demonstration
Questions
Updates and Announcements
Introduction to the 2015 CAASPP Online Reporting System
3
Overview
What is the
Online Reporting System?
The ORS is a web-based system that
displays preliminary score reports and
participation data for each student who
takes part in the CAASPP assessments.
Introduction to the 2015 CAASPP Online Reporting System
5
Why Online Reporting?
• Timely data is more useful for affecting learning
and instruction.
• Move to computer-based assessments
facilitates faster reporting.
• Policymakers made timeliness a priority
(Education Code 60641(b)).
Introduction to the 2015 CAASPP Online Reporting System
6
Important Reminder
The data available in the CAASPP ORS represents partial
and preliminary results that are not appropriate for public
release. As a real-time system, results will change as
additional data is received and relevant appeals and
rescores are processed. These changes may result in final
scores being higher or lower than the preliminary results
posted to this system. The California Department of
Education (CDE) recommends that data from the ORS
only be released publically following the state level release
of assessment data that occurs in August.
Introduction to the 2015 CAASPP Online Reporting System
7
Process for Release of
Preliminary Test Results
• Hand score
• Machine score
• AI Score
Score
Check
• Confirm
accuracy by
grade and
content area
• Preliminary
scores available
in the ORS
Report
Introduction to the 2015 CAASPP Online Reporting System
8
Reporting Timeline for Online
Smarter Balanced Summative Tests
Week 0:
Student completes a
content area
Weeks 1-3:
Student responses
are scored and
merged; preliminary
results are checked
Week 4:
LEA accesses ORS
to view preliminary
results
Introduction to the 2015 CAASPP Online Reporting System
9
Why Preliminary Results?
• Test administrations may still be in progress
• Factors that could change results
– Security breaches
– Problematic test questions
– Appeals processed and test rescores
Introduction to the 2015 CAASPP Online Reporting System
10
Preliminary Test Results
•
Provides information to support teachers in:
– Focusing instruction
– Improving learning
•
Preliminary results include:
– Student and aggregate results
– Overall scores reported within one of four
achievement levels Achievement Levels
LEVEL
4
Exceeded the Standard
LEVEL
3
Met the Standard
LEVEL
2
Nearly Met the Standard
LEVEL
1
Has Not Met the Standard
Introduction to the 2015 CAASPP Online Reporting System
11
Estimated Timeline for Release
of Preliminary Results
Grade and Content Area
Estimated Date
Smarter Balanced Grade 3 mathematics
Smarter Balanced Grade 5 mathematics
Smarter Balanced Grade 7 ELA
May 3 (today) for students
completed testing on or
around April 1
Smarter Balanced Grade 6 ELA and mathematics
Smarter Balanced Grade 7 mathematics
Smarter Balanced Grade 8 ELA and mathematics
TBD
Smarter Balanced Grade 3 ELA
Smarter Balanced Grade 4 ELA
Smarter Balanced Grade 5 ELA
TBD
Smarter Balanced Grade 4 mathematics
Smarter Balanced Grade 11 ELA and mathematics
TBD
CST/CMA/CAPA Science Grades 5, 8 and 10
STS RLA Grades 2-11
TBD
Introduction to the 2015 CAASPP Online Reporting System
12
Reporting Timeline for
Final Results
Week 0:
Student completes a
content area
Weeks 1-3:
Student responses
are scored and
merged; preliminary
results are checked
Week 4:
LEA accesses ORS
to view preliminary
results
4 Weeks After Test
Administration
Window Closes:
LEAs accesses and
downloads final
student data file
from TOMS
Introduction to the 2015 CAASPP Online Reporting System
13
Examples of Recommended Uses
• Use preliminary overall scores and claim
information to focus instruction
• Combine with other academic information about
the student to identify learning opportunities
Introduction to the 2015 CAASPP Online Reporting System
14
Not Recommended
• Adding these results to your local student
information system
• Presenting these results as final scores
• Using these results as the sole basis for
determining a student’s learning opportunities
• Comparing these results to last year’s STAR
results
Introduction to the 2015 CAASPP Online Reporting System
15
Accessing the ORS
User Roles and Access in the ORS
•
ORS operates in a secure, role-based structure
•
Access to reports and data in ORS depends on
your user role in the Test Operations Management
System (TOMS)
–
Example: LEA-level users can view data for all schools
and students in their LEA; school-level users can view
data only for students in their school.
Introduction to the 2015 CAASPP Online Reporting System
17
User Roles and Access in the ORS
(cont.)
Access Level
School Listing
Student Listing
Preliminary Student
Results Report
Summary Statistics
Retrieve Student Results
Plan and Manage Testing
Test Completion Rates
Search Students
LEA CAASPP
Coordinator
Test Site
Coordinator
(SC)















Test
Administrator
(TA)


Introduction to the 2015 CAASPP Online Reporting System
18
Accessing the ORS
• Note: To log onto ORS, users must have an
authorized TOMS username and password.
1. Select the [Online Reporting System] button
on the CAASPP portal at http://caaspp.org.
2. Log onto ORS using your
TOMS logon credentials.
Introduction to the 2015 CAASPP Online Reporting System
19
Accessing the ORS
3. Upon logging on, the Welcome page will
appear.
Introduction to the 2015 CAASPP Online Reporting System
20
Accessing the ORS
•
•
Note: ORS contains students’ personally
identifiable information (PII). For security purposes,
it is important that you log out to ensure that the
reports cannot be accessed by unauthorized users
following your session.
ORS has a timeout feature that automatically logs
you out if you have been inactive for 30 minutes.
Introduction to the 2015 CAASPP Online Reporting System
21
Understanding the ORS
Interface
Understanding the ORS Interface
• Two primary features of ORS:
1. Test Management Center (Completion Status
System)
2. Score Reports
Introduction to the 2015 CAASPP Online Reporting System
23
ORS Welcome Page
Takes you to the
Completion
Status System
Takes you
preliminary student
test scores
Introduction to the 2015 CAASPP Online Reporting System
24
Test Management Center
•
•
•
•
•
Same as the Completion Status System
Provides participation data for students taking tests
Can be used to determine which students need to
complete and/or start testing
Can be used to view a summary statistics report
and download student data files
A detailed overview of accessing and using student
participation reports is provided in the Completion
Status User Guide
Introduction to the 2015 CAASPP Online Reporting System
25
Score Reports
• Provide score data for each administered test
• Compare score data between individual
students and the school, LEA, or overall state
average scores
• Provide information about performance on
claims
Introduction to the 2015 CAASPP Online Reporting System
26
ORS Banner
Completion
Status
Reports
Preliminary
student test
results
Opens a pop-up
window where you can
search for a student
and access his/her
preliminary scores
Displays Retrieve
Student Results page,
where you can access
student performance
data files
Links to the
ORS User
Guide
(forthcoming)
Exports the
data
displayed on
the page in a
.xls file
Introduction to the 2015 CAASPP Online Reporting System
27
Viewing Score Reports
in the ORS
Viewing ORS Reports
1. From the Select drop-down list, select the LEA or
school whose reports you want to view.
– A list will appear only if you are associated with more
than one school or LEA
2. Select [Score Reports].
Note: All score reports data, except for individual students’
score reports, can be disaggregated into subgroups for
detailed analysis. For example, you can view a Grade 5
Mathematics report for a for an entire school or for a
LEA.
Introduction to the 2015 CAASPP Online Reporting System
29
Available Preliminary Score
Reports in the ORS
Report
Home Page Dashboard
Summary of performance (to date) across grades and subjects
or courses for the current administration.
Subject Detail
Data for a subject within a particular grade or course for the
current administration.
Claim Level Detail (Smarter Balanced)
Claim/Block-level percentage at each performance level for a
subject within a particular grade or course for the current
administration.
Student Listing
Roster of all students who belong to a school with their
associated subject or course (and claim, if applicable) scores for
the current administration.
Student Detail
Detailed information about a selected student’s performance in a
specified subject or course (and claim, if applicable).
District
Level
School
Level








Student
Level

Introduction to the 2015 CAASPP Online Reporting System
30
Home Page Dashboard Report
•
•
•
•
•
Displays the overall summary of score data and
testing progress for your LEA and/or school
Starting point for data analysis
Define the students whose aggregated scores you
want to view
Navigate to more detailed score reports from the
Home Page Dashboard
Note: The score data you see are dependent on
your role
Introduction to the 2015 CAASPP Online Reporting System
31
Home Page Dashboard Report
(cont.)
Introduction to the 2015 CAASPP Online Reporting System
32
Subject Detail Report
• The aggregation tables on the Home Page
Dashboard display score data for students by
grade and subject and provide access to
Subject Detail Reports.
Introduction to the 2015 CAASPP Online Reporting System
33
Subject Detail Report (cont.)
Introduction to the 2015 CAASPP Online Reporting System
34
Subject Detail Report (cont.)
•
Each Subject Detail Report consists of the following
components:






Report descriptor
Test name (subject and grade)
Administration year
Entity (e.g., LEA, school)
The title of the score report table
Name, number of students, average scale scores,
percent in each achievement level
 All data are based on the total number of students who have
taken and completed the test and submitted it for scoring
Introduction to the 2015 CAASPP Online Reporting System
35
Claim Level Detail Report
2
1
Introduction to the 2015 CAASPP Online Reporting System
36
Claim Level Detail Report (cont.)
Introduction to the 2015 CAASPP Online Reporting System
37
Claim Level Detail Report (cont.)
•
The Claim Level Detail Report consists of the following
components:
–
Report name
 [Entity] Performance for Each Claim. What are my [entity’s]
strengths and weaknesses in [Subject or Course]?
–
–
–
–
–
Test name (subject and grade)
Administration year
Entity (e.g., LEA, school, or roster)
The title of the score report table
Name, number of students, average scale score, claims,
and percentage in each claims achievement level
Introduction to the 2015 CAASPP Online Reporting System
38
Student Listing Report
2
1
Introduction to the 2015 CAASPP Online Reporting System
39
Student Listing Report (cont.)
Introduction to the 2015 CAASPP Online Reporting System
40
Student Listing Report (cont.)
•
•
•
•
Students’ SSIDs are displayed
Scores for individual students are not mean scale
scores
Different procedure for viewing scale score data by
demographic subgroup
Print on the Student Listing Report prints the
current page and also generates a PDF file of
individual preliminary student results for all the
students in the roster
Introduction to the 2015 CAASPP Online Reporting System
41
Student Detail Report
2
1
Introduction to the 2015 CAASPP Online Reporting System
42
Student Detail Report (cont.)
Introduction to the 2015 CAASPP Online Reporting System
43
Student Detail Report (cont.)
• Displays the breakdown of the student’s
preliminary scale score, achievement level for
the selected subject, and performance and
claim description for each claim
• Includes average scale scores for the state and
LEA for comparison purposes
– Note: State-level scale score averages will not
be available until formally released by the CDE.
Introduction to the 2015 CAASPP Online Reporting System
44
System Enhancements
under Consideration
• Target reports
• Creating roster groups
Introduction to the 2015 CAASPP Online Reporting System
45
Live Demonstration
Questions?
Submitting Feedback on the ORS
New systems require patience, we appreciate your
understanding and welcome your feedback:
https://etsforms.formstack.com/forms/ors_feedback
Introduction to the 2015 CAASPP Online Reporting System
48
Updates and Announcements
Updates and Announcements
• The Online Reporting System User Guide is
forthcoming.
• Next Webcast: 2015 CAASPP Post-Test
Workshop
– Date: Wednesday, May 20, 2015
– Time: 1–3:30 p.m.
Introduction to the 2015 CAASPP Online Reporting System
50
Resources and Support
Help Desk Support
The California Technical Assistance Center
(CalTAC) is here to support all LEA CAASPP
coordinators!
Monday–Friday from 7 a.m.–5 p.m. PT
E-mail: caltac@ets.org
Phone: 800-955-2954
Web site: http://caaspp.org
Introduction to the 2015 CAASPP Online Reporting System
52