California Assessment of Student
Performance and Progress (CAASPP)
Spring 2015 California Alternate
Assessments (CAA) Field Test for
English–Language Arts (ELA) and
Mathematics
March 11, 2015
Sheila Self, M.Ed., California Department of Education
Kellie Taylor-White, MA, Lead Test Developer, ETS
Dan Stanley, Program Manager, ETS
Purpose and Target Audience
•
This Webcast will help LEA CAASPP Coordinators to:
–
–
–
•
Learn all you need to know to prepare for administering the
CAA Field Test in your LEA
Learn how to prepare your Site Coordinators for CAA Field
Testing
Site Coordinators may also wish to view this Webcast
This Webcast also serves as an introduction and
explanation of the CAA Field Test for Site
Coordinators and CAA Examiners
–
CAA Test Examiners are also required to view a Test
Administrator Tutorial that will be available starting March 20
CAA Field Test Introduction and Instructions for Administration
2
Agenda
•
•
•
•
•
•
•
•
•
•
•
Overview
Student Eligibility Requirements
Purpose of the CAA
Features of the CAA
Roles and Responsibilities
Test Security
Registering Students for CAA
Universal Tools, Designated Supports, and
Accommodations
Preparing Technology
Administering the Tests
After Testing
CAA Field Test Introduction and Instructions for Administration
3
Overview—Test Name
•
•
•
•
A new alternate assessment for California as part of the
CAASPP program
The California Alternate Assessments (CAA)
– English–language arts (ELA)
– Mathematics
Replaces the California Alternate Performance
Assessment (CAPA) for ELA and mathematics
Commonalities with other “next generation” alternate
assessments
CAA Field Test Introduction and Instructions for Administration
4
Overview—Testing Window
•
Testing window for all LEAs is April 15 through June
10, 2015.
– All scheduling within this window is according to
local preference
– Do not need to inform ETS or CDE of testing
schedule
– All testing must take place within window,
including make-ups
CAA Field Test Introduction and Instructions for Administration
5
Overview—Test Takers
•
All eligible students in grades 3–8 and grade 11 must
participate
– Students in ungraded programs will need to be
assigned a grade level for testing
– Students in grades 5, 8, and 10 still take the CAPA
for Science (in addition to CAA at grades 5 and 8)
CAA Field Test Introduction and Instructions for Administration
6
Overview—Test Delivery
•
•
•
•
•
Testing is online only, using a compatible Webbrowser with no software installation necessary.
One-on-one test sessions with a Test Examiner, who
is familiar with the student.
All items are grade level, aligned with the Common
Core State Standards (CCSS) and based on the
Core Content Connectors developed by NCSC.
Three different levels of complexity, and all three
levels will be administered to all students.
Items are being tested, not students.
CAA Field Test Introduction and Instructions for Administration
7
Overview—Resources
Key resources for CAA Field Test:
• CAA Field Test Administration Manual (TAM)
• Sample Directions for Administration (DFA)
– Will be paired for use with a Training Test when
available
• Grade-level DFAs
• Individual Student Assessment Accessibility Profile
(ISAAP) Tool
• Test Administration Tutorial
http://caaspp.org/about/caa/
CAA Field Test Introduction and Instructions for Administration
8
Student Eligibility Requirements
1. The student has a significant cognitive disability.
2. The student is learning content derived from the
Common Core State Standards (CCSS).
3. The student requires extensive direct individualized
instruction and substantial supports to achieve
measurable gains in the curriculum.
Additional detail on these requirements can be found in
the TAM on page 5.
CAA Field Test Introduction and Instructions for Administration
9
Purpose of the CAA
•
Give students with significant cognitive disabilities the
opportunity to demonstrate their achievement of the
Common Core State Standards (CCSS) by taking a
test commensurate with their abilities.
– Assessed through Core Content Connectors
(CCCs) that are rewritten main achievement
standards from the CCSS.
– Provide more access to students with significant
cognitive disabilities.
CAA Field Test Introduction and Instructions for Administration
10
Features of the CAA
•
•
•
•
CAA is a grade-level assessment only
Student grade levels are taken from CALPADS.
– For students in ungraded programs, testing
grades are calculated by the Test Examiner using
the “subtract 5” rule
For the Field Test, students in the same grade will
see items reflecting all levels of difficulty, from least
complex to more complex.
Approximately 15 items per content area that take
30–60 minutes to administer.
CAA Field Test Introduction and Instructions for Administration
11
Features of the CAA (cont.)
•
•
Tests are administered with grade-specific and
version-specific DFAs.
Versions are assigned at the school level.
– Version assignments for each school in California
will be posted to caaspp.org in late March,
shortly before DFAs are available for download in
TOMS.
http://caaspp.org/about/caa/
CAA Field Test Introduction and Instructions for Administration
12
Roles and Responsibilities—
LEA CAASPP Coordinator
For the CAA Field Test, LEA CAASPP Coordinators are
responsible for all of the following activities:
• Registering students in TOMS
• Training CAASPP Test Site Coordinators (SCs)
• Configuring student test settings in TOMS (with SCs)
• Confirming with SCs that each CAA Test Examiner has
viewed the Test Administration Tutorial and verifying
training completion
• Ensuring that all Test Examiners have read and signed
a test security affidavit
• Reporting all test security irregularities and/or breaches
CAA Field Test Introduction and Instructions for Administration
13
Roles and Responsibilities—
CAASPP Test Site Coordinator
For the CAA Field Test, CAASPP Test Site Coordinators
are responsible for the following activities:
• Identifying certificated CAA Test Examiners, ensuring
that they view the CAA Test Administration Tutorial,
and collecting the resulting certificate of completion.
– CAA does not use the train-the-trainer model of
providing instruction to Test Examiners. Instead,
Test Examiners must participate in the Test
Administration Tutorial.
– The Test Administration Tutorial is required for all
Test Examiners.
CAA Field Test Introduction and Instructions for Administration
14
Roles and Responsibilities—
CAASPP Test Site Coordinator (cont.)
• Collecting signed CAASPP Test Security Affidavits from
CAA Text Examiners
• Adding each new CAA Test Examiner into TOMS and
tracking completion of the Test Administration Tutorial
• Entering and/or verifying test settings for students in
TOMS (universal tools, designated supports, and
accommodations [UDAs] that were uploaded into
TOMS with the ISAAP tool)
• Creating or approving testing schedules and
procedures for the school
CAA Field Test Introduction and Instructions for Administration
15
Roles and Responsibilities—
CAASPP Test Site Coordinator (cont.)
• Working with technology staff to ensure that testing
devices are functioning properly
• Coordinating with CAA Test Examiners so they administer
all assessments
• Mitigating and reporting all test security incidents in a
manner consistent with state and LEA policies
• Collecting secure DFAs and any print-on-demand items
from Test Examiners following testing and securely
destroying them
CAA Field Test Introduction and Instructions for Administration
16
Roles and Responsibilities—
CAA Test Examiner
For the CAA Field Test, CAA Test Examiners are
responsible for the following activities:
• Reading and submitting a signed CAASPP Test
Security Affidavit to your SC
• Viewing the Test Administration Tutorial, printing the
resulting certificate, and submit to your SC
• Reviewing designated supports and accommodations
in TOMS for each student you are testing
• Downloading from TOMS and reading the DFAs
before testing
CAA Field Test Introduction and Instructions for Administration
17
Roles and Responsibilities—
CAA Test Examiner (cont.)
• Conducting a student response check for each
student (required)
• Completing the learner characteristics inventory for
each student (optional)
• Administering the tests in accordance with the CAA
Field Test Administration Manual
• After testing, give any paper copies of the DFAs to
your SC for secure destruction and work with your SC
to permanently delete any PDF copies
CAA Field Test Introduction and Instructions for Administration
18
Test Security
•
•
Follow the same security guidelines as for other
CAASPP assessments.
Information about Field Test items, including DFA
content, should not be shared with anyone not directly
responsible for test administration.
CAA Field Test Introduction and Instructions for Administration
19
Registering Students for CAA
•
•
Students are registered for CAA in TOMS
Any students previously registered for “alternate” using the
CAPA flag in the CAASPP registration upload template
(available since November) should automatically be
registered for CAA by TOMS.
–
•
•
Because tests are grade-level only, students must be
assigned a grade for testing.
For students who are in ungraded programs and whose
IEP teams designate that they take the CAA Field Test,
their grade for testing is calculated by subtracting five from
their chronological age on September 1, 2014.
–
•
Registrations were accepted for all CAA/CAPA grades (3–8, 10,
11).
A student who was 10 years old on September 1 would be given
the grade five test (10 – 5 = 5).
Confirm the grade assignment is correct for each student.
CAA Field Test Introduction and Instructions for Administration
20
Registering Students for CAA (cont.)
•
Starting March 9, one-by-one student registration
available in TOMS
–
–
–
–
•
Registration for CAA/CAPA automatically un-registers the
student for any other CAASPP assessments
When CAPA or CAA is selected, student will be registered for
both.
To unregister a student from CAPA/CAA, a CST or CMA must
be selected and CAA must have a selection of “No”.
Starting on March 20, CAA registrations will be viewable in the
TOMS student roster report
Bulk registration is still available, for CAA only, using the
same paper-pencil registration template
–
–
In TOMS, select “Paper test assignments” from the “Students”
menu
Registering for CAA/CAPA will not automatically change any
CAPA materials orders. If needed, contact CalTAC to order
additional CAPA materials through the supplemental order
process.
CAA Field Test Introduction and Instructions for Administration
21
Registering Students for CAA (cont.)
•
There is no specific deadline for registering students
for CAA, however:
–
–
Registration should be completed as soon as possible
so that student rosters will accurately reflect the
correct test assignments.
Students should be registered for the correct
assessments before test settings (designated supports
and accommodations) are configured for those
students in TOMS.
CAA Field Test Introduction and Instructions for Administration
22
Universal Tools, Designated Supports,
and Accommodations
The CAA field test makes many of the same universal tools,
designated supports, and accommodations (UDAs) available as for
other CAASPP assessments.
• Universal tools are available to all students by default, although
they can be disabled if they are distracting for a student.
• Designated supports can be provided at the discretion of a
teacher familiar with the student’s needs.
• Accommodations must be specified in a student’s IEP.
– Note that, for the field test, students may use any
accommodation routinely used in the classroom, even if not
currently in IEP.
– During the next cycle of IEP review, you may update with
state testing accommodations.
CAA Field Test Introduction and Instructions for Administration
23
Universal Tools, Designated Supports,
and Accommodations (cont.)
Many of the same UDAs available for other CAASPP assessments
are available in the same way for the CAA field test, with the
following exceptions listed below:
• Text-to-speech, which is not available as an embedded support
but may be provided as a non-embedded support and/or
through an assistive device.
• Braille, which is not available for the CAA field test.
• American Sign Language, which is not available as an
embedded support.
• Translations/Glossaries, which are not available as embedded
supports.
• Closed Captioning, which is not available for the CAA field test.
CAA Field Test Introduction and Instructions for Administration
24
Universal Tools, Designated Supports,
and Accommodations (cont.)
The Individual Student Assessment Accessibility Profile
(ISAAP) tool should be used to identify the right UDAs for
each student taking the CAA.
• Same tool for Smarter Balanced is now enhanced to work
for CAA. Select the “CAA” checkbox to configure the tool
according to UDA availability for the CAA Field Test.
• Enter the specific needs of students and tool will
automatically generate a file that can be uploaded into
TOMS.
• ISAAP tool is available from caaspp.org, from
http://caaspp.org/about/caa/.
CAA Field Test Introduction and Instructions for Administration
25
Preparing Technology
•
•
The CAA are designed for one-on-one administration
between a student and a certificated Test Examiner.
Online testing using a supported Web browser (secure
browser not used)
–
–
•
•
A list of supported systems, devices and browsers can be
found at http://ca.browsers.airast.org/supported-browsers/
Refer to “TA Sites” column in the table
A single device is needed to launch a test session and
administer the test. (TA Interface not used)
Other technology and device specifications, including
headphones and keyboard requirements, are generally
the same as for Smarter Balanced.
– Assistive devices may be used, according to the
communication and response method of each student.
CAA Field Test Introduction and Instructions for Administration
26
Other Technology Resources
Refer to the following CAASPP resources at
http://caaspp.org/administration/instructions/ for other
instructions and guidelines. (Note that parts of these
manuals will not apply to CAA, such as use of the secure
browser, TA Interface, and the test settings not available
for the CAA field test):
• Technical Specifications Manual—for instructions on
preparing technology for testing
• Online Test Administration Manual—for general
CAASPP system instructions such as security
• 2015 Online Student Test Settings User Guide—for
information about assigning accessibility tools and
supports
CAA Field Test Introduction and Instructions for Administration
27
Administering the Tests
•
A separate, grade- and version-specific Directions for
Administration (DFA) document guides Test Examiners through
the CAA with item-specific administration instructions, including
scoring rubrics for writing items.
– Version assignments for each school in California will be
posted at http://caaspp.org/about/caa/ and DFAs will be
available for download from TOMS before the testing
window opens.
– DFAs are in PDF format and may be printed or used
electronically according to local preference.
– Because these DFAs contain live testing material, they must
be kept secure at all times and must be securely destroyed
after testing.
 LEA or site coordinators must ensure that electronic
copies are securely destroyed on each device they were
loaded to.
CAA Field Test Introduction and Instructions for Administration
28
Administering the Tests (cont.)
Administration steps on test day:
1. On the device that will be used for testing, Test Examiner
opens a supported Web browser and navigates to the CAA
Online Testing Interface, with DFA in-hand or on a separate
device.
2. Test Examiner logs on using TOMS username and password.
3. Test Examiner logs onto student test session using student
first name and SSID.
4. Test Examiner is presented with three testing options:
a. Learner Characteristics Inventory (LCI)
b. ELA
c. Mathematics
5. Test Examiner completes the LCI (optional)
CAA Field Test Introduction and Instructions for Administration
29
Administering the Tests (cont.)
Administration steps on test day:
6. Test Examiner launches ELA or mathematics test and engages student
for the start of testing
7. Test Examiner is prompted to administer a Student Response Check
(SRC) to observe and record the student response mode and how the
student will indicate their responses during testing. Answering these
SRC questions is required for all students. Examples of student
responses include:
 Mouse/keyboard
 Verbal response
 Touch screen, gestures, or point
 Augmentative and/or alternate communication device
 Eye gaze
8. ELA and mathematics tests are untimed, and the Test Examiner may
administer as many sessions over as many days as necessary
according to individual student needs.
CAA Field Test Introduction and Instructions for Administration
30
Administering the Tests (cont.)
Learner Characteristics Inventory
• An optional Learner Characteristics Inventory (LCI) is
available after logging in to a student’s tests. If
completed, the LCI questions should be answered by
the Test Examiner before the start of ELA and
mathematics testing for each student, based on his/her
familiarity with that student.
• Please note that LCI responses will be aggregated;
neither Test Examiners nor student(s) will be identified
in or by the results.
CAA Field Test Introduction and Instructions for Administration
31
Administering the Tests (cont.)
Student Response Check
• CAA Test Examiners must administer the Student
Response Check (SRC) to each student at the
beginning of testing. The purpose of the SRC is to
document that the student has a consistent and
observable way to communicate his/her response to the
test items, using the intended response mode(s).
• Hand-over-hand or other physical prompting by the test
examiner is not permitted in the CAA field test.
• The SRC does not need to be administered twice. (Can
be skipped at the start of the second content area).
CAA Field Test Introduction and Instructions for Administration
32
Administering the Tests (cont.)
Student Response Check (Cont.)
There are three possible outcomes from the SRC:
1. The student demonstrates an observable, consistent
response
2. The student demonstrates an observable, but
inconsistent, response
3. The student does not demonstrate any observable
responses
CAA Field Test Introduction and Instructions for Administration
33
Administering the Tests (cont.)
Student Response Check (Cont.)
• If the outcome is “1,” Test Examiners administer the entire
assessment.
• If the outcome is “2,” Test Examiners administer the first four ELA
or mathematics items. If a consistent, observable response is
elicited for any of these four items, administer the entire
assessment.
– The response does not need to be correct but rather the
student orients to the test and attempts to respond.
– If no observable response for the first four items, the Test
Examiner should stop testing and report this to the Site
Coordinator.
• If the outcome is “3,” Test Examiners do not administer the
assessment.
CAA Field Test Introduction and Instructions for Administration
34
Administering the Tests (cont.)
Pause Rules
The following pause rules apply regardless of whether the
student or the Test Examiner pauses the test or there was a
technical issue that resulted in the student being logged off.
The CAA may be paused and resumed as many times as
necessary to elicit the student’s best testing performance.
• When a test is paused, you must log back on to resume
testing. Upon resumption, students will automatically be
directed to the first page that has an unanswered item.
• In the event of a technical issue (e.g., power outage or
network failure), student’s test will be paused and the
students will be logged off.
• Students will return to their last unanswered test question
when logged on again.
CAA Field Test Introduction and Instructions for Administration
35
Administering the Tests (Cont’d)
Test Timeout (Due to Inactivity)
•
As a security measure, test sessions are automatically logged off
after 30 minutes of test inactivity. This timeout also results in the
test being paused automatically.
•
Activity is defined as selecting an answer or navigation option in
the test (e.g., selecting [Next] or [Back] or using the Questions
drop-down list to navigate to another item). Selecting an empty
space on the screen is not considered activity.
•
Before the system logs off of the test, a warning message will be
displayed on the screen. If you do not select [OK] within 30
seconds after this message appears, he or she will be logged off.
CAA Field Test Introduction and Instructions for Administration
36
CAA Training Test
•
•
•
For practice and training purposes, Test Examiners
will have access to an online training test starting in
early April.
Used in conjunction with the sample DFA, this
training test can help both Test Examiners and
students become familiar with the new testing format
and functionalities of the CAA assessments.
More information about the Training Test will be
provided when available.
CAA Field Test Introduction and Instructions for Administration
37
After Testing
•
•
•
Site Coordinators ensure all DFAs are collected and
securely destroyed
– Shred or otherwise destroy printed copies
– Permanently delete electronic PDF copies from all
devices
As necessary, enter any special testing condition
codes in TOMS.
After testing, you may be asked to complete a survey
about your field-testing experience. This information
will help to improve the test.
CAA Field Test Introduction and Instructions for Administration
38
Comparing CAPA to CAA
Aspect
CAA FT
CAPA
Training methodology for Test Examiners
Training is provided via an on-demand test
administration tutorial and is required in order to
administer the CAA.
Testing mode
Online only (A print-on-demand accommodation
is available through the online system for test
items by request)
Test is administered one on one.
Paper only
Grade level only; calculation is used to assign
grades to students in un-graded program for
testing.
No physical prompting of any kind is permitted.
Levels and grades
Training
Train-the-trainer model: Training was provided
by SCs who were trained by LEA CAASPP
Coordinators.
Administration
Testing format
Levels
Physical prompting of students
Test administered one on one.
Hand-over-hand prompting was permitted.
Entering responses
The student may enter their responses using a
CAPA Examiner filled in the answer document.
mouse or keyboard or the CAA Test Examiner
enters responses into the online testing interface
reflecting the student’s indicated response.
Scoring
CAA Test Examiner provides scores for writing
items as directed in the DFA. Scoring of other
items is done by the online system. No student
scores will be available for the Field Test.
CAPA Examiner provided scores.
Second Rating for Test Examiner-scored items.
There is no second rater.
A second rater scored 10% of exams.
Tools
Physical stimulus, manipulatives
None
Test used physical stimuli and manipulatives.
Availability
Similar universal tools, designated supports, and None; the CAPA was administered and adapted
accommodations to other CAASPP
based on student need.
assessments, with some exceptions.
Accommodations
CAA Field Test Introduction and Instructions for Administration
39
Support for the CAA Field Test
The CAASPP portal at http://caaspp.org/ is the home for all
CAASPP assessment administration information, including
the CAA field test. All CAA materials and resources will be
available on the CAASPP CAA Field Test Web page at
http://caaspp.org/about/caa/.
The California Technical Assistance Center
(CalTAC) is here to support all LEA CAASPP coordinators!
Monday–Friday from 7 a.m.–5 p.m. PT
E-mail: caltac@ets.org
Phone: 800-955-2954
Web site: http://caaspp.org
CAA Field Test Introduction and Instructions for Administration
40
Questions?
CAA Field Test Introduction and Instructions for Administration
41