2014 Smarter Balanced Field Test In-Service Transitioning from STAR to CAASPP February 27, 2014 1 Agenda • New Testing System: California Assessment of Student Performance and Progress (CAASPP) • General Overview of Smarter Balanced System • Smarter Balanced Field Test Overview • Site Coordinator and Test Administrator Responsibilities • Test Security, Planning and Logistics for Field Test • Overview of Student Supports & Accommodations • Preparing Teachers & Students • On-line Test Administration • Classroom Activity & Performance Task • After Smarter Balanced Testing • California Standards Tests (CSTs) for Science and CAPA 2 Components of 2014 State Tests Math and English Language Arts Grades 3-8 CST/CMA Science Grades 5&8 Identified Special Education Students Grades 2- 8 Math and English Language Arts 3 Smarter Balanced Assessments: Improving Teaching & Learning Common Core State Standards specify K-12 expectations for college and career readiness Summative: College and career readiness assessments for accountability Teachers and schools have information and tools they need to improve teaching and learning Formative resources: Supporting classroom-based assessments to improve instruction All students leave high school college and career ready Interim: Flexible and open assessments, used for actionable feedback 4 2014 SBAC Field Test 5 Field Test Purpose A field test is not designed to be a valid and reliable measure of student achievement; rather, it is designed to help the test developers evaluate whether the tests, individual items, and the technology platform work as intended before the first operational administration. — Deborah S. Delisle U.S. Department of Education 6 Benefits of Field Test Participation • Students will have hands-on experience with the functionality of a computer-based assessment. • Teachers and administrators will gain valuable exposure to administration logistics during a trial run. • Local educational agencies (LEAs) will benefit from having learned where technology gaps may exist, and then can fully prepare for operational assessments. 7 Smarter Balanced Spring 2014 Field Test Overview CA TAM 7, 29-30 • Field Test window for BCSD: April 7– May 16, 2014 • April 10-11 and May 8-9 are black-out days for testing • April 14-21 Spring Break • 20 school days for testing • Grades 3–8: All students are expected to participate with the exception of • CAPA students • EL less than 12 months do not test in ELA but do test in math • (R&E to provide translation of SAY Boxes upon request) • Parent exemptions • Signed parent letters 8 Smarter Balanced Spring 2014 Field Test Overview • Test will be approximately 3.5-4.5 hours long though it is untimed • No paper-and-pencil version will be available • No student, school, or district score reports will be produced • Results will not be factored into any state or federal accountability calculations 9 Smarter Balanced Spring 2014 Field Test Overview Each participating student in Grades 3-8 will experience: • 25 ELA questions • 25 mathematics • 1 performance task • Either ELA or Math • We will be assigned one by March 10, 2014 • R&E to inform schools 10 Smarter Balanced Spring 2014 Field Test Overview Item Types • • • • • • CA TAM 71 Selected Response (SR) • Previously multiple choice Constructed Response (CR) • Items of greater complexity than SR. Require more analytical thinking and reasoning; develop answers without suggested answer choices. Extended Response • Short text and essay Performance Tasks (PT) • Requires a classroom activity and independent student response. Measure capacities such as depth of understanding, interpretive and analytical ability, basic recall, synthesis, and research Technology-Enabled • Video, animation, sound • Respond with SR or CR Technology-Enhanced • Students manipulate information • Require students to do something other than write text or numbers, or select from among a set of options such as reordering text, selecting and changing text, selecting text, and selecting from drop-down menus 11 Roles and Responsibilities 12 New Terminology for Roles Smarter Balanced terminology California terminology District Test Coordinator (DC) LEA CAASPP Coordinator School Test Coordinator (SC) Test Site Coordinator Test Administrator (TA) Test Examiner and Test Proctor Other important roles ◦ LEA Technology Coordinator (John Deaton) ◦ LEA CALPADS Administrator (Robert Burks) 13 CA TAM 9-10 Roles and Responsibilities TEST SITE COORDINATOR (SC) • Collect security affidavits from Test Administrators (TAs) and retain at your test site. • Train TAs • Submit appropriate supports and accommodations to R&E. • Ensure that testing in the school is conducted in accordance to test security policies. • Create test schedules and procedures for the school. • Report test security incidents to R&E • Monitor testing and address problems, as needed. TEST ADMINISTRATOR (TA) • Read and sign new security affidavit. • Attend test administration training. • Confirm student information prior to testing, to ensure it is correct. • Ensure that appropriate designated supports and accommodations have been set prior to testing. • Set up test sessions & proctor testing. • Report test security incidents to SC. • Encourage students to do their best. 14 Test Security 15 New Test Security Forms for the 2013–14 School Year CA TAM 20-22 • Site Coordinators must sign the new CAASPP Test Security Agreement and retain at the LEA. • Test Administrators must sign the new test security affidavit and provide a copy for LEA. • Forms specify no reviewing, discussing, or analysis of test items, reading passages, or writing prompts at any time during between, or after sessions. • Agree to not email, fax, or replicate items. • No form or type of answer key may be developed for items 16 17 Test Security CA TAM 23-24, 63-70 There are three different types of test security incidents that must be reported: • Impropriety • Irregularity • Breach Notify R&E within 24 hours for processing and next steps. 18 Test Impropriety • Unusual circumstance that has a low impact on the testing individual or group of students • Low risk of affecting student performance, test security, or test validity • Correctable and containable at local level • Example: Students talking during testing 19 Test Irregularity • Unusual circumstance that impacts the testing individual or group of students • May affect student performance, test security, or test validity • Correctable and containable at the local level • Submitted in the online system for resolution of the Appeal for testing impact • Examples: Student cheating or providing answers to another; student accessing or using electronic equipment (e.g., cellphone) 20 Test Breach • Event that threatens test validity (e.g., release of secure materials) • External implications for the Consortium • Examples: − Administrator or coordinator modifying student responses or record at any time − Administrator allowing students to take home test items, passage, prompts, or scratch paper 21 Planning and Logistics Field Test Details CA TAM 9 • Preparation Checklist in CA TAM page 79-89 • Secure Web Browser has been downloaded for you. • Test Administrator & Proctors = State-certified school staff that have been trained to use the Smarter Balanced Testing system need to be present in each testing room. 22 Planning and Logistics Integration of LEA and CDE Student Information Systems Integration with CALPADS • CALPADS is the sole source for student demographic data • Frequent data extracts from CALPADS • First data pull was February 18, 2014 CA TAM 28 Test registration system • CALPADS information is transferred to TIDE (Test Information Distribution Engine) • TIDE is the system used to manage student information and user accounts for online testing. • Since data is already loaded in TIDE, school coordinators will primarily utilize this system for look-up. • Username and passwords will be emailed to you. • Student demographic/program data cannot be changed by LEAs once in TIDE • LEAs will be responsible for providing accommodations data 23 Testing Times and Scheduling The Field Test is expected to take 3.5 to 4 hours to complete CA TAM 35-36 ◦ April 7 – May 16 ◦ Field Test ◦ Black-out days April 10-11 & May 8-9 24 Scheduling Testing Time: • Recommended administration: CA TAM 37 • First: Non-PT section • Second: Classroom Activity • Third: Individually administered on-line PT (Classroom & PT not on the same day) • Administering the PT section: • Classroom activity should be completed within 3 days prior to starting the PT. • ELA: 2 parts; can be completed in 2 days. • Math: 1 part; can be completed in 1 day. 25 Tentative Schedules 26 Scheduling Considerations for Special Populations CA TAM 29-30 • Ensure that TAs know which students are not to test. • Will appear in TIDE but should not be tested: • CAPA students. • Students with Parent Exemptions will not be tested • English Learners 12 months or less should not take ELA • Schedule activities for students not taking exam (or that finish early). 27 Smarter Balanced Usability, Accessibility, and Accommodations Guidelines 28 Smarter Balanced Usability, Accessibility, and Accommodations Guidelines Three types of student support offered: CA TAM 45-47 reference 1. Universal tools 2. Designated Supports 3. Accommodations The Guidelines document is available on the CDE Smarter Balanced Accessibility and Accommodations Web page at http://www.cde.ca.gov/ta/tg/sa/access.asp. Accommodations Module 29 Universal Tools, Designated Supports, and Accommodations Embedded Breaks, Calculator, Digital Notepad, English Dictionary, English Glossary, Expandable Passages, Global Notes, Highlighter, Keyboard Navigation, Mark for Review, Math Tools, Spell Check, Strikethrough, Writing Tools, Zoom Non-embedded Breaks, English Dictionary, Scratch Paper, Thesaurus Embedded Color Contrast Masking Text-to-speech Translated Test Directions Translations (Glossary) Translations (Stacked) Turn off any Universal Tools Non-embedded Bilingual Dictionary, Color Contrast, Color Overlay, Magnification, Read Aloud, Scribe, Separate Setting, Translation (Glossary) Embedded American Sign Language, Braille, Closed Captioning, Text-to-speech Non-embedded Abacus, Alternate Response Options, Calculator, Multiplication Table, Print on Demand, Read Aloud, Scribe, Speech-to-text 30 Usability, Accessibility, and Accommodations Guidelines UAA Guide Glossary 6-17 1. Universal tools: available to all students. Embedded Tools Highlighter Digital Notepad English Glossary English Dictionary Keyboard Mark for Navigation Review Expandable Passages Global Notes Strike Through Writing Tools Non Embedded Options Breaks English Dictionary Scratch Paper Thesaurus 31 Usability, Accessibility, and Accommodations Guidelines Grades 3–5 Calculator Policy • Do not allow for calculator usage. Grades 6–8 Calculator Policy • Mathematics assessments are divided into two sections: Calculator Available and Calculator Not Available. • Grade 6 allows an online four-‐function calculator during the Calculator Available section. • Grades 7 and 8 allow an online scientific calculator during the Calculator Available section. 32 Usability, Accessibility, and Accommodations Guidelines UAA Guide Glossary 6-17 2. Designated supports, available to identified students, including English learners, struggling readers, and students with attention deficits that the student has been using during instruction or other assessments. Examples include color-contrast text, the blocking of distracting content, stacked translations, and bilingual dictionary. Translation of test math instructions and items are available. 33 Usability, Accessibility, and Accommodations Guidelines UAA Guide Glossary 6-17 3. Accommodations, available to students with an individualized education program or a 504 plan that specifies the need for such an accommodation. Examples include closed captioning, Braille, calculator, and scribe. R&E staff will input designated support and accommodations, process is on next slide. 34 Accommodations & Supports Checklist for IEPs, 504s & ELs* Highlight all that apply for students currently using these tools during instruction *ELs less than twelve months Complete and return to R&E no later than March 24th 35 Preparing Staff and Students for the Field Test CA TAM 17-18 Recommended Training Modules Module Time TA Test Administration Overview 55:00 X TA Interface 17:00 X Performance Task Overview 11:13 X Let’s Talk Universal Tools 14:07 X Student Interface 22:48 X What is a Field Test? 5:52 X Accessibility and Accommodations 35:00 X Students Includes portions of TA and Student Interface X X (Sp Ed) 37 Resources: Training Test and CA TAM 30 Practice Test Training Test Practice Test Provide students with an opportunity to quickly become familiar with the software and interface features Provide students with a grade specific testing experience that is similar in structure and format to the field test Grade Levels Grade bands •3-5 •6-8 Each grade •3-8 Number and Types of Items Approximately 15 items per grade band (6 in ELA and 8-9 in math) Approximately 30 items in ELA and 30 items in math per grade level No Performance Tasks Includes 1 ELA PT and 1 math PT per grade level Purpose Universal Tools, All included on field test are Designated included Supports, and Accommodations Most included Refresh scheduled for late April 38 Practice and Training Tests California Smarter Balanced Portal: http://sbac.portal.airast.org/ca/practice-test-ca/ For student Interface option: “Guest User” is defaulted and sign in For TA Interface option: Uncheck default “Guest User” to practice authentically creating session and student login. 39 Field Test Administration Establishing Appropriate Testing Conditions • Remove or cover displayed information that might be used by students to help answer questions including: • Rubrics, vocabulary charts, student work, posters, graphs, charts, etc. CA TAM 28, 48-51 • Cell phones or other electronic devices are not allowed to be used during testing (i.e., turned off and put away). Post an “Unauthorized Electronic Devices May Not Be Used at Any Time During the Testing Session” • Place a “TESTING—DO NOT DISTURB” sign on the door. • May also be helpful to post signs in halls and entrances rerouting hallway traffic. 41 Establishing Appropriate Testing Conditions Establish procedures to maintain a quiet testing environment throughout the test session. • Remember that some students will finish more quickly than others. Plan quiet activities for students who finish early 42 Basic Test Rules During the Field Test CA TAM 32 • Students must answer all test items on a page before moving on to next page. • Within each test there may be segments. A student may not return to a segment once it has been submitted. • Students must answer all test items before ending their test. • Students may not return to a test once it has been submitted. • Students may flag a response and return to it before ending their test. • Students may use scratch paper but it must be collected and shredded at the end of each test session. 43 Accessing the Field Test: http://sbac.portal.airast.org/ca/field-test-ca/ • Select [Field Tests]. • Select [TA Interface for California Users]. 44 TA Interface: Teacher Login • User Names & Passwords will be emailed once uploaded. • Confirm teachers who need administration access. • Default will be all teachers • Submit names to R&E no later than March 7, 2014 • The first time logging on, you will be required to change your password. Write it down, we do not have access to it. 45 Starting a Test Session Click ‘Start Session’ • Select test to be administered • Tab will turn red and will change name to “Stop Session” • Write Session ID on the board for students to log into your test session 46 Student Login: Joining TA’s Session • Click Secure Browser icon • All other applications must be closed • Prior to Student Login: TAs must follow the “DFA SAY” script exactly each time a test is administered. Refer to the DFA section of the CA TAM page 51. • Please contact R&E for Spanish Translation of “DFA SAY” boxes • Student will need: − Confirmation Code: Student’s legal first name as spelled in CALPADS − State-SSID: State abbreviation (CA) followed by a hyphen and the student’s Statewide Student Identifier − Session ID 47 Student Login Data Cards R & E will provide a Sign-In Card with Student Login Data • Junior High/Middle Schools consider how you want cards sorted Delivered to school sites prior to testing window • Use the information cards during the “Training Test” for students • Students will type information EXACTLY as printed Collect students cards after each administration and keep secure 48 Student Login: Verify & Select Test • Students will be prompted through a series of questions prior to starting their test session 49 Student Login: Sound Check • Students may be prompted to do a sound check if taking ELA test with listening portions. • If prompted, students should have headphones on and press green button to test volume and then press ‘Yes, I heard the voice’ or adjust as needed. 50 Student Login: Test Instructions • Student have a “Help Guide” available at anytime • Menu of keyboard commands • Students can “Begin Test Now” or “Cancel” 51 Reminder: Student Universal Tools CA TAM 45 All students have the ability to: • Highlight text in passages and test questions • Zoom in and out of test pages (affects all item content) • Mark specific items for review • Strikethrough (cross out answer options) • Use the Notepad or Calculator (depending on the test) • Use the [Expand] button to display a reading passage or a science simulation UNIVERSAL TOOLS MODULE 52 Student Technology Ribbon • Ribbon tools will vary depending on type of item • Test will automatically “Save” but it is recommend that students save during their essay 53 TA Approve Students for Testing • TAs can either select [Approve] for each individual student or select [Approve All Students]. • Reminder: If any student’s test settings are incorrect, do NOT approve that student. • To refresh the list of students awaiting approval at any time, select the [Refresh] button at the top of the pop-up window. 54 TA Actively Monitors • • • The “Students in Your Test Session” table displays students who have logged in and been approved for testing. Screen displays the item the student is currently solving and which students are paused (by inactivity or choice) Actively proctor for student engagement 55 Pausing/Stopping • Through the Test Administrator Interface: CA TAM 33 • Stop an entire session; or • Pause individual student tests. • Users should exit or log out of the Test Administrator Interface only after stopping the test session. • If there is a technical issue (i.e. power outage or network failure), students will be logged out and the test will automatically be paused. • Regardless of when or how users log out or navigate away from the Test Administrator Interface, student data will NOT be lost. Note: If a test is paused for any reason or length of time, the student must log in again to resume testing. Highlighted text and item notes will not be preserved. 56 TA Interface: Pausing/Stopping 57 Pause Rules • During the Non-Performance Task, if testing is paused for more than 20 minutes, the student is: • • • Presented with the test page containing the test item he or she was last working on (if page contains at least 1 unanswered item) OR Presented with the next test page (if all items on the previous test page were answered); NOT permitted to review or change any test items on previous test pages. • There is no pause rule for the Performance Task. • • Even if a test is paused for 20 minutes or more, the student can return to the current section and continue. ELA PTs are divided into 2 parts. After a student completes Part 1, he or she cannot return to it. Note: If a test is paused for any reason or length of time, the student must log in again to resume testing. Highlighted text and item notes will not be preserved. 58 Test Timeout Due to Inactivity • As a security measure, students are automatically logged out of the test after 20 minutes of inactivity. CA TAM 34 • Activity means: • • • Selecting an answer Using a navigation option in the test (i.e. selecting [Next] or [Back], using the Past/Marked Questions drop-down list) NOTE: Moving the mouse or selecting an empty space on the screen is NOT considered activity. • Before the system logs out, a warning message will be displayed. • If a student does NOT select [Ok] within 30 seconds, the student will be logged out. • Selecting [Ok] will restart the 20 minute inactivity timer. 59 Student Reviewing of Items 60 Student Ending the Test 61 TA Stopping Tests: Entire Session • To stop the session (and pause tests for all students in the session): • Select the [Stop Session] button in the upper-left corner of the screen. • An “Important!” box will appear, requesting verification to end the session and log students out. • Select [OK] to continue or [Cancel] to keep the test session open. 62 Performance Task Two Components: Whole Class Activity Student Independent Response 63 Classroom Activity Administration Guidelines • Classroom Activity Purpose: To provide students with important context, concepts, and key terms prior to taking the PT. • All students will have a classroom activity before they complete the PT. All students need to actively participate. • Site Coordinator will be notified to download the assigned classroom activity and provide to Test Administrators at least 1–2 days prior to prepare materials. CA TAM 49-50; Checklist TAM 81 Performance Task Module 64 Classroom Activity Administration Guidelines • Designed to fit into a 30-minute window. • Will vary due to complexity of topic and individual student needs. • Provide the appropriate accommodations for students who normally use them during instruction. • Recommended: No more than a 3-day lapse between the classroom activity and the PT administration. 65 Classroom Activity Administration Guidelines • May be administered in a classroom or any other appropriate space. TA may need a chalkboard or dry-erase board. • Students may take notes during the classroom activity, but the notes must be collected before proceeding to the PT. • PT will be completed by individual students in the computerbased test delivery system. 66 Classroom Activity & Performance Task Administration Process Step 1: Site Coordinators will identify the classroom activities for your school. • Assigned by grade for each individual school. • Assignments will be posted on the SBAC portal. • Locate your state and then use the menus within the file to filter for your school. • Beside the name of the school will be a listing of the classroom activities assigned to each grade/content area. 67 Classroom Activity & Performance Task Administration Process Step 2: Site Coordinators will download the classroom activity. • Materials are posted on the site in alphabetical order. Step 3: Test Administrators will review the teacher directions for the classroom activity. Step 4: Test Administrators will complete the classroom activity. • Site Coordinators should ensure that the Test Administrator has presented the classroom activity prior to performance task administration. • Classroom activities include directions associated with that specific classroom activity. 68 Classroom Activity & Performance Task Administration Process Step 5: Administer the make-up classroom activity- may require Test Administrator to be the student partner. Step 6: Administer the performance task after classroom activity. • The students must work independently • The PT is administered online • ELA PT Components: Part 1: Students will read sources and answer three research questions. • Students may take notes electronically or on paper • Test Administrators are to collect and secure scratch paper and redistribute to student for Part 2 Part 2 (full write): Students will provide a written response using those sources. • Students are allowed a hard copy of dictionary or thesaurus/on-line option available Recommended: Each part administered on separate days. • Math PT Component: One part: Student will receive a stimulus (ex. Chart) and respond to items. Recommended: To be administered in 1 session. 69 After Smarter Balanced Testing • Shred Test Materials • Student Sign-in Cards • Scratch Paper CA TAM 57 70 STAR: Paper and Pencil Testing 71 Overview of Responsibilities Materials ◦ ◦ ◦ ◦ Verify inventory and notify R&E within 24 hours of receipt No access to test materials before first day of testing Daily inventory - in/out logs Prepare for return Provide training for examiners ◦ Use DFAs for training Identify testing locations ◦ Determine Quiet Room ◦ Conduct Environmental Sweeps - instructional materials directly related to test content must be removed Keep track of students needing make-ups ◦ All information posted on R&E website – http://www.bcsd.com/research/STAR 72 CST/CMA Science - Grades 5 & 8 Students Eligible for CMA ◦ Have an IEP that specifies CMA and subject(s) ◦ May take CMA for some subjects; CST for others ◦ If CMA, not eligible to take CAPA ◦ Must be tested separately from CST Examiners and Proctors ◦ ◦ ◦ ◦ Must be district employees (certificated preferred) Includes long-term subs and daily subs Must be trained Must sign Test Security Affidavit 73 CAPA – Grades 2-8 Performance-based ◦ Administered one-on-one ◦ For students with significant cognitive disabilities unable to take CST or CMA IEP must specify CAPA level Mark CAPA Level in 7a 74 CAPA – Grades 2-8 Schools double-rate ten percent (10%) of CAPA students per level ◦ R&E randomly selects and will send to sites before Spring Break Observers must mark Observer and sign the answer document 75 CAPA – Grades 2-8 For the observer, complete the following fields on a blank document 1 3 5 6 7a 7b 11 12 Name, etc. Date of Birth Name Gender CAPA Level Grade Student ID SSID 76 Test Administration Use Directions for Administration (DFAs) ◦ Read “SAY” boxes exactly as written ◦ Students fill out Section 1 ◦ Use district-provided translations only ◦ Cannot translate test questions, prompts, or passages Additional materials ◦ No. 2 pencils with erasers ◦ Scratch paper (math and science tests only) ◦ Must collect from classrooms and shred Quiet Activities 77 Back of Answer Document A1 for R&E use, only A2 Special Conditions ◦ Grid specific condition for content area(s) A3 Accommodations and Modifications ◦ a. Grid IEP or Section 504 Plan even if accommodations or modifications not used ◦ b. Grid specific accommodation for content area(s) A4 English Learner Test Variations ◦ Grid specific variation for content area(s) 78 Scorables & Nonscorables • Transcribe damaged answer docs/booklets to a new matched version document • Place new document with Scorables • Write “Void Damaged” on damaged document and place with Nonscorables. • Pre-id students who left before first day of testing Write “Void Moved” and place with Nonscorables 79 Organizing Materials for Return • Separate Scorables from Nonscorables R&E will organize and pack 80 Documents with No Pre-Id R&E will: hand grid demographics, SGID, & Master File Sheet 81 Pick-Up Schedule CAASPP Testing Pick-up Schedule Grades 5 & 8 May 7 - 9, 2014 Please have your materials ready. We are on a very tight schedule. Wednesday May 7 Approximate Time Thursday May 8 Friday May 9 Team A Team B Team A Team B Team A Team B 8:30 Hills Sequoia Owens Intermediate Williams Jefferson Harding 9:15 Longfellow Wayside Owens Primary Horace Mann Noble Compton 10:00 Downtown Casa Loma Fremont Mt Vernon College Heights Chavez 10:45 Munsey Pauly McKinley Garza Eissler Thorner 11:30 Curran Frank West Emerson Sierra Chipman Hort Harris Evergreen Roosevelt Pioneer Nichols Stiern Washington Wm Penn Voorhies Franklin Lunch 1:00 1:45 82