CELLA 2013 Comprehensive English Language Learning Assessment TEST CHAIRPERSON TRAINING SESSION FEBRUARY 7, FEBRUARY 8, AND FEBRUARY 20, 2013 Assessment, Research, and Data Analysis Felicia Mallory, Executive Director Denetra Collins, Staff Specialist Program Overview Comprehensive English Language Learning Assessment (CELLA) is designed to meet the requirements of Title III of the NCLB. A four-skill language proficiency assessment in • Speaking • Listening • Reading • Writing Administered to all English Language Learners (ELLs) in grades K-12. TAM, p.6 Standards for Measuring English Language Learners (ELLs) Meeting Annual Measureable Achievement Objectives (AMAOs) AMAO 1 is based on progress in English language acquisition as measured in each of the three CELLA areas. Making progress is defined as either increasing a proficiency level or staying within the “Proficient” level in a specific area. AMAO 2 is based on achieving English proficiency as measured by CELLA. Achieving proficiency is defined as scoring within the proficient level in all three domains: Listening/Speaking, Writing, and Reading. AMAO 2 is established separately for four grade clusters: K-2, 3-5, 6-8, and 9-12. AMAO 3 is based on demonstrating proficiency in reading and mathematics on the FCAT. Demonstrating proficiency is defined as scoring at achievement level three or higher. Administration Features Kindergarten • Individual administration required in all four modalities Grades 1-12 • Individually administered in Speaking(One-on-One) • Group administered in Listening, Reading, and Writing TAM, pp. 7-8 Training for Coordinators and Administrators of CELLA • Speaking Administration Training (One-On-One) General training conducted by the Division of Bilingual Education and World Languages and Charter Schools Training for Special Education English Language Learners conducted by the Division of Special Education • Group Administration Training Conducted by Student Assessment and Educational Testing (SAET) Schedule of Activities Activity Date One-on-One Test Administration Training (Regional Trainings by the Division of Bilingual Education and World Languages and Charter Schools) Scheduled by Region One-on-One Test Administration Training for Special Education February 19, 2013 Delivery of Training Kits, Test Materials, and PreID labels to Schools February 11-15, 2013 Individual CELLA Administration March 4– April 5, 2013 Group CELLA Administration (including make-up) March 11 – April 5, 2013 Return all “To Be Scored” and “Not to Be Scored” (Schedule pick-up through K2 Logistics) Schedule pick up by April 12, 2013 District Coordinator Envelope Send to TDC via School Mail April 19, 2013 Complete the 2013 CELLA Customer Satisfaction Survey online at http://www.fldoe.org/aala/CELLA.asp Upon completion of testing (Test Administrators and School Coordinators) Score report delivery to schools May – June 2013 Packet, p. 2 Students To Be Tested All ELLs in grades K through 12 in membership on March 4, 2013 All former ELLs who exited from the English for Speakers of Other Languages (ESOL) program on or after September 4, 2012 Pre ID labels provided for all eligible students in membership for Survey 7 TAM, p. 8 CELLA Levels of Testing Overview Booklet levels for the 2013 CELLA are as follows: Grades K – 2 Level A Grades 3 – 5 Level B Grades 6 – 8 Level C Grades 9 – 12 Level D Test Sections – Listening, Speaking/One-on-One, Reading, and Writing Test Format: Listening: All Multiple-Choice (MC) Speaking: All Constructed-Response (CR) Reading : All MC Writing: MC and CR All students will test “on grade level” in all modalities. TAM, pp. 10-11 Test Materials: Level A Grades K - 2 The following materials are needed to administer the Level A test: • • • • Student Test Book One-on-One Prompt Book Levels A2 & B2 Listening CD Directions for Administration (DFA) TAM, p. 10 Test Materials: Level A Grades K – 2 (Reading) The Reading section contains: Core items (#1–15) • for all students in grades K–2 • read aloud by the Test Administrator • approximate testing time: 15 minutes Extension items (#16–25) • for grade 2 ONLY • completed by the student independently • approximate testing time: 20 minutes Test Materials: Level A Grades K – 2 (Writing) The Writing section contains: Core items (#1–7) • for all students grades K–2 • read aloud by the Test Administrator • approximate testing time: 15 minutes Extension items (#8–16) • for grade 2 ONLY • completed by the students independently • approximate testing time: 15 minutes Test Materials: Level A Grades K – 2 (One-on-One/Speaking) The One-on-One section is administered individually using the One-on-One Prompt Book. The student’s responses are scored by the Test Administrator, who records the scores in the student’s Level A test book. The Test Administrator must have completed the CELLA training (either the Questar/FDOEdeveloped or district-developed training). Approximate testing time: 15 minutes per student. Test Materials: Level A Grades K – 2 (One-on-One/Speaking) Test Administrators score the student’s responses and record the scores in the One-on-One Scoring Section of the test book. Page 39 of the Level A Test Book: Marking Answer Responses-Level A (Listening) Correct Incorrect ADMINISTERING LEVEL A The Stopping Rule The Stopping Rule is ONLY applicable to individually administered sections. If a student cannot answer several questions in a row, you should administer at least the first question of each type. 1. If the student is able to respond to the question even minimally, continue to administer the questions of that type. 2. If the student cannot or does not respond to the first question of the type, skip to the next type. Then fill in the NR bubbles of the corresponding questions in the One-on-One scoring area of the student’s test book. DFA, pp. 11, 22, 29, & 36 TEST MATERIALS: LEVELS B, C, and D Grades 3 - 12 Level B (Grades 3 – 5): Levels B2 Test Book Levels A2 & B2 Listening CD Level D(Grades 9 – 12): Level D2 Test Book Levels C2 & D2 Listening CD Level C (Grades 6 – 8): Level C2 Test Book Levels C2 & D2 Listening CD Levels B, C, and D: Levels B, C, or D Answer Sheet Directions for Administration TAM, pp. 10-11 ADMINISTERING LEVELS B, C, and D Reading The Reading section is divided into two parts: Part One: The student answers discrete vocabulary questions. Assesses knowledge of English vocabulary through use of synonyms, antonyms, idioms, roots, and affixes Part Two: The student reads passages and answers 4–6 questions. Assesses reading comprehension Approximate testing time: 45 minutes All reading items are multiple-choice ADMINISTERING LEVELS B, C, and D Writing The Writing section is divided into four parts: Parts One & Two: The student answers multiple-choice questions. • Grammar, Structure, Written Expression: tests knowledge of grammar • Paragraph Choices: tests elements of extended writing such as use of transitions, and topic and concluding sentences • Recognizing Errors: tests editing skills ADMINISTERING LEVELS B, C, AND D WRITING (CONTINUED ) and Parts Three & Four: The student writes sentences paragraphs. Writing Sentences: The student writes a sentence based on a picture. Writing Paragraphs: The student writes a paragraph based on a prompt. Paragraph might be descriptive, persuasive, comparative, etc., depending on grade level. Approximate testing time: 70 minutes ADMINISTERING LEVELS B, C, AND D SPEAKING The Speaking section is administered individually to all students. The student’s responses are scored by the Test Administrator, who records the scores on the Answer Sheets for Levels B, C, or D. The Test Administrator must have completed the CELLA training (either the Questar/FDOE-developed or district-developed training). Approximate testing time: 10–15 minutes per student Recording Speaking Responses for Levels B, C, and D CELLA Document Configuration by Grade Level (Review) Grade Level Test Book(s) Kindergarten (K) and Grades 1 & 2 Level A Student Test Book Level A One-on-One Prompt Book Grades 3, 4, and 5 Level B Listening, Speaking, Reading and Writing Test Book Level B Answer Sheet (Blue) Grades 6, 7, and 8 Level C Listening, Speaking, Reading and Writing Test Book Level C Answer Sheet (Tan) Grades 9, 10, 11, and 12 Level D Listening, Speaking, Reading and Writing Test Book Level D Answer Sheet (Green) Materials shown in bold are answer documents. TAM, pp. 10-11 CELLA Materials for Students’ Use Large Print/Braille Materials The Large Print version of CELLA has the same content as the standard version of the test. The Braille version of CELLA differs from the standard version of the test (Level A and some questions from other levels have been omitted). Note: The Test Administrator will also need a regular-print version of the test booklet that matches the Braille level being administered. TAM, p. 11 Preparing for Testing: Receiving Materials Verify that all materials indicated on the packing list were received. Notify the District Coordinator immediately if any of the materials on the packing list are missing. Should you need to order additional materials, please call TDC at 305-995-3743. Store materials in a secure access-restricted location. Retain ALL boxes in which materials were received for return (except the ones which contained large print and Braille documents). TAM, pp-31-32 School Packing List TAM, pp. 31-32 Shipment Summary List TAM, pp. 31-32 Box Content List TAM, pp. 31-32 Security Numbers security number TAM, p. 23 check number TAM, p. 20 Retrieving the Pre-populated Administration Record/Security Checklist https://fl-servicepoint.questarai.com/Login.aspx Select “Click Here” TAM, p. 24 Retrieving the Pre-populated Administration Record/Security Checklist Choose your school from the dropdown Choose “MiamiDade” from the dropdown Click on Security Checklist under School Files TAM, p. 24 2013 CELLA ADMINISTRATION RECORD/SECURITY CHECKLIST Pre-populated version TAM, p. 24 Preparing for Testing: Scheduling Test Sessions Schedule grade K testing and the Speaking section in grades 1-12 for individual administration • Individually administered sections may be administered at anytime during the testing window. It is recommended that CELLA group sections be administered March 11 – April 5, 2013. Administration Times for Grades K-2 Level A Approx. Time in Grade K (individually) Approx. Time in Grade 1 Approx. Time in Grade 2 15 min. 15 min. 15 min. Listening 15 min. 15 min. 15 min. Reading 15 min. 15 min. 35 min. Writing 15 min. 15 min. 30 min. Speaking (One-on-One) Sections shown in bold require more time because they contain additional, “extension” items only for grade 2 students. TAM, p. 7 Administration Times for Grades 3-12 Levels B, C, or D Approx. Time Speaking 10-15 min. Listening 25 min. Reading 45 min. Writing 70 min. TAM, p. 8 Preparing for Testing: Training Test Administrators Review the Test Administration Manual Review the relevant sections from the Directions for Administration (DFA) Have test administrators familiarize themselves with the Listening CDs and equipment for playing them Remind test administrators of the State’s and District’s security policies and procedures (including those concerning electronic devices) After the training, collect signed 2013 CELLA Administration and Security Agreement forms located in Appendix C TAM, pp. 32-36 Preparing for Testing: Accommodations for all ELLs Review the list of accommodations in the DFA . The following accommodations ARE permissible for all ELLs: Flexible timing: CELLA is not a timed test, sufficient time should be given to complete the test. Assistance in heritage language: Directions may be translated into heritage language on an individual basis, if requested. TAM (Appendix A), pp. 53-59 Preparing for Testing: Accommodations for all ELLs The following accommodations ARE NOT permissible for any student: No English-to-heritage language translation dictionary or heritage language-to-English translation dictionary. No help in the ELLs’ heritage language on specific test questions NOR translate any part of the test other than the directions. Test Administrators may NOT translate directions to entire classes, ONLY if all ELLs in the classroom speak the same heritage language. TAM (Appendix A), pp. 54-55 Preparing for Testing: Accommodations for SPED Students Students with a current IEP or 504 plans are eligible for accommodations based on their individual needs. An accommodation for a student must be explicitly permitted on the IEP or 504 plan. • Presentation of Test Materials • Responding to Questions • Test Scheduling • Test Setting • Assistive Devices Refer to the Visually Impaired and Hearing Impaired section in the DFA for allowable accommodations and/or exemptions. TAM, pp. 55-59 Preparation of Testing Materials Pre-ID labels are provided. Only the Test Chairperson or the person designated by the site administrator may open the packages and affix labels. Pre-ID labels should be affixed only to answer documents; only one answer document per student: Level A Student Test Book, or Level B, C, & D Answer Sheets TAM, p. 19 Preparation of Testing Materials Level A Student Test Book TAM, p. 19 Levels B, C, D Answer Sheets Preparation of Testing Materials (continued) ALL of the following information MUST be correct for the student label to be used: 1. 2. 3. 4. 5. 6. Student’s Last Name Student ID Number District Number School Number (code) Grade Level ELL Status TAM, p. 20 Pre-ID Label Used: Information to be Completed Box 1 Box 2 Box 3 Box 4 Box 5 Box 9 Box 21 Box 22 Box 23 Box 24 Student Name Test Administrator Name School Name School Number District Name (MDCPS) and Number (13) Test Date Do Not Score (DNS)/ UNDO (if applicable) Accommodations (if applicable) Receives Special Services (if applicable) Program Participation TAM, p. 20 and/or pp. 14-18 (chart), Packet , pp. 6-7 No Pre-ID Label Used: Information to be Completed If a student has no pre-identified label, the following additional information MUST be gridded on all student answer documents: Students or school staff Boxes 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, and 16 School staff only Boxes 2, 12, 17, 18, 19, 20, 21, 22, 23, and 24 Student demographics provided on the student list provided by SAET are indicated in bold. TAM, p.13; Packet pp. 6-7 Student List for Gridding Packet, p. 9 Gridding Demographic Information (No Pre ID Label) Fill in the bubble for the following fields as indicated below using the student list provided by SAET : Box 16 - Home Language Box 17 – Primary Exceptionality Box 18 - Length of Time Enrolled in ESOL Program Box 19 - Title III YES/NO Box 20 – Date of Home Language Survey TAM, pp. 13-18 , Packet, p. 7 Preparation of Testing Materials Non-Pre ID Label Used: Other Fields Fill in the bubble for the following fields as indicated below: Box 23 - Receives Special Services F/R Lunch if “Y” is indicated on the student list Migrant if “Y” Box 24 - Program Participation ESOL if “Y” (do not mark if student is former ELL) Dual Language Program – determined by school Packet, p. 7 Test Security Policies and Procedures Florida Test Security Statute 1008.24 and Florida State Board of Education – Test Security Rule 6A – 10.042 M-DCPS: Standards, Guidelines, and Procedures for Test Administration and Test Security http://oada.dadeschools.net/TestChairInfo/29testsecuritymanual.pdf The security of all test materials must be maintained before, during, and after the test administration. Under no circumstances are students permitted to handle secure materials before or after the test administration. Test administrators MUST NOT administer CELLA to their family members. TAM, pp. 66-70 Distribution and Tracking of Materials Distribute and collect all One-on-One test materials daily. Distribute all testing materials for group administration just prior to test sessions, collect immediately afterwards, and store in a secure location. Distribute copies of the 2013 CELLA Administration Record/Security Checklist https://fl-servicepoint.questarai.com TAM, p. 24 and pp. 35-36 Distribution and Tracking of Materials Individually Administered Sessions Administrators need: Test Level/Grade Student Material Needed Teacher Material Needed Level A / Kindergarten (all four sections) Level A Student Test Book Directions for Administration (DFA) Test Administration Manual (TAM) One-on-One Prompt Book Listening CD (Levels A & B) CD Player/ computer with speakers Level A / Grades 1 – 2 (One-on-One/Speaking) Level A Student Test Book DFA TAM One-on-One Prompt Book Levels B-D / Grades 3 – 12 (Speaking) Listening, Speaking, Reading & Writing Student Test Book (Appropriate level) Levels B,C, or D – Answer Sheet DFA TAM TAM, pp. 9 - 10 Distribution and Tracking of Materials Group Administration for Grades 1 – 12 (Listening) Administrators need: Test Level/Grade Student Material Needed Teacher Material Needed Level A / Grades 1 – 2 Level A Student Test Book Directions for Administration (DFA) Test Administration Manual (TAM) Listening CD (Levels A & B) CD Player/ Computer with speakers Level B / Grades 3 - 5 Level C/ Grades 6 - 8 Level D/ Grades 9 - 12 Listening, Speaking, Reading & Writing Student Test Book (Appropriate level) Levels B, C, or D – Answer Sheet DFA & TAM Listening CD (Appropriate Level) CD Player/ Computer with speakers TAM, pp. 9 - 10 Distribution and Tracking of Materials Group Administration for Grades 1 – 12 (Reading and Writing) Administrators need: Test Level/Grade Student Material Needed Teacher Material Needed Level A / Grades 1 – 2 Level A Student Test Book Directions for Administration (DFA) Test Administration Manual (TAM) Level B / Grades 3 - 5 Level C/ Grades 6 - 8 Level D/ Grades 9 - 12 Listening, Speaking, Reading & Writing Student Test Book (Appropriate level) Levels B, C, or D – Answer Sheet DFA TAM TAM, pp. 9-10 Administering the Test Rooms used for testing must: be free from distractions have comfortable seating have good lighting have adequate ventilation have sufficient workspace Remove or cover all visual aids in the testing rooms. Make sure that: test administrators have a copy of the TAM and DFA required administration information is collected for each student test scripts are read verbatim there is sufficient space between students so that students cannot view other student’s answer sheets students and assessors record all answers using a #2 pencil TAM, pp. 33 - 36 Administering the Test The Listening portion must be administered by playing the Listening CD. Test administrators are required to actively monitor students during a test session. Test administrators are required to record all required administration information on the CELLA Administration Record/Security Checklist. Test administrators are required to maintain an accurate 2013 CELLA Security Log. Test administrators and proctors must sign the 2013 CELLA Administration and Security Agreement. Test administrators must transcribe/record students’ exact responses from the large print or Braille version of the test to the regular print version. DFA, pp. 4 - 7, Appendix C Security Log TAM, pp. 35 & Appendix C Test Invalidation Procedures From Level B,C, or D Answer Sheet Box #21 DO NOT SCORE – Contains DNS and UNDO bubbles for each test section TAM, pp. 25 - 27 Test Invalidation Procedures The appropriate DNS bubble MUST be gridded if: 1) The document is defective and USED. 2) Any of the following information on the PreID label is INCORRECT and affixed to a USED document. - Last Name - Student ID Number - District Number - School Number - Grade Level - ELL Status 3) A student label has been placed over another PreID label on a USED document. 4) A student has an electronic device during testing. 5) A student is caught cheating. TAM, pp. 25 - 27 Test Invalidation Procedures The appropriate DNS bubble MAY be gridded if: 1) 2) 3) 4) A student becomes ill and is unable to complete the session. A student is not allowed the correct amount of time to finish. A student is given an accommodation NOT allowed in CELLA. A student is given an accommodation NOT indicated on the student’s IEP, Section 504 plan, or ELL Plan. 5) A student was not provided an allowable accommodation which was indicated on the student’s IEP, Section 504 plan, or ELL Plan. 6) A student is disruptive during testing. Used answer documents with gridded DNS or UNDO bubbles should be submitted with TO BE SCORED materials. TAM, p. 27 Test Invalidation Procedures It is not necessary to invalidate an answer document if: 1) The preidentified document belongs to a student who has withdrawn prior to testing. 2) The preidentified document belongs to a student who has been absent during the entire testing window. 3) The document is defective and UNUSED. 4) The preidentification information document is UNUSED. TAM, p. 26 is incorrect and the Test Invalidation Procedures Defective Materials If a defective test book is identified during testing, give the student a replacement test book. If the defective book is a Level A , upon completion of the test, the student must (under supervision of the test administrator) transcribe the exact answers to the appropriate replacement document. If the defective book is Level B, C, or D, the student will continue using the Levels B, C, or D answer sheet. If a test book is defective or becomes soiled, notify Student Assessment and Educational Testing at 305-995-7520. TAM, p. 27 Preparing Materials for Return All 2013 CELLA materials must be returned to Questar as follows: PINK return label – TO BE SCORED answer documents YELLOW return label – NOT TO BE SCORED materials TAM, pp. 41 – 45, Packet, pp. 10 - 11 Preparing Materials for Return Preparing TO BE SCORED Test Materials Verify that all distributed secure materials have been returned by the test administrators. TO BE SCORED (Pink label) materials include: Used Level A Test Books Used Levels B, C, and/or D Answer Sheets Documents with gridded DNS bubbles (packed at the bottom of the stack) TAM, pp. 40 - 43 Preparing Materials for Return Preparing TO BE SCORED Test Materials Complete the Document Count Form for TO BE SCORED materials (PURPLE) NOTE: Complete one form per grade level and be sure to include documents with gridded DNS bubbles in your counts. TAM, pp. 40 - 43 Preparing Materials for Return Preparing TO BE SCORED Test Materials Complete the School Return Summary for To Be Scored materials (RED) NOTE: ONE PER SCHOOL TAM, pp. 41 - 43 Preparing Materials for Return TO BE SCORED Test Materials Place the School Return Summary on top of the materials in Box 1. Affix the PINK label to the top of the box At the bottom of each PINK label, write the number of the box and the total number of boxes of TO BE SCORED materials that you are returning. (Ex. 1 of X, 2 of X, 3 of X, etc.) TAM, pp. 41– 43 & Packet, pp. 11 Preparing Materials for Virtual School Students Preparing Test Materials for FLVS or Miami-Dade Online Complete the district and school number on the answer document as appropriate (see chart below). District Name and Number Florida Virtual School (71) Miami-Dade (13) School Name and Number Florida Virtual School Full-Time K-8 (0300) Florida Virtual School Full-Time 9-12 (0400) Miami-Dade Online (7001) TAM, p. 13 Preparing Materials for Return for Virtual School Students Preparing TO BE SCORED Test Materials for FLVS or Miami-Dade Online for Return Follow the materials return instructions that begin on page 40 of the TAM. Use the header sheets sent via email - Complete the Document County Form(s) and School Return Summary forms sent via email using the appropriate district and school number. - Make copies for your records. - Package the To Be Scored materials, Document Count Form(s), and School Return Summary for the virtual school students in a large envelope and place on top of your school’s materials in Box 1 of your PINK labeled TO BE SCORED materials. TAM, p. 40 Preparing Materials for Return Preparing NOT TO BE SCORED Test Materials NOT TO BE SCORED materials include: Unused Level A Test Books (including used and unused large print test books) Level A One-on-One Prompt Books Used and Unused Levels B, C, and D Test Books (including large print and Braille) Unused Levels B, C, or D Answer Sheets Listening CD’s - Levels A2 & B2 and Levels C2 & D2 Directions for Administration Training CD’s for Speaking Defective Materials (if applicable) TAM, pp. 44 – 45 and 47, Packet, p. 11 Preparing Materials for Return NOT TO BE SCORED Test Materials Place a copy of the CELLA Administration Record/Security Checklist on top of the materials in Box 1. Affix the YELLOW label to the top of the box. At the bottom of each YELLOW label, write the number of the box and the total number of boxes of NOT TO BE SCORED materials that you are returning. (Ex. 1 of X, 2 of X, 3 of X, etc.) TAM, pp. 44 – 45 and 47, Packet, p. 11 Preparing Materials for Return Destroy at School Site Test Administration Manuals* Unused Paper Bands Unused Document Count Forms and School Return Summaries Unused Questar Return Labels Remaining PreID Student Rosters and PreID Student Labels *Keep test administration manuals until results have been provided, then destroy at the school site. TAM, pp. 45 & 48 Preparing Materials for Return District Coordinator ONLY Envelope Items to include: Original signed CELLA Administration Record/Security Checklist Original 2013 CELLA Security Logs Copy of School Return Summary for the To Be Scored Materials Copy of Document Count Form for To Be Scored Materials by Grade Accounting for All 2013 CELLA Materials School Procedural Checklist Bill of Lading On the front of the envelope write “DISTRICT COORDINATOR ONLY” Signed Administration and Security Agreements should be kept on file at your school site. TAM, p. 45 & Packet, p. 11 Scheduling Pick-Up through K2 Logistics Schedule your CELLA pick-up 24 - 48 hours in advance Access the online booking site at www.k2assessments.com Follow instructions on page 50 of the Test Administration Manual Check your email to ensure that a confirmation was issued by K2 Logistics Ensure that you obtain the Bill of Lading provided by K2 Logistics upon pick-up TAM, pp. 50 - 51 Review of the Return of CELLA 2013 Test Materials Materials should be returned March 11, 2013 – April 12, 2013 K2 Logistics To Be Scored Materials Not To Be Scored Materials School Mail: 9038 – Test Distribution Center District Coordinator Envelope TAM, pp. 50-51 & Packet, pp. 10-11 Comment Form 2013 CELLA Customer Satisfaction Survey Please access the following site after testing has been completed. http://www.fldoe.org/aala/CELLA.asp Receiving 2013 CELLA Student Score Reports Score reports will be delivered to schools in June Questar provides schools with only two copies of the CELLA Student Report Reports should be distributed in the following manner: • File one copy in the student’s LEP folder that goes in the cumulative records • Send one copy home with the student Contact Information STUDENT ASSESSMENT AND EDUCATIONAL TESTING Felicia Mallory Fmallory@dadeschools.net Denetra Collins Collinsd@dadeschools.net 305-995-7520 Fax – 305-995-7522