2011 STAR Order Management Workshop October 20, 2010 Start Time 9:00 a.m. Presenters: Mark Hansen and Michael McDaniel October 2010 2011 STAR Order Management || Slide 1 Objectives District STAR coordinators will identify: • Which materials to order • How to order test materials October 2010 2011 STAR Order Management || Slide 2 Agenda Major changes and reminders for 2011 ordering • Eligibility for tests CMA CAPA STS • What needs to be ordered? Discussed by grade • Eligibility for end-of-course (EOC) California Standards Tests (CSTs) Mathematics Science World History • Eligibility for EOC California Modified Assessment (CMA) Mathematics (Algebra I and Geometry) • Eligibility for EOC Standards-based Tests in Spanish (STS) Mathematics (Algebra I and Geometry) October 2010 2011 STAR Order Management || Slide 3 Agenda • Ordering materials Overview of process Ordering options – Online ordering – Batch file submission October 2010 2011 STAR Order Management || Slide 4 What’s New • • The California Writing Standards Test (CST for Writing) and the California Modified Writing Standards Test (CMA for Writing) for grade four have been reinstated Three new CMA multiple-choice tests will be available for CMAeligible students: Grade ten: The CMA for English–Language Arts (ELA) (Grade 10) Grade eleven: The CMA for ELA (Grade 11) Grades eight through eleven: The CMA for Geometry; eligible students must have completed a Geometry course in the 2010 summer session or will complete one during the 2010–11 school year. • Three additional braille versions of CMA tests are available: CMA for ELA (Grade 9)—Order using the Order Management module CMA for Algebra I—Order using the Order Management module CMA for Life Science (Grade 10)—Order using the supplemental order process only October 2010 2011 STAR Order Management || Slide 5 What’s New (Cont.) • Additional version 1 test booklets are no longer available to order separately. This column has been removed from various screens in the graphical user interface. • The “Audio CDs” column has been removed from various screens in the graphical user interface. • A new field on the Approve Order screen allows entry of text to explain an order for which there was a warning message(s). • In the Test Admin Summary, if the calculated Pre-ID cutoff date for a writing or multiple-choice administration falls on January 1, the date will be automatically recalculated to fall on the next school district working day. October 2010 2011 STAR Order Management || Slide 6 District Preparation Checklist Obtain school calendars Including nonpublic, nonsectarian schools (NPS) Obtain warehouse calendar Identify dependent charter schools Identify NPSs providing special education services Collect order numbers from test site coordinators October 2010 2011 STAR Order Management || Slide 7 Site Preparation Checklist: Give to District STAR Coordinator School calendar(s) Number and type of test booklets required CST – Grade-level tests ELA (grades 2–11) Mathematics (grades 2–7) Science (grades 5 and 8); Life Science (grade 10) History–Social Science (grades 8 and 11) Based on individualized education programs (IEPs) – CMA ELA (grades 3–11) Mathematics (grades 3–7) Science (grades 5 and 8); Life Science (grade 10) Algebra I (grades 7–11) Geometry (grades 8–11) October 2010 2011 STAR Order Management || Slide 8 Site Preparation Checklist: Give to District STAR Coordinator CAPA Special versions Large print available for all tests Braille » CSTs—All tests » CMA—All tests except for grades 10 and 11 ELA, Geometry » STS—All tests October 2010 2011 STAR Order Management || Slide 9 Site Preparation Checklist: Give to District Coordinator (cont.) Based on mathematics department chair input – CST for EOC mathematics Algebra I (grades 7–11) Other EOC mathematics (grades 8–11) – STS for EOC Mathematics Algebra (grades 7–11) Geometry (grades 8–11) Based on science department chair input – CST for EOC science Grades 9–11 Based on history–social science department chair input – CST for EOC World History Grades 9–11 Based on English learner status – STS Grades 2–11 October 2010 2011 STAR Order Management || Slide 10 Site Preparation Checklist: Give to District Coordinator (cont.) Number of additional DFAs needed for CST, CMA, and STS: CMA DFAs calculated 1 per 10 students CST/STS DFAs calculated 1 per 20 students October 2010 2011 STAR Order Management || Slide 11 General Rules for Ordering • Do NOT over-order Districts are sent 5% overage Schools are sent 10% overage • Be realistic with quantities Especially large print, braille – These materials are expensive! Mobile populations (county offices) – Average prior-year orders, then use “Supplemental Order” process if necessary October 2010 2011 STAR Order Management || Slide 12 California Modified Assessment Who Takes the CMA? • Available in the following grades: – – – • 2. 3. • Grades 8–11 for EOC Geometry Grades 5, 8, and 10 for science Eligible students shall, in part: 1. • Grades 3–11 for ELA – Grades 3–7 for mathematics – Grades 7–11 for EOC Algebra I Have an IEP that specifies that they take the CMA for one or more subjects; and Scored below basic or far below basic in a previous CST administration (students may have taken the CST with modifications, or may have taken CMA test(s) previously); and Are not eligible to take the California Alternate Performance Assessment (CAPA). Student may take the CMA for one or more content areas, and the CST for others, e.g. the CMA for Mathematics and the CST for ELA See the complete CMA participation criteria at http://www.cde.ca.gov/tc/tg/sr/participcriteria.asp October 2010 2011 STAR Order Management || Slide 13 California Alternate Performance Assessment Who Takes the CAPA? • Student must have an IEP. • Student’s IEP must specify CAPA and whether the student should take the grade-assigned CAPA level or CAPA Level I. Note: Student counts by CAPA level are not necessary during the ordering process. October 2010 2011 STAR Order Management || Slide 14 What needs to be ordered? CAPA • Examiner’s Manual: One for each examiner Can be used for several students All CAPA levels covered One version per school district • Answer documents One per student taking the CAPA All CAPA levels covered 10% double-rated (use overage) – Will be discussed in the Pre-Test Workshop October 2010 2011 STAR Order Management || Slide 15 CAPA Manipulatives • Not ordered through ETS • School districts need to assemble manipulatives • Dependent upon CAPA version – Assigned by ETS (available in early December) CAPA level – Determined by IEP team Designate the use of Level I or gradeappropriate level • Manipulatives list indicates the following: Single-use items Multi-use items October 2010 2011 STAR Order Management || Slide 16 Standards-based Tests in Spanish Who takes the STS? • • October 2010 Available in grades 2–11 Spanish-speaking English learners (EL) – Required to take STS if they: Are receiving instruction in Spanish OR Will have attended U.S. schools less than 12 cumulative months on the first day of testing (regardless of whether they receive instruction in English or Spanish) – At the option of the school district: Any other Spanish-speaking English learners Not to be administered to non-ELs – E.g. English-only, I-FEP, R-FEP students – English learners whose primary language is not Spanish 2011 STAR Order Management || Slide 17 What Needs to Be Ordered, by Grade October 2010 2011 STAR Order Management || Slide 18 What needs to be ordered? Grade 2 • CST grade-level test booklet for each student to be tested Contains: – CST for English–Language Arts – CST for Mathematics • STS grade-level test booklet for each eligible student Contains: – STS for Reading/Language Arts – STS for Mathematics October 2010 2011 STAR Order Management || Slide 19 What needs to be ordered? Grade 3 • CST grade-level test booklet for each student to be tested with one or more CST CST test booklet contains: – CST for English–Language Arts – CST for Mathematics • CMA test booklet for each eligible student Contains: – CMA for English–Language Arts – CMA for Mathematics • STS grade-level test booklet for each eligible student Contains: – STS for Reading/Language Arts – STS for Mathematics October 2010 2011 STAR Order Management || Slide 20 What needs to be ordered? Grade 4 • CST grade-level test booklet for each student to be tested with one or more CST Contains: – CST for English–Language Arts – CST for Mathematics • CMA grade-level test booklet for each eligible student Contains: – CMA for English–Language Arts – CMA for Mathematics • STS grade-level test booklet for each eligible student Contains: – STS for Reading/Language Arts – STS for Mathematics October 2010 2011 STAR Order Management || Slide 21 What needs to be ordered? Grade 4 (cont.) • CST for Writing test booklet for all students taking the CST for ELA • CMA for Writing test booklet for all students taking the CMA for ELA October 2010 2011 STAR Order Management || Slide 22 What needs to be ordered? Grade 5 • • • October 2010 CST grade-level test booklet for each student to be tested with one or more CST Contains: – CST for English–Language Arts – CST for Mathematics – CST for Science CMA grade-level test booklet for each eligible student Contains: – CMA for English–Language Arts – CMA for Mathematics – CMA for Science STS grade-level test booklet for each eligible student Contains: – STS for Reading/Language Arts – STS for Mathematics 2011 STAR Order Management || Slide 23 What needs to be ordered? Grade 6 • CST grade-level test booklet for each student to be tested with one or more CST Contains: – CST for English–Language Arts – CST for Mathematics • CMA grade-level test booklet for each student eligible to be tested with one or more CMA Contains: – CMA for English–Language Arts – CMA for Mathematics • STS grade-level test booklet for each eligible student Contains: – STS for Reading/Language Arts – STS for Mathematics October 2010 2011 STAR Order Management || Slide 24 What needs to be ordered? Grade 7 • CST grade-level test booklet for each student to be tested Contains: – CST for English–Language Arts – CST for Mathematics For students not completing an Algebra I course • CMA grade-level test booklet for each student eligible to be tested with one or more CMA Contains: – CMA for English–Language Arts – CMA for Mathematics For students not completing an Algebra I course • CST or CMA Algebra I test booklet For students completing a standards-aligned Algebra I course October 2010 2011 STAR Order Management || Slide 25 What needs to be ordered? Grade 7 (cont.) • STS grade-level test booklet for each eligible student Contains: – STS for Reading/Language Arts – STS for Mathematics (grade-level) – No STS for Writing test • Algebra I STS test booklet For students completing a standards-aligned Algebra I course • CST for Writing test booklet for all students taking the CST for ELA • CMA for Writing test booklet for all students taking the CMA for ELA October 2010 2011 STAR Order Management || Slide 26 What needs to be ordered? Grade 8 • CST grade-level test booklet for each student to be tested Contains: – CST for English–Language Arts – CST for Science – CST for History–Social Science • CMA grade-level test booklet for each eligible student Contains: – CMA for English–Language Arts – CMA for Science October 2010 2011 STAR Order Management || Slide 27 What needs to be ordered? Grade 8 (cont.) • EOC CST or CMA mathematics test booklet All grade 8 students MUST take a mathematics test – CST or CMA for Algebra I – CST for General Mathematics – CST or CMA for Geometry – CST for other EOC mathematics – Courses aligned with the CST or CMA blueprint October 2010 2011 STAR Order Management || Slide 28 What needs to be ordered? Grade 8 (cont.) • STS grade-level test booklet for each eligible student Contains: – STS for Reading/Language Arts • EOC mathematics STS test booklet For students completing a standards-aligned Algebra I or Geometry course October 2010 2011 STAR Order Management || Slide 29 What needs to be ordered? Grade 9 • CST grade-level test booklet for each student to be tested Contains: – CST for English–Language Arts • CMA grade-level test booklet for each eligible student Contains: – CMA for English–Language Arts • EOC mathematics CST or CMA test booklet All grade 9 students MUST take a mathematics test Contains: – CST or CMA for Algebra I or Geometry – CST for General Mathematics – CST for other EOC mathematics Courses aligned with the CST or CMA blueprint October 2010 2011 STAR Order Management || Slide 30 What needs to be ordered? Grade 9 (cont.) • EOC science CST test booklet Contains: – Biology, or Earth Science, or Physics, etc. – Courses aligned with the CST blueprint • CST for EOC World History test booklet Course aligned with the CST blueprint • STS grade-level test booklet for each eligible student Contains: – STS for Reading/Language Arts • EOC mathematics STS test booklet for each student eligible to take an end-of-course test Contains: – STS for Algebra I or STS for Geometry October 2010 2011 STAR Order Management || Slide 31 What needs to be ordered? Grade 10 • CST grade-level test booklet for each student to be tested Contains: – CST for English–Language Arts – CST for Life Science • CMA grade-level test booklet for each eligible student Contains: – CMA for English–Language Arts – CMA for Life Science • EOC mathematics CST or CMA test booklet Contains: – CST or CMA for Algebra I or Geometry – CST for Algebra II, or CST for Geometry, and so forth Courses aligned with the CST or CMA blueprint October 2010 2011 STAR Order Management || Slide 32 What needs to be ordered? Grade 10 (cont.) • EOC science CST test booklet Contains: – Earth Science, or Biology, or Physics, etc. – Courses aligned with the CST blueprint • CST for EOC World History test booklet Course aligned with the CST blueprint • STS grade-level test booklet for each eligible student Contains: – STS for Reading/Language Arts • EOC mathematics STS test booklet for each student eligible to take an end-of-course test Contains: – STS for Algebra I or STS for Geometry October 2010 2011 STAR Order Management || Slide 33 What needs to be ordered? Grade 11 • • • • CST grade-level test booklet for each student to be tested Contains: – CST for English–Language Arts – CST for U.S. History CMA grade-level test booklet for each eligible student Contains: – CMA for English–Language Arts EOC for mathematics CST or CMA test booklet Contains: – CST or CMA for Algebra I or Geometry – CST for Algebra II, or CST for Geometry, and so forth Courses aligned with the CST or CMA blueprint EOC science CST test booklet Contains: – Earth Science or Biology or Physics, and so forth Courses aligned with the CST blueprint October 2010 2011 STAR Order Management || Slide 34 What needs to be ordered? Grade 11 (cont.) • CST for EOC World History test booklet Course aligned with the CST blueprint • STS grade-level test booklet for each eligible student Contains: – STS for Reading/Language Arts • STS for EOC mathematics test booklet for each student eligible to take an end-of-course test Contains: – STS for Algebra I or STS for Geometry • EAP Essay test booklet October 2010 2011 STAR Order Management || Slide 35 Who Takes CST or CMA for EOC Mathematics? • • Grades 2–9 ALL students take a mathematics test Grades 10 and 11 Students may take no mathematics test if – – – • They are not enrolled in a mathematics course and did not previously complete Algebra II or a course aligned with the Integrated Mathematics 3 test blueprint; or They are enrolled in the first year of a two-year mathematics course; or They are enrolled in General Mathematics or Business Mathematics. Determine which mathematics test will be administered in grades 7–11 Review state blueprints available at: http://www.cde.ca.gov/ta/tg/sr/resources.asp Review curriculum Review guidelines in the Order Management Manual CMA students in grades 8 and 9 should take the CMA for Algebra I, Geometry, an EOC mathematics CST, or the CST for General Mathematics October 2010 2011 STAR Order Management || Slide 36 Who takes the STS for EOC Mathematics? • The STS for Algebra I is available for students in grades 7–11 • The STS for Geometry is available for students in grades 8–11 Must be – Spanish-speaking English learners (EL) – Not to be administered to non-ELs such as English-only, I-FEP, R-FEP • Same EOC rules as CST/CMA Completing the course or completed it in the summer of 2010 October 2010 October 2010 2011 STAR Order Management || Slide 37 Which Students Take the CST for General Mathematics? • Students in grades 8 or 9 only who are Not taking a mathematics course; or Taking any mathematics course prior to Algebra I (e.g. Pre-Algebra, Algebra Readiness, Business mathematics, and so forth); or Not in the last year of a multi-year Algebra I course October 2010 2011 STAR Order Management || Slide 38 Which Students Take Algebra I? • Students in grades 7–11 who are completing Algebra I or completed it in the summer of 2010 • Grade 7 students not completing Algebra I take the grade 7 mathematics test October 2010 2011 STAR Order Management || Slide 39 Which Students Take Summative High School Mathematics? • Students in grades nine through eleven who: Previously completed Algebra I, Algebra II, and Geometry and completed an advanced mathematics course in the 2010 summer session or will complete it during the 2010– 11 school year; or Previously completed a course aligned with the Integrated Mathematics 3 test blueprint and completed an advanced mathematics course in the 2010 summer session or will complete it during the 2010–11 school year; or Completed Algebra I, Algebra II, and Geometry courses in previous years and are taking no mathematics course this year; or Completed a course aligned with the Integrated Mathematics 3 test blueprint in a previous year and are taking no mathematics course this year. October 2010 2011 STAR Order Management || Slide 40 Example 1 • Carol 2010 – Grade 9 student – Previously completed Algebra I and Geometry courses and took an Algebra II course in 2010 – Took the CST for Algebra II 2011 – Grade 10 student – Not studying mathematics – Takes CST for Summative High School Mathematics October 2010 2011 STAR Order Management || Slide 41 Example 2 • Robert 2010 – Grade 9 student – Took Geometry (but has not taken Algebra II) – Took the CST for Geometry 2011 – Grade 10 student – Not studying mathematics – Takes no mathematics test October 2010 2011 STAR Order Management || Slide 42 Example 3 • Juan 2010 – Grade 10 student – Took an advanced mathematics class – Took the CST for Summative High School Mathematics 2011 – Grade 11 student – Not studying mathematics – Takes the CST for Summative High School Mathematics October 2010 2011 STAR Order Management || Slide 43 Example 4 • Juliet 2010 – Grade 10 student – Took an advanced mathematics course – Took the CST for Summative High School Mathematics 2011 – Grade 11 student – Studying Calculus – Takes the CST for Summative High School Mathematics October 2010 2011 STAR Order Management || Slide 44 Which students take End-of-Course Science? • Students in grades 9–11 Completing or completed in summer of 2010 California standards-aligned science course – Review state blueprints – Review curriculum • All students in grade 10 take the Life Science test Test printed in the grade 10 test booklet (both CST and CMA) Grade 10 students will also take an EOC science CST if taking a standards-aligned science course. October 2010 2011 STAR Order Management || Slide 45 Which students take End-of-Course World History? • Students in grades 9–11 Completed the World History course in the 2010 summer session or will complete it during the 2010–11 school year • All students in grade 11 will take the CST for U.S. History The test is printed in the grade 11 test booklet Grade 11 students will also take the CST for World History if they meet the criteria above October 2010 2011 STAR Order Management || Slide 46 EAP Essay • Grade 11 Order for all students Must be taken in conjunction with additional ELA multiple-choice questions to achieve an exemption from the CSU and participating California Community Colleges. Order entered on the Early Assessment Program administration (EP01) Approve order separately October 2010 2011 STAR Order Management || Slide 47 15-minute break for everyone! October 2010 2011 STAR Order Management || Slide 48 Test Administrations • Multiple-choice Up to 8 administrations allowed MC01 – MC08 Each administration should have different date for “First day of testing” Each administration’s order needs to be approved separately – • Enter orders for ALL administrations before starting approval process – Report for 2011 provides list of schools without orders – Report generated when each order approved Writing Application (grades 4 and 7) CST and CMA Administration WA01 – March 1 with makeup on March 2 if in session on March 1 Administration WA02 – May 3 with makeup on May 4 ONLY if not in session on March 1 Enter and approve order separately from multiple-choice • Early Assessment Program (EAP) Essay Administration EP01 – March 1 through March 31 Enter and approve order separately from multiple choice October 2010 2011 STAR Order Management || Slide 49 Ordering Procedures 1. Set up test administration dates • See the Webcast “2010 STAR Test Administration Setup” • Manual posted at www.startest.org/sms.html 2. Confirm “Profile” information • Primary Shipping Address • Secondary Shipping Address (optional) – Can be used for Supplemental Order delivery • Report Shipping Address (optional) – Used to deliver Reports in summer 2011 – Address must be set up before June 1, 2011 October 2010 2011 STAR Order Management || Slide 50 Ordering Procedures 1. 2. October 2010 Enter order quantities for each Multiple-Choice, Writing Application, and the Early Assessment Program administration. • Screen entry or • Produce file containing order quantities and upload Approve order for each administration. 2011 STAR Order Management || Slide 51 Software Demonstration 1. Verify Profile 2. Enter order online 3. Approve order October 2010 2011 STAR Order Management || Slide 52 October 2010 2011 STAR Order Management || Slide 53 October 2010 2011 STAR Order Management || Slide 54 October 2010 2011 STAR Order Management || Slide 55 October 2010 2011 STAR Order Management || Slide 56 October 2010 2011 STAR Order Management || Slide 57 October 2010 2011 STAR Order Management || Slide 58 October 2010 2011 STAR Order Management || Slide 59 October 2010 2011 STAR Order Management || Slide 60 October 2010 2011 STAR Order Management || Slide 61 October 2010 2011 STAR Order Management || Slide 62 October 2010 2011 STAR Order Management || Slide 63 School district cannot approve order when errors still exist. October 2010 2011 STAR Order Management || Slide 64 October 2010 2011 STAR Order Management || Slide 65 October 2010 2011 STAR Order Management || Slide 66 Book count warnings Schools with no orders School district can still approve order when warnings exist. October 2010 2011 STAR Order Management || Slide 67 October 2010 2011 STAR Order Management || Slide 68 October 2010 2011 STAR Order Management || Slide 69 October 2010 2011 STAR Order Management || Slide 70 October 2010 2011 STAR Order Management || Slide 71 October 2010 2011 STAR Order Management || Slide 72 STAR Order Management File upload option October 2010 2011 STAR Order Management || Slide 73 Changes to Batch Procedure • Do not need to zero-fill quantities Numbers must be right-justified E.g., 0012 may also be entered as _ _12 • No special sequence of records required October 2010 2011 STAR Order Management || Slide 74 Order File Upload Format • Included in Order Management Manual • Fixed-length field; text file only • Orders for one administration only per file October 2010 2011 STAR Order Management || Slide 75 Sample File October 2010 October 2010 STAR Order Management 2011 STAR Order Management || Slide 76 76 Sample E-Mail October 2010 2011 STAR Order Management || Slide 77 Review and Approve • Review orders using Online screens and/or Printouts generated by STAR Management System • Adjust orders where necessary using Online entry to adjust quantities • Approve orders All orders entered through “File load” need to be approved by the district STAR coordinator October 2010 2011 STAR Order Management || Slide 78 In Conclusion October 2010 2011 STAR Order Management || Slide 79 Excessive Materials Ordered Charges • 90% of the value of materials ordered minus value of materials used • Be careful ordering! Do not over-order Large print and braille materials are very expensive • No excessive order charges for EAP Essay test books • ETS will invoice approximately December 2011 October 2010 2011 STAR Order Management || Slide 80 Excessive Materials Ordered Charges • Value of materials ordered ($x) Does not include overages Order quantities will be adjusted based on number of Pre-ID records • Value of materials used ($y) • If (x-y) ≤ 10%(x) then no charges will apply • If (x-y) > 10%(x) then charges will apply October 2010 2011 STAR Order Management || Slide 81 Excessive Materials Ordered Charges Example 1 • • • • • October 2010 Value of materials ordered = $5,000 Value of materials used = $4,600 Difference between ordered and used = $400 10% of materials ordered = $500 Difference ($400) is less than 10% of materials ordered ($500) No charges will apply 2011 STAR Order Management || Slide 82 Excessive Materials Ordered Charges Example 2 • • • • • October 2010 Value of materials ordered = $5,000 Value of materials used = $4,100 Difference between ordered and used = $900 10% of materials ordered = $500 Difference ($900) is greater than 10% of materials ordered ($500) Charge will apply Charge will be $900 – $500 = $400 2011 STAR Order Management || Slide 83 Review • Set up test administration dates NPS schedules may be different from district schedules • Enter order quantities for each administration Multiple-choice Writing (CST and CMA, grades 4 and 7) EAP Essay • Approve orders Every administration October 2010 2011 STAR Order Management || Slide 84 Timetable • Gather data from sites immediately • Enter orders: Now through December 1, 2010 • Orders reviewed by ETS throughout December and January • Adjustments to initial orders up until “Order Sent for Packaging” date • If required, place supplemental orders after “Order Sent for Packaging” date • Track order status October 2010 2011 STAR Order Management || Slide 85 Next Webcast 2011 STAR Pre-ID Workshop Webcast November 17, 2010 October 2010 2011 STAR Order Management || Slide 86