Institute on Beginning Reading III Instructional Implications: Interpreting Student Performance Data Acknowledgments • Oregon Department of Education • Institute for the Development of Educational Achievement, College of Education, University of Oregon • U.S. Department of Education, Office of Special Education Programs Simmons & Harn © 2004 2 Content Development Content developed by: Deb Simmons, Ph.D. University of Oregon Roland Good, Ph.D. University of Oregon Beth Harn, Ph.D. University of Oregon Additional support: Patrick Kennedy-Paine Katie Tate University of Oregon Simmons & Harn © 2004 3 Copyright • All materials are copy written and should not be reproduced or used without expressed permission of Dr. Edward J. Kame’enui or Dr. Deborah C. Simmons. Selected slides were reproduced from other sources and original references cited. Simmons & Harn © 2004 4 A Schoolwide Beginning Reading Model Ongoing Progress Monitoring and Differentiated and Individualized Instruction for Each Student For Each Student Assessment Goals For All Students Instruction Simmons & Harn © 2004 5 IBR Guiding Questions 1. 2. Today’s Focus 3. 4. 5. 6. Goals: What outcomes do we want for our students in our state, district, and schools? Knowledge: What do we know and what guidance can we gain from scientifically based reading research? Progress Monitoring Assessment: How are we doing? What is our current level of performance as a school? As a grade? As a class? As an individual student? Outcome Assessment: How far do we need to go to reach our goals and outcomes? Core Instruction: What are the critical components that need to be in place to reach our goals? Differentiated Instruction: What more do we need to do and what instructional adjustments need to be made? Simmons & Harn © 2004 6 Objectives: What You Will Learn and Do The goals and objectives of this session: 1. Conduct an overall analysis of class lists. 2. Analyze class lists for instructional samenesses and groupings. 3. Review DIBELS probes for diagnostic information. 4. Design and plan instruction using the alterable variables chart and create CSI maps. Simmons & Harn © 2004 7 Types of Reports to Examine • All reports can be accessed or generated at http://dibels.uoregon.edu/ • Class & Student Level Reports: • Class List • Class Progress Summaries • Individual Student Profile • Benchmark Probe Booklets (from your own school) Simmons & Harn © 2004 8 How Do You Identify Students Needing Additional Instructional Support? • Different reports answer different questions: • Class list/Teacher reports rank order students by instructional recommendation status from highest to lowest • Grade list reports rank order students from highest to lowest on the selected DIBELS measure • Individual student performance reports all DIBELS data collected on an individual student across grades Simmons & Harn © 2004 9 Instructional Status Terminology Quarterly Benchmark Goals Final Benchmark Goals and Later Low Risk Some Risk At Risk Established Emerging Deficit Used for all measures except ORF! Simmons & Harn © 2004 10 Kindergarten DIBELS Benchmark Goals So how are we doing in Winter? Simmons & Harn © 2004 11 First Grade DIBELS Benchmark Goals So how are we doing in Winter? Simmons & Harn © 2004 12 Second & Third Grade DIBELS Benchmark Goals 2nd Grade 3rd Grade Simmons & Harn © 2004 So how are we doing in Winter? 13 4-Step Process for Interpreting Reports & Planning Instruction 1. Overall Analysis: Review class list report to identify general patterns and levels of performance. 2. Instructional Analysis: Identify from class list reports students with similar instructional profiles (instructional sameness). 3. Diagnostic Analysis: For students who need intensive instructional support, review probes to determine current reading skills. 4. Instructional Planning: Review alterable variables chart and design interventions using CSI maps. Simmons & Harn © 2004 14 Step 1: Overall Analysis of Class List Reports • Class list reports provide the following information for each student: • Raw scores on all measures administered • Percentiles: compares a child’s performance to other children in your school/district • Skill status: Established, Emerging, Deficit or Low, Some, At-Risk • Instructional Recommendation: Benchmark, Strategic, Intensive Simmons & Harn © 2004 15 DIBELS Instructional Recommendation • The Instructional Recommendation is created by analyzing a student’s performance across all of the measures administered. • Provides a general description of the instructional intensity needed for the student to achieve the next benchmark goal. • Instructional recommendation categories: • Benchmark: Established skill performance across all administered measures. • Strategic: One or more skill areas are not within the expected performance range. • Intensive: One or more skill areas are significantly atrisk for later reading difficulty Simmons & Harn © 2004 16 Reviewing Class Lists • Use the instructional recommendations as a guideline for instruction. • Remember the category is an approximation not gospel. That is, use them but don’t treat scores on the cusp as definitive and verify any scores that don’t match instructional observations. • Review the list to see how many logical instructional groupings there are. • Determine which students have similar skills and can be taught together (use brackets to indicate possible groups). Simmons & Harn © 2004 17 A Closer Look at Grade 3 Simmons & Harn © 2004 18 Step I: Overall Analysis How to Use Class Lists • What are the measures and their critical values for the time period of the report? • Are the scores typical of student performance? • What is the range of performance in your classroom? • Which children are benefiting adequately from core instruction? • Which children need increased instructional support (supplemental or intensive intervention)? Simmons & Harn © 2004 19 3rd Grade Winter Class List Report Oral Reading Fluency Name Instructional Recommendation Score Percentile Status Jason 12 1 At Risk Intensive - Needs Substantial Intervention John 21 2 At Risk Intensive - Needs Substantial Intervention Luis 37 7 At Risk Intensive - Needs Substantial Intervention Wes 48 13 At Risk Intensive - Needs Substantial Intervention Emily 51 16 At Risk Intensive - Needs Substantial Intervention Kevin 64 22 At Risk Intensive - Needs Substantial Intervention Corey 67 26 Some Risk Strategic - Additional Intervention Michael 71 27 Some Risk Strategic - Additional Intervention Amanda 75 31 Some Risk Strategic - Additional Intervention Nathan 76 31 Some Risk Strategic - Additional Intervention Anthony 77 32 Some Risk Strategic - Additional Intervention Cynthia 86 40 Some Risk Strategic - Additional Intervention Peter 87 41 Some Risk Strategic - Additional Intervention Erica 89 43 Some Risk Strategic - Additional Intervention Tanya 92 46 Low Risk Benchmark - At Grade Level Ryan 93 48 Low Risk Benchmark - At Grade Level Danielle 106 56 Low Risk Benchmark - At Grade Level Devin 112 61 Low Risk Benchmark - At Grade Level Jared 124 70 Low Risk Benchmark - At Grade Level Simmons & Harn © 2004 20 Step 2: Instructional Analysis • Review the following class list: • Identify groups of students who have similar instructional profiles. • Draw brackets around groups of students who are likely to benefit from the same type and amount of instruction. • Once you’ve identified groups, determine primary instructional goal for students in each category: Comprehension and fluency • Benchmark: ____________________________________ • Strategic: ______________________________________ Fluency, comprehension, and advanced phonics and fluency development • Intensive: Phonics ______________________________________ • Which children would likely benefit from a reading fluency intervention? • Use the “Winter Worksheet” to document groups. Simmons & Harn © 2004 21 Step 3: Diagnostic Analysis Use Probes And/Or Other Assessment Data To Diagnose Difficulty: What More Information do You Need & Have? Third Grade Example: Diagnosing Reading 1. 2. Is fluency an appropriate objective? 17 17 / 26 = 65% accuracy What types of errors are being made? Sight words and decoding errors 3. What type of instruction is needed? Continued phonics instruction, sight words, and attention to accuracy and fluency to improve comprehension Simmons & Harn © 2004 23 Third Grade Example: Diagnosing Reading 45 1. 2. 3. Is fluency an appropriate objective? 45 / 49 = 91% accuracy What types of errors are being made? Decoding errors that preserve meaning, many selfcorrections What type of instruction is needed? Continued phonics instruction but with a focus on fluency instruction Simmons & Harn © 2004 24 Analysis of a Third Grade Struggling Reader • Following is the cover sheet for a 3rd grader showing reading performance from Fall and Winter • Is this child learning? • Is the child learning enough? • Calculate Oral Reading Fluency growth need from Winter to Spring. • Spring Goal Score - Winter Score/15 (# weeks instruction) 110 – 59 = 51; 51/15= 3.1 word growth a week • Analyze types of errors on the accompanying probe. • Use the CSI map and design an intervention plan for this learner and ones like him/her in the same class (return to grade 3 class list). Simmons & Harn © 2004 25 Third Grade - Student 1 Test of Reading Fluency (Children’s Educational Services, 1987) Simmons & Harn © 2004 26 Simmons & Harn © 2004 27 Step 4: Instructional Planning Using Alterable Variables and CSI Maps Summary of Research Findings • Use explicit, systematic reading program (select from supplemental & intervention list). • Include word identification and reading fluency practice. • Teach with small teacher/student ratios: 1/1 1/5. • Teach intensively (1 hr per day with variety of aligned strategies). • Provide extensive opportunities for practice and feedback. • Double dose if possible. Simmons & Harn © 2004 29 Alterable Variables to Intensify Instruction Alterable Components Options 1 2 Use core program & explicitly teach priority skills. Schedule & deliver 90 minutes of daily Time reading (Opportunities instruction to Learn) (minimum 30 minutes small group). Grouping for Instruction Check group placemen t & provide comb ination of whole & small group instruction. 3 4 Use extensions of the core program (e.g., add examp les) Supp lement core wi th reteaching or intervention componen ts of core. Replace current core program wi th intervention program. Increase opportunities to respond during core instruction. Schedule core + supplemen ta l period daily. (90 + 30 or 60 + 30) Schedule two intervention sessions daily (no less than 90 minutes total) Schedule small group opportunity for specific practice Reduce group size Provide individual instruction 5 Implement specially designed program Increasing Intensity Program Emphasis Level of Specific Enhancements Increasing Intensity Simmons & Harn © 2004 30 Alterable Elements • Program: Is the learner likely to benefit from the core? If not, what supplement or intervention/acceleration program is available? • Time: A minimum of 30 + 30 minutes of small group intensive instruction in addition to typical whole group instruction. • Grouping/Organization: As small a group as possible with the most skilled instructor available Simmons & Harn © 2004 31 CSI Map: Grade 3 Goals Listed below are the high-priority goals for each big idea. Refer to the curriculum maps for a complete list. Vocabulary Learns and use s unfamiliar words that are introduced in stories and pas sages Increases kno wl edge o f vocabulary through independen t reading Comprehension Answers literal, inferential, and evaluative questions Answers questions about main characters, setting, theme, plot Distinguishes main idea/details; fact/opinion; cause/effect Uses structure of informational text to aid understanding Uses information in tables, graphs, diagrams, map s, charts Retells the main ideas of stories or informational texts Spe lling Spe lls phone tically regular words correctly Instructional Details Instructional Need Program/Materials Core / Benchmark Time/day Assessment Grouping Size DIBELS Measure (based on Winter progressive benchmarks) No DIBELS benchmark. Use progra m specific or district/school determined measures. Strategic Intensive Core / Benchmark No DIBELS benchmark. Use progra m specific or district/school determined measures. Strategic Intensive Core / Benchmark No DIBELS benchmark. Use progra m specific or district/school determined measures. Strategic Intensive Simmons & Harn © 2004 32 CSI Map: Grade 3 Goals Listed below are the high-priority goals for each big idea. Refer to the curriculum maps for a complete list. Instructional Details Instructional Need Program/Materials Time/day Assessment Grouping Size DIBELS Measure (based on Winter progressive benchmarks) Alphabetic Principle Reads regular multisyllabic words Core / Benchmark NWF Measure may be helpful in determining alphabetic principle skills. Strategic Intensive Fluency wi th Connected Te xt Reads 110 -120 words per minute by the end of the year Core / Benchmark ORF ³ 92: Assess quarterly Strategic 67 ² OR F < 92: Assess once o r twice a mon th Intensive ORF < 67: Assess 2-4 times monthly Increases independent reading END OF YEAR GOAL = 110 ORF Simmons & Harn © 2004 33 Examples of Reading Program Implementation: Grade 2/3 A Core Programs Selected from Analysis of Core Programs B Supplemental/Intensive Programs To be Determined C Time/Grouping Conditions 30-45 minutes small group teacher directed instruction Supplement of 30 minutes prioritized reading instruction Small groups (4-5) or individualized tutoring Highly trained teachers with high quality implementation Simmons & Harn © 2004 34 A Closer Look at Grade 1 Simmons & Harn © 2004 35 Step I: Overall Analysis How to Use Class Lists • What are the measures and their critical values for the period of the report? • Are the scores typical of student performance? • What is the range of performance in your classroom? • Which children are benefiting adequately from core instruction? • Which children need increased instructional support (supplemental or intensive intervention)? Simmons & Harn © 2004 36 Analyzing Winter Grade 1 • Class List/Teacher Report, Winter Grade 1 Name Phoneme Segmentation Fluency Nonsense Word Fluency Oral Reading Fluency Instructional Recommendation Score %ile Status Score %ile Status Score %ile Status Casey 8 N/A Deficit 11 N/A Deficit 1 N/A At Risk Greg 55 N/A Established 20 N/A Deficit 8 N/A Cassie 30 N/A Emerging 26 N/A Deficit 2 N/A At Risk Intensive - Needs Substantial Intervention Sandra 39 N/A Established 28 N/A Deficit 6 N/A At Risk Intensive - Needs Substantial Intervention Rachel 66 N/A Established 32 N/A Emerging 14 N/A Ben 59 N/A Established 35 N/A Emerging 7 N/A Jill 27 N/A Emerging 35 N/A Emerging 12 N/A Some Risk Strategic - Additional Intervention Ivan 43 N/A Established 36 N/A Emerging 12 N/A Some Risk Strategic - Additional Intervention Katie 18 N/A Emerging 39 N/A Emerging 37 N/A Erin 16 N/A Emerging 40 N/A Emerging 13 N/A Some Risk Strategic - Additional Intervention Page 64 N/A Established 41 N/A Emerging 12 N/A Some Risk Strategic - Additional Intervention Juanita 30 N/A Emerging 41 N/A Emerging 31 N/A Low Risk Jose 59 N/A Established 44 N/A Emerging 0 N/A At Risk Taylor 46 N/A Established 46 N/A Emerging 17 N/A Justin 37 N/A Established 56 N/A Established 21 N/A Low Risk Benchmark - At Grade Level Sarah 78 N/A Established 67 N/A Established 56 N/A Low Risk Benchmark - At Grade Level Simmons & Harn © 2004 Intensive - Needs Substantial Intervention Some Risk Intensive - Needs Substantial Intervention Some Risk Strategic - Additional Intervention At Risk Low Risk Intensive - Needs Substantial Intervention Benchmark - At Grade Level Benchmark - At Grade Level Intensive - Needs Substantial Intervention Some Risk Strategic - Additional Intervention 37 Step 2: Instructional Analysis • Review the following class list: • Identify groups of students who have similar instructional profiles. • Draw brackets around groups of students who are likely to benefit from the same type and amount of instruction. • Once you’ve identified groups, determine primary instructional goal for students in each category: • Benchmark: • Strategic: • Intensive: • Using the curricular maps and established early literacy goals, the instructional objectives would be one or more of the following: • Fluency • Phonics • Phonological awareness • Use the Winter Worksheet to document groups. Simmons & Harn © 2004 38 Analyzing Patterns of Performance • Class List/Teacher Report, Winter Grade 1 Name Phoneme Segmentation Fluency Nonsense Word Fluency Oral Reading Fluency Instructional Recommendation Score %ile Status Score %ile Status Score %ile Status Casey 8 N/A Deficit 11 N/A Deficit 1 N/A At Risk Greg 55 N/A Established 20 N/A Deficit 8 N/A Cassie 30 N/A Emerging 26 N/A Deficit 2 N/A At Risk Intensive - Needs Substantial Intervention Sandra 39 N/A Established 28 N/A Deficit 6 N/A At Risk Intensive - Needs Substantial Intervention Rachel 66 N/A Established 32 N/A Emerging 14 N/A Ben 59 N/A Established 35 N/A Emerging 7 N/A Jill 27 N/A Emerging 35 N/A Emerging 12 N/A Some Risk Strategic - Additional Intervention Ivan 43 N/A Established 36 N/A Emerging 12 N/A Some Risk Strategic - Additional Intervention Katie 18 N/A Emerging 39 N/A Emerging 37 N/A Erin 16 N/A Emerging 40 N/A Emerging 13 N/A Some Risk Strategic - Additional Intervention Page 64 N/A Established 41 N/A Emerging 12 N/A Some Risk Strategic - Additional Intervention Juanita 30 N/A Emerging 41 N/A Emerging 31 N/A Low Risk Jose 59 N/A Established 44 N/A Emerging 0 N/A At Risk Taylor 46 N/A Established 46 N/A Emerging 17 N/A Justin 37 N/A Established 56 N/A Established 21 N/A Low Risk Benchmark - At Grade Level Sarah 78 N/A Established 67 N/A Established 56 N/A Low Risk Benchmark - At Grade Level Simmons & Harn © 2004 Intensive - Needs Substantial Intervention Some Risk Intensive - Needs Substantial Intervention Some Risk Strategic - Additional Intervention At Risk Low Risk Intensive - Needs Substantial Intervention Benchmark - At Grade Level Benchmark - At Grade Level Intensive - Needs Substantial Intervention Some Risk Strategic - Additional Intervention 39 Step 3: Diagnostic Analysis Use Probes to Diagnose Difficulty What More Information do You Need & Have? Analysis of a First Grade Struggling Reader • Following are two different readers (Rachel & Greg) in Winter of first grade: • Are they learning? • Are they learning enough? • Determine the level of intensity of instructional for both to reach the end-of-year first grade goal of 40 cwpm on ORF • Analyze types of errors on the accompanying probes. • Use the CSI map and design an intervention plan for these learners and ones like him/her in the same class (return to grade 1 class list). Simmons & Harn © 2004 41 Diagnosing Reading: Rachel First Grade Reader NWF Performance: 1. 2. Analyze strengths and areas of instructional need? Is the primary need accuracy or fluency of letter sounds? Fluency What type of instruction is needed? Fluency in whole word recognition to aid in reading sentences Simmons & Harn © 2004 42 Diagnosing Reading: Rachel First Grade Reader ORF Performance: 14 1. Analyze strengths and areas of instructional need? Is the primary need accuracy or fluency? 2. Fluency What type of instruction is needed? Fluency in reading sight words and sentences. Simmons & Harn © 2004 43 Diagnosing Reading: Greg First Grade Reader NWF Performance: 13 7 20 1. Analyze strengths and areas of instructional need? Is the primary need accuracy or fluency of letter sounds. Accuracy and then fluency 2. What type of instruction is needed? Letter sound instruction, blending, and fluency in blending Simmons & Harn © 2004 44 Diagnosing Reading: Greg First Grade Reader ORF Performance: 8 1. Analyze strengths and areas of instructional need? Is the primary need accuracy or fluency? Accuracy 2. What type of instruction is needed? Phonics instruction, blending whole words, and reading with accuracy to improve comprehension Simmons & Harn © 2004 45 Step 4: Instructional Planning Using Alterable Variables and CSI Maps Alterable Variables to Intensify Instruction Alterable Components Options 1 Use core program & explicitly teach priority skills. Schedule & deliver 90 minutes of daily Time reading (Opportunities instruction to Learn) (minimum 30 minutes small group). Grouping for Instruction Check group placemen t & provide comb ination of whole & small group instruction. 2 3 4 Use extensions of the core program (e.g., add exa mples) Supp lement core wi th reteaching or intervention componen ts of core. Replace current core program wi th intervention program. Increase opportunities to respond during core instruction. Schedule core + supplemental period daily. (90 + 30 or 60 + 30) Schedule two intervention sessions daily (no less than 90 minutes total) Schedule small group opportunity for specific practice Reduce g roup size Provide individual instruction 5 Implement specially designed program Increasing Intensity Program Emphasis Level of Specific Enhancements Increasing Intensity Simmons & Harn © 2004 47 Examples of Reading Program Implementation: Grade 1 A Core Programs Selected from Analysis of Core Programs B Supplemental/Intensive Programs To be Determined C Time/Grouping Conditions 30-45 minutes small group teacher directed instruction Supplement of 30 minutes prioritized reading instruction Small groups (4-5) or individualized tutoring Highly trained teachers with high quality implementation Simmons & Harn © 2004 48 Prepare CSI Maps 1. Review your class lists. 2. Identify children who need intensive intervention and strategic intervention who can be taught in the same group. (Use winter worksheet attached). 3. Examine performance by big idea. 4. Determine the instructional program. 5. Designate and protect instructional time. 6. Select best possible instructor. 7. Keep group as small as possible. 8. Teach every day. Simmons & Harn © 2004 49 Sample CSI Map: Grade 1 Phonological Awareness Goals Listed below are the high-priority goals for each big idea. Refer to the curriculum maps for a complete list. Instructional Details Instructional Need Phono logical Awareness (PA) Blends 3-4 phoneme s into a whole word Core / Benchmark Time/day Tim e in Core + 3-5 min. in small group. Use whole class & small group. Use Core program & Benchmark plan above for students who are close to PSF 35. Differentiate instruction for students who are furthere r from the target. Add small group daily practice on segmentation with 3-phoneme words then 4-phoneme words. Review Grade 1 PA prog rams from supplemental list to identify those that focus on blending & segmentation at the phoneme level. Integrate with spelling and reading. Add 5-10 minutes daily Small group (< 6 students) 10 ² PSF > 35: Assess once o r twice a mon th Determine whether Core PA instruction is appropriate. Review PSF probe to identify level of segmentation Provide targeted practice on blending & segmentation at the phoneme leve l. Review PA intervention programs that focus on blending & segmentation. Add to daily instruction until student reaches benchmark consistently. Add two 5-10 min. periods daily Small group (< 4 students) PSF < 10: Assess 2-4 times monthly Program/Materials Use Core program. To continue PA deve lopment, supplement core program with daily practice on more adva nced PA skills (e.g., substituting and manipulating sounds in words) Seg ments 3- and 4phoneme, 1-syllable words Strategic Intensive Assessment DIBELS Measure Grouping Size / Delivery Simmons & Harn © 2004 (based on Winter progressive benchmarks) PSF 35: As sess quarterly *Based on end of K benchmark 50 Sample CSI Map: Grade 1 Alphabetic Principle Listed below are the high-priority goals for each big idea. Refer to the curriculum maps for a complete list. Instructional Need Program/Materials Alphabetic Principle Produces sound s to comm on letter combinations Decodes words with letter comb inations Reads regular 1-syllable words fluently Reads wo rds wi th common word parts Reads co mmon s ight words automa tically Assessment Instructional Details Goals Core / Benchmark Strategic Intensive Time/day Continue Core program. Supplement with daily practice on targeted skill including new lettersound correspondences, blending words & sight word reading. Ensure a review schedule that is sufficient. Provide daily practice. Use Core program & Benchmark plan above for students who are close to NWF 50. Differentiate instruction for students who are further from the target. Review NWF probe to diagnose difficulties. Add small group daily practice on specific letter sounds and blending. Review Grade 1 Phonics programs from supplemental list to identify those that focus on letter sound and word rea ding instruction. Add 5-10 min. daily practice on target skills. Determine whether core is appropriate. Review NWF probe to identify whether student knows LS and how to blend into words. Focus on blending & reading CVC and VC word types. Review Phonics intervention progra ms that focus on letter sound and blending words. Add to daily instruction until student reaches benchmark consistently. Simmons & Harn © 2004 Grouping Size / Delivery Use whole class & small group. Small group (< 6 students) DIBELS Measure (based on Winter progressive benchmarks) NWF ³ 50: Assess quarterly 30 ² N WF < 50: Assess once o r twice a mon th NWF < 30: Assess 2-4 times monthly Add two 7-10 min. periods daily. Small group (< 4 students) Administer letter sound test & s ight word list to determine specific areas of instruction. 51