RTI2 ASSESSMENT PRACTICES: AIMSWEB LITERACY MEASURES September 11, 2012 Every student achieving; everyone responsible What instructional strategies do we use? What do we want students to know or do? 2 RtI How do we know if they have learned it? How do we respond when they haven't learned it? What do we do for those who already know it? Key Questions in a Response to Instruction and Intervention (RtI2) Framework Types of Assessment Summative Assessments: evaluate whether the instruction or WKCE intervention provided enough to help all Measure is of powerful Academic Progress (MAP) students achieve or exceed standards by the End ofgrade-level Unit Tests Final Exams end of each year. Formative: Brief, targeted, and frequent measures of Thumbs Up/Thumbs Down Quickgoals. Writes Used as feedback for progress toward short-term Exit Slips refining instruction/learning. GOMs Universal Screener/Benchmarking and Progress General Outcome Measures monitoring: Brief and targeted assessments, focused on “indicators” Curriculum-Based of broad skillMeasures domains. AIMSweb Running Records Diagnostic: inform instructional planning in order to meet Phonics Surveys the most critical needs Reading of individual Inventoriesstudents Assessment Framework Matrix: Putting it all together Question is about… Type of Assessment Purpose of Assessment System or System Unit Summative/Outco me Drive Long-Term Improvement Planning Patterns of progress toward system outcome goals Universal Identify groups Screening/Benchm “on-track” and arking “off-track” Reference Individual shortterm progress Actions Analogy Benchmarks Comparables High Performers -How are we doing Continue, refine overall? How did or change the we do? plan -What direction are we headed? -Where should we focus efforts to improve? Standings Relevant benchmarks Who is responding to instruction? Who is not responding to instruction? Continue, refine or change instruction Scoreboard Aimline Is this student mastering the essential skills? Is the instructional program working for this student? Continue, extend, Play by play refine or change outcomes materials, pace, instructional approach, etc… Progress Monitoring (CBM) Individual status or Formative growth toward specific learning objectives. Answers Big Ideas in Reading: And all of the parts need to be in working order… Alphabetic Principle Phonemic Awareness Comprehension Fluency Vocabulary How Do Children Learn to Read? Big Idea 1: Phonemic Awareness Phonemic Awareness is: the ability to hear and manipulate the sounds in spoken words and the understanding that spoken words and syllables are made up of sequences of speech sounds (Yopp, 1992; see References). essential to learning to read in an alphabetic writing system, because letters represent sounds or phonemes. Without phonemic awareness, phonics makes little sense. fundamental to mapping speech to print. • If a child cannot hear that "man" and "moon" begin with the same sound. he or she may have great difficulty connecting sounds with their written symbols or blending sounds to make a word. How Do Children Learn to Read? Big Idea 1: Phonemic Awareness Phonemic Awareness is: essential to learning to read in an alphabetic writing system a strong predictor of children who experience early reading success. The best predictor of reading difficulty in kindergarten or first grade is the inability to segment words and syllables into constituent sound units (phonemic awareness)" (Lyon, 1995) Phoneme Awareness is NOT Phonics – It Permits Phonics Phoneme awareness provides the foundation for learning phonics and for differentiating similar words in speech /b/ /r/ br /I/ igh /t/ t How do Children Learn to Read? Big Idea 2: The Alphabetic Principal The Alphabetic Principal is: Alphabetic Understanding: Words are composed of letters that represent sounds Phonological Recoding: Using systematic relationships between letters and phonemes to retrieve the pronunciation of an unknown printed string or to spell words Phonics Because our language is alphabetic, decoding is an essential and primary means of recognizing words. There are simply too many words in the English language to rely on memorization as a primary word identification strategy. Correlations Between Decoding and Comprehension 0.9 0.89 0.83 0.8 0.77 0.71 0.7 0.69 0.69 0.7 0.63 0.63 Gr8 Gr9 0.6 0.5 0.4 0.3 0.2 0.1 0 Gr1 Gr2 Gr3 Gr4 Gr5 Gr6 Gr7 How Do Children Learn to Read? Big Idea 3: Fluency Fluency is the ability to “break the code” effortlessly. A fluent reader is one whose decoding processes are automatic, requiring no conscious attention. Such capacity then enables readers to allocate their attention to the meaning of text. Fluency is a very strong predictor of comprehension. Fluency is developed through repeated oral reading practice with text that is readily decodable. Fluency is also improved through oral reading with systematic guidance and feedback How Do Children Learn to Read? Big Idea 4: Comprehension Comprehension is a complex cognitive process that cannot be understood without a clear description of the role of vocabulary instruction Good Comprehenders… Relate new information to existing knowledge Have well developed vocabularies Can summarize, predict and clarify Use questioning strategies to guide comprehension Text comprehension can be improved by explicit instruction that helps readers use specific strategies Instruction in using strategies flexibly and in combination is important. How Do Children Learn to Read? Big Idea 5: Vocabulary Children use their knowledge of word meanings to make sense of the words they see in print Beginning readers have a much more difficult time reading words that are not already part of their oral vocabulary. Children learn word meanings indirectly in three ways: They engage in daily oral language They listen to adults read to them They read on their own How Do Children Learn to Read? Big Idea 5: Vocabulary Vocabulary research shows… Children who are not yet proficient readers learn little vocabulary through the reading process. Children from low SES environments are exposed to dramatically fewer words on a daily basis. Words per hour: Lowest SES 616 Low 1251 Average 2153 Independent reading by proficient readers has a substantial effect on vocabulary development. Basic Early Literacy Skill Predictive of reading acquisition and later reading achievement Something we can do something about, i.e. something we can teach Something that improves student outcomes for children when we teach it Steps for Successful Readers (Schools in Kalamazoo County 2004-2006) Probability of “Staying on Track” .85 (n=770) .82 (n=849) .83 (n=910) Fluency (Spr, 1st) .62 (n=1178) Alphabetic .14 (n=336) Principle (Win, 1st) Phonemic .16 (n=114) Awareness (Spr, Kdg) Fluency (Spr 2nd) .97 (n=372) Fluency (Spr, 5th) .92 (n=561) Fluency (Spr, 4th) .09 (n=185) Fluency (Spr, 3rd) 0 (n=190) .03 (n=401) .05 (n=238) Probability of “Catching-Up” n = number of students What do we know from Research Reading trajectories are established early Readers on a low trajectory tend to stay on a low trajectory and tend to fall further behind Unless… We change the trajectory GOM, CBM, AIMSweb, Universal Screening… Undertanding CBM & GOM Article 1 Ah-ha and 1 Question Things to Always Remember About CBM Designed to serve as “indicators” of general reading achievement: CBM probes don’t measure everything, but measure the important things. Standardized tests to be given, scored, and interpreted in a standard way Researched with respect to psychometric properties to ensure accurate measures of learning Things to Always Remember About CBM (cont.) Are sensitive to improvement in brief intervals of time Also tell us how students earned their scores (qualitative information) Designed to be as short as possible to ensure its “do ability” Are linked to decision making for promoting positive achievement and Problem-Solving What is AIMSweb? Curriculum-based measures (CBMs) What it is: short reliable and valid probes of academic skills; a “snapshot” Research: useful tools to identify student needs and track progress Identifies students who are “at-risk” for poor academic outcomes Indicators of larger skill set On-line data management and tools Benchmarking Three times a year, for all students (Tier 1, Universal support) Progress As 22 monitoring needed basis, for students that have academic concerns (Tier 2, Targeted support and Tier 3, Intensive support) AIMSweb Probes in Literacy BIG IDEA Probe What Students Do Letter Naming Fluency (LNF) Say the name of visually presented letters in 1 minute Phonemic Awareness Phoneme Segmentation Fluency (PSF) Say the phonemes in orally presented words n 1 minute Beginning Phonics Letter Sound Fluency (LFS) Say the sound of visually presented letters in 1 minute Beginning Phonics Nonsense Word Fluency (NWF) Read non-real, three letter words in 1 minute Advanced Phonics and Fluency in Connected Text Reading- CBM (R-CBM) Read narrative and expository stories in 1 minute Comprehension MAZE Identify the correct word from 3 choices in a passage Using AIMSweb for Problem-Solving and Instructional Decision-Making How Does it Fit Together? Standard Treatment Protocol Step 2 Step 1 All Students at a grade level Intensive Supplemental Step 3 Step 4 Addl. Diagnostic Assessment Instruction Individual Diagnostic Individualized Intensive 1-5% 5-10% Standard Protocol Behavior Academics Small Group Differentiated By Skill BenchMark Assessment Annual Testing weekly 2 times/month Core ODRs Monthly Bx Screening Results Monitoring 80-90% None Continue With Core Instruction Grades Classroom Assessments Yearly Assessments Problem Identification 140 Peers 120 100 80 Benchmark 60 40 Student 20 0 0 1 2 3 4 5 6 7 8 9 10 11 12 Problem Identification 140 120 100 80 Benchmark 60 Peers 40 Student 20 0 0 1 2 3 4 5 6 7 8 9 10 11 12 Problem Identification 140 120 100 80 Benchmark 60 Peers 40 Student 20 0 0 1 2 3 4 5 6 7 8 9 10 11 12 x 70 College and Career Readiness Explore/Plan/ACT 60 50 MAP 40 CBM MAP EPAS 30 CBM x x x x 20 x x xx x x x x x x 10 x x x x 0 4K K 1 2 3 4 5 6 7 8 9 10 11 12 What it really looks like: Universal Screening during benchmark period three times per year (required for grades 1 and 2 only) One-to-one administration Approximately 5-7 minutes/student Paper booklets for each student In the future: Progress monitoring for those receiving supplemental interventions at regular intervals (e.g., 1x/week) Classroom Report Example Student Benchmarking Report Tier 1 Benchmark All students assessed three times per year at grade level Organizes Curriculum-Based Measurement (CBM) data for Benchmark Assessment Fall, Winter, and Spring Prepares Reports for Teachers, Principals, and Administrators on Individual Students, Classes, Grades, Schools, and School Districts Early Identification of Students At-Risk of Academic Difficulty Objectively Determines Rates of Progress for Individual Students, Schools, and NCLB Risk Groups Allows Evaluation at Multiple Levels of Comparison Prints Professional Reports for Parent Conferences and Other Meetings Tier 2 Strategic Monitor Monthly assessments at grade level Increase assessment frequency for students who have been identified as questionable or at-risk in the Benchmark process Monthly assessment intervals provide more frequent opportunity to evaluate the effectiveness of instructional changes Tier 3 Progress Monitor Frequent assessment towards goals using goal level assessments Frequently assess students in need of intensive instructional services, including “Best Practices” IEP goals for students who receive special education services Translate annual IEP or any goals into expected rates of progress (Aim lines) automatically Monitor progress (Trend lines) towards goals Document the effects of intervention and instruction Print professional reports for periodic and annual reviews Administration and Scoring Letter Naming Fluency 36 Administration and Scoring of Letter Naming Fluency 37 What examiners need to do… Before testing students While testing students After testing students Things You Need Before Testing: Letter Naming Fluency Specific Materials Arranged: • Student copy of Letter Naming Fluency (not numbered) • Examiner copy of Letter Naming Fluency (numbered for easy scoring) • Clipboard to provide a hard surface for recording student answers • Stopwatch Things You Need While Testing Letter Naming Fluency Standard Directions for 1- Minute Administration: 1) 2) 3) 4) 5) Place the unnumbered copy in front of the student. Place the numbered copy in front of you, but shielded so the student cannot see what you record. Say: “Here are some letters [point to the student copy]. Begin here, [point to the first letter] and tell me the names of as many letters as you can. If you come to a letter you don’t know, I’ll tell it to you. Are there any questions? Put your finger under the first letter. Ready, begin.” Start you stopwatch. If the student fails to say the first letter name after 3 seconds, tell the student the letter name and mark it as incorrect. Point to the next letter to indicate for the child to move on. If the student provides the letter sound rather than the letter name say, “Remember to tell me the letter name, not the sound it makes.” This prompt may be provided once during the administration. Things You Need While Testing Letter Naming Fluency Standard Directions for 1- Minute Administration (Continued) 6) 7) 8) 9) If the student does not get any correct letter names within the first 10 letters (1 row), discontinue the task and record a score of 0. Follow along on your copy. Put a slash ( / ) through letter names given incorrectly. The maximum time for each letter is 3 seconds. If a student does not provide the next letter within 3 seconds, tell the student the letter name and mark it as incorrect. Point to the next letter and say, “What letter?” At the end of 1 minute, place a bracket ( ] ) after the last letter and say, “Stop.” Things to Do After Testing Letter Naming Fluency • Score immediately to ensure accurate results • Students receive 1 point for every correct letter named in 1 minute What is a Correct Letter Name? • A correctly named letter • Confused I’s and L’s as a function of font • Self Corrections What is an Incorrect Letter Name? • Substitutions of a different letter for the stimulus letter (e.g., “P” for “D”) • Omissions of a letter • Stops or struggles with a letter for more than 3 seconds Note: Skipped Row. If a student skips an entire row, draw a line through the row and do not count the row in scoring Calculating & Reporting Scores Letter Naming Fluency • Count the total number of letters the student read • Count the number of errors and subtract • Report the total number of letters named correctly Example of Calculating Scores • Angela finished letter naming after 1 minute at the 20th letter, so she named 20 letters total • Angela also made 5 errors • Therefore, her recorded score is 15 (20 – 5 = 15) Reported as 15 Let’s Practice! Administration and Scoring Letter Sound Fluency Big Ideas in Reading: And all of the parts need to be in working order… Alphabetic Principle Phonemic Awareness Comprehension Fluency Vocabulary Big Idea: Phonemic Awareness Phoneme: a speech sound. It is the smallest unit of language that has no inherent meaning Phonemic Awareness: The understanding that spoken words and syllables are made up of sequences of speech sounds, and the ability to hear and manipulate these sounds. Phonics: Use of the code (sound-symbol relationships) to recognize words Big Idea: Phonemic Awareness Gives readers a way to approach sounding out and reading new words Helps readers understand the alphabetic principle (that letters in words are systematically represented by sounds) Things You Need Before Testing Letter Sound Fluency Specific Materials Arranged: • Student copy of Letter Sound Fluency (not numbered) • Examiner copy of Letter Sound Fluency (numbered for easy scoring) • Clipboard to provide a hard surface for recording student answers • Stopwatch Things You Need While Testing Letter Sound Fluency Standard Directions for 1- Minute Administration 1) Place the unnumbered copy in front of the student. 2) Place the numbered copy in front of you, but shielded so the student cannot see what you record. 3) Say, “Begin here, [point to the first letter] and tell me the sounds of as many letters as you can. If you come to a letter you don’t know, I’ll tell it to you. Are there any questions? Put your finger under the first letter. Ready, begin.” 4) Start your stopwatch. If the student fails to say the first letter sound after 3 seconds, tell the student the letter sound and mark it as incorrect. Point to the next letter and say, “What sound?” 5) If the student provides the letter name rather than the letter sound say, “Remember to tell me the sound the letter makes, not its name.” This prompt may be provided once during the administration. 52 Things You Need While Testing Letter Sound Fluency Standard Directions for 1- Minute Administration (Continued) 6) If the student does not get any correct letter sounds within the first 10 letters (1 row), discontinue the task and record a score of 0. 7) Follow along on your copy. Put a slash ( / ) through letter sounds given incorrectly. The maximum time for each letter is 3 seconds. 8) The maximum time for each letter is 3 seconds. If a student does not provide the next sound within 3 seconds, tell the student the letter sound and mark it as incorrect. Point to the next letter and say, “What sound?”. 9) At the end of 1 minute, place a bracket ( ] ) after the last letter and say, “Stop.” 53 Things to Do After Testing Letter Sound Fluency 1. Count the total number of letter sounds read 2. Count the number of errors and subtract 3. Total the number of letter sounds read correctly What is a Correct Letter Sound? Students provide the most COMMON sound of the letter. (A pronunciation guide for the most common sounds is in the Appendix of the AIMSweb Training Workbook) • • Confused I’s and L’s as a function of font • Self Corrections What is an Incorrect Letter Sound? • Substitutions of a different letter sound for the stimulus letter (e.g., /puh/ for /D/) • Omissions of a letter sound • Stops or struggles with a letter sound for more than 3 seconds Note: Skipped Row. If a student skips an entire row, draw a line through the row and do not count the row in scoring Let’s Practice! Administration and Scoring Phoneme Segmentation Fluency Things You Need Before Testing Phoneme Segmentation Fluency Specific Materials Arranged: • Examiner copy of Phonemic Segmentation Fluency (numbered for easy scoring) • Clipboard to provide a hard surface for recording student answers • Stopwatch Things You Need While Testing: Phoneme Segmentation Fluency Standard Directions for 1- Minute Administration 1) Place the examiner copy in front of you, but shielded so the student cannot see what you record. 2) Say: “I am going to say a word. After I say it, I want you to tell me all the sounds in the word. So, if I say, “Sam,” you would say /s/ /a/ /m/. Let’s try one [1 second pause]. Tell me the sounds in “mop.” Correct Response Very good. Incorrect Response The sounds in “mop” are /m/ /o/ /p/. Your turn. Tell me the sounds in “mop”. 3) Say: “Okay. Here is your first word.” 4) Give the student the first word and start your stopwatch. If the student does not say a sound segment after 3 seconds, give the second word and score the first word as 0 segments produced. Things You Need While Testing: Phoneme Segmentation Fluency Standard Directions for 1- Minute Administration (Continued) 5) 6) 7) 8) 9) As the student says the sounds, mark the student’s response in the scoring column. Underline ( __ ) each sound segment produced correctly. Put a slash ( / ) through sounds produced incorrectly. As soon as the student is finished saying the sounds, present the next word promptly and clearly. If the student does not get any sounds correct in words 1-5, discontinue the task and record a score of 0. The maximum time for each sound segment is 3 seconds. If the student does not provide the next sound segment within 3 seconds, give the student the next word. If the student provides the initial sound only, wait 3 seconds for elaboration. At the end of 1 minute, stop presenting words and scoring further responses. Place a bracket ( ] ) after the last segment produced. Add the number of sound segments produced correctly. Record the total number of sound segments produced correctly on the bottom of the scoring sheet. Things to Do After Testing: Phoneme Segmentation Fluency Students receive 1 point for every correct segment provided in 1 minute Calculate total number of segments correct What is a Correct Segment? Complete segmentation (Example: “T…r…i…ck”) Incomplete segmentation (Example: “Tr…ick”) Overlapping segmentation (Example: “Tri…ick”) Schwa sounds (Example: “Tuh…ruh…i…kuh”) Additions (Example: “T…r…i…ck…s”) Articulation and dialect; imperfect pronounciations due to dialect or articulation (Example: “r…e…th…t” for “rest”) Elongated sounds (Example: “rrrrreeeesssstttt”) What is an Incorrect Segment? • Omissions (Example: “t…ick”) • No segmentation (Example: “trick”) • Segment mispronunciation (Example: “t…r…i…ks”) Let’s Practice! Administration and Scoring Nonsense Word Fluency Big Ideas in Reading: And all of the parts need to be in working order… Alphabetic Principle Phonemic Awareness Comprehension Fluency Vocabulary Big Idea: Alphabetic Principal Alphabetic Understanding: Words are composed of letters that represent sounds Phonological Recoding: Using systematic relationships between letters and phonemes to retrieve the pronunciation of an unknown printed string or to spell words Phonics: The system of letter-sound relationships that is the foundation for decoding words in print (Kaminski and Good) Big Idea: Alphabetic Principal Letter-Sound Knowledge is a prerequisite to word identification The combination of instruction in phonemic awareness and letter-sound relationships appears to be the most effective for successful early reading. Good readers have high capacity to apply the alphabetic principle. During the alphabetic phase, reading must have lots of practice phonologically recoding the same words to become familiar with patterns for reading and spelling Administration and Scoring of Nonsense Word Fluency What Examiners Need to Do . . . Before testing students • While testing students • After testing students • Things You Need Before Testing: Nonsense Word Fluency Specific Materials Arranged: • Student copy of Nonsense Word Fluency (not numbered) • Examiner copy of Nonsense Word Fluency (numbered for easy scoring) • Practice Examples • Clipboard to provide a hard surface for recording student answers • Stopwatch Things You Need While Testing: Nonsense Word Fluency Standard Directions for Practice Items Administration 1) Place the practice items in front of the child. Nonsense Word Practice Items bim lat 2) Say: “Look at this word (point to the first word on the practice probe). It’s a make-believe word, not a real word. All the letters have sounds: (point to the letter “b”) /b/, (point to the letter “i”) /i/, (point to the letter “m”) /m/. Altogether the sounds are /b/ /i/ /m/ (point to each letter) or “bim” (run your finger fast through the whole word). Remember, it is a made up word. You can say the sounds of the letters, /b/ /i/ /m/ (point to each letter), or you can say the whole word “bim” (run your finger fast through the whole word). Be sure to say any sounds you know. Ready? Lets try one. Read this word the best you can (point to the word “lat”). Point to each letter and tell me the sound or tell me the whole word.” Things You Need While Testing: Nonsense Word Fluency Standard Directions for Practice Items Administration (Continued) Correct Response That’s right. The sounds are /l/ /a/ /t/ or “lat” Incorrect Response Watch me: (point to the letter “l”) /l/, (point to the letter “a”) /a/, (point to the letter “t”) /t/. Altogether the sounds are /l/ /a/ /t/ (point to each letter) or “lat” (run your finger fast through the whole word). Remember, you can say the sounds or you can say the whole word. Let’s try again. Read this word the best you can (point to the word “lat”). Things You Need While Testing: Nonsense Word Fluency Standard Directions for 1- Minute Administration 1) Place the unnumbered copy in front of the student. 1) Place the numbered copy in front of you, but shielded so the student cannot see what you record. 1) Say: “Here are some more make-believe words (point to the student probe). When I say “begin”, start here (point to the first word), go across the page (point across the page), and read the words the best you can. Remember, you can tell me the sound of the letter or read the whole word. Put your finger under the first word. Ready, begin.” 1) Start your stopwatch. If the student does not respond with a sound after 3 seconds, tell the student the sound and say, “next sound?” Things You Need While Testing: Nonsense Word Fluency Standard Directions for 1- Minute Administration (Continued) 5) If the student does not get any sounds correct in words 1-5, discontinue the task and record a score of 0. 6) Follow along on the examiner copy of the probe and underline each phoneme the student provides correctly. Put a slash ( / ) over each phoneme read incorrectly or omitted. 5) At the end of 1 minute, place a bracket ( ] ) after the last letter and say, “Stop.” Things to Do After Testing: Nonsense Word Fluency Students receive 1 point for every correct sound or word provided in 1 minute. b a v = 3 points b a v = 3 points What is a Correct Letter Sound? • • • • • Correct letter sounds Correct words Self Corrections Sounds Produced Out of Order Blended Letter Sounds Let’s Practice! Understanding Norms Tables National norms in AIMSweb are based on all student data in the database nationwide Norms tables will tell you The score that corresponds with each percentile rank The ROI (Rate of Improvement) made for students who scored at that percentile ROI on the norms table is the Spring score minus the Fall score (Winter minus Fall) divided by 36 weeks (or 18 weeks) Size of the sample at each benchmark, mean score, and standard deviation Example: National Norms for 2nd Grade R-CBM Planning for Implementation Planning and Implementation Identify Population to screen Name, Student id, class, school – send to External Coach Review Implementation Checklist in Binder for Action Steps RTI coach will come help you with screening based on your created schedule. Please allow time for up front practice in administration/fidelity check This coach will partner up with the data point person to show how to enter the data and how database works. Could be the same day of administration. This person will have rights to the database. Walk through report options that MMSD will use. Once report(s) are generated, a problem-solving meeting MUST occur. An outcome of this P-S meeting may be to progress monitor a FEW individuals. Additional training will be provided to those that need it.