Transition Academy @ Shadd October 5, 2010 Woodcock Johnson III Tests of Achievement (Adapted from Riverside Publishing Materials) Purpose of Academic Testing • Identify areas of academic strength and weakness • Diagnose specific learning disabilities • Develop recommendations to address academic deficiencies Learning Disabilities • Two operational definitions: – – • Significant discrepancy between ability and achievement Response to intervention Different from DSM-IV-TR Learning Disorder, yet related. Issues with Academic Testing • Age norms vs. Grade norms – Kids enter school at different ages – Children can be accelerated or retained in grade – Which normative sample to use? • If you are computing ability-achievement discrepancies, you will need to use age norms. • However, if the student has been retained or accelerated in grade, should also re-compute scores using grade norms. Psychometrics of WJ-III ACH • Major strength of the WJ-III ACH test battery – co-normed with the WJ-III COG test battery. • Standardization sample: 8,818 subjects • 100+ geographic areas (including Moorpark!) • Reliability: – For individual subtests across all ages: .70 to .96 – For clusters across all ages: .93 to .99 – Alternative form reliabilities for Passage Comprehension range between .85 and .96 over all ages. Psychometrics of WJ-III ACH • Validity – Differential correlational patterns between different cluster scores suggest discriminant validity. – Factor structure aligns with the narrow Cattell-HornCarroll abilities • Validity studies with the KTEA and WIAT show modest agreement – Total Achievement on WJ-III with KTEA: .79 – Total Achievement on WJ-III with WIAT: .65 – Convergent and Discriminant validity coefficients are a little higher e.g.: • WJ-III Broad Reading with WIAT Reading Composite: .82 • WJ-III Broad Reading with WIAT Math Composite: .50 Psychometrics of WJ-III ACH • Discriminant validity with WJ-III COG, WAIS-III, and Kaufman Adolescent & Adult Intelligence testing produces lower coefficients (most below .50). Organization • Achievement (BLUE) – Standard Battery – 12 Tests – 10 Clusters – Extended Battery – 10 Tests – 9 Clusters – (Form A & B) Clusters • • • • • • • • • • • Broad Reading Basic Reading Skills Broad Math Math Calculation Skills Math Reasoning Broad Written Language Written Expression Academic Knowledge Academic Skills Academic Fluency Oral Language Cluster and Subtest Structure • Broad Reading – Letter-Word Identification – Reading Fluency – Passage Comprehension • Basic Reading Skills – Letter-Word Identification – Word Attack (extended battery, commonly administered) • Broad Math – Calculation – Math Fluency – Applied Problems Cluster and Subtest Structure • Math Calculation Skills – Calculation – Math Fluency • Math Reasoning – Applied Problems – Quantitative Concepts (extended battery) • Broad Written Language – Spelling – Writing Fluency – Writing Samples Cluster and Subtest Structure • Written Expression – Writing Fluency – Writing Samples • Academic Knowledge – Academic knowledge (3 extended subtests: Sci, SS, Hum) • Academic Skills – Letter-Word Identification – Spelling – Calculation Cluster and Subtest Structure • Academic Fluency – Reading Fluency – Writing Fluency – Math Fluency • Oral Language – Story Recall – Understanding Directions Cluster and Subtest Structure • Total Achievement – – – – – – – – – Letter-Word Identification Reading Fluency Passage Comprehension Spelling Writing Fluency Writing Samples Calculation Math Fluency Applied Problems “IQ tests are a poor way to judge people's abilities, failing as they do to account for magic, which has its own importance, both by itself and as a compliment to logic.” - Robert James Waller Test Administration: Tools & Techniques • Strive for exact & brisk administration • Have necessary materials – – – – – – – – Test books Test records Subject response booklet Tapes Tape player Headphones, Y-jack, Earpiece Pencils Stopwatch/Timer Administration Tool: Test Session Observation Checklist • Level of conversational proficiency • Level of cooperation • Level of activity • Attention & concentration • Self-confidence • Care in responding • Response to difficult tasks Administration Time (ACH) • Preliminary Testing at BC – Intake Booklet & – Ach Tests 2, 5, 6, 7, 8, 11 – Approx. 120 minutes • Standard Achievement – Tests 1, 3, 4, 9, 10 – 60 minutes • Extended Achievement – Tests 13 - 18 – 60 minutes Standard Battery • • • • • • Letter-Word Identification Reading Fluency (timed) Story Recall (taped) Understanding Directions (taped) Calculation Math Fluency (timed) • • • • • • Spelling Writing Fluency (timed) Passage Comprehension Applied Problems Writing Samples Story Recall – Delayed Extended Battery • Word Attack • Picture Vocabulary • Oral Comprehension (taped) • Editing • Reading Vocabulary • • • • • Quantitative Concepts Academic Knowledge Spelling of Sounds Sound Awareness Punctuation & Capitalization Administration Techniques • • • • • • Know your tools Follow standardized procedures Use suggested starting points Know basal and ceiling rules Know pronunciations Don’t penalize mispronunciations due to speech deficits or regional differences • Student must smoothly pronounce words for credit (automaticity is critical) “If you think you are over your head with a client, you probably are.” - Richard Bolles Administration Techniques • Test by complete pages when subject sees items • Score LAST response given • Give benefit of doubt every other time • Don’t repeat items unless directed • Use pause button if needed • If interrupted by a sound, repeat item later Administration Techniques • Adhere to time limits • Record exact finishing times • On timed written tests, direct student to cross out errors • Check questionable responses later; proceed as if correct “We never do anything well till we cease to think about the manner of doing it.” - William Hazlitt “The IQ and the life expectancy of the average American recently passed each other going in the opposite direction.” - George Carlin Tests of Achievement Standard Battery & Extended Battery Tests of Achievement • Standard Battery – – – – – – – – – – – – Letter-Word Identification Reading Fluency Story Recall Understanding Directions Calculation Math Fluency Spelling Writing Fluency Passage Comprehension Applied Problems Writing Samples Story Recall – Delayed • Extended Battery – – – – – – – – – – Word Attack Picture Vocabulary Oral Comprehension Editing Reading Vocabulary Quantitative Concepts Academic Knowledge Spelling of Sounds Sound Awareness Punctuation & Capitalization Standard Battery • Test 1: Letter-Word Identification – Identifying printed letters and words • Test 2: Reading Fluency – Reading printed statements rapidly and responding true or false (Y or N) • Test 3: Story Recall – Listening to and recalling details of stories • Test 4: Understanding Directions – Listening to a sequence of instructions and then following the directions (by pointing) • Test 5: Calculation – Performing various mathematical calculations; retrieving math facts. Standard Battery • Test 6: Math Fluency – Adding, subtracting, and multiplying rapidly • Test 7: Spelling – Spelling orally presented words • Test 8: Writing Fluency – Formulating and writing simple sentences rapidly with a visual cue • Test 9: Passage Comprehension – Completing a sentence by giving the missing key word that makes sense in the context. • Test 10: Applied Problems – Performing math calculations in response to orally presented problems with a visual cue. Standard Battery • Test 11: Writing Samples – Writing meaningful sentences for a given purpose with a visual and oral prompt. • Test 12: Story Recall – Delayed – Recalling previously presented story elements • Test 13: Word Attack – EXTENDED Battery test, often included in most standard battery administrations – Reading phonically regular non-words Extended Battery • Test 13: Word Attack – Reading phonically regular non-words • Test 14: Picture Vocabulary – Naming pictures • Test 15: Oral Comprehension – Completing an oral sentence by giving the missing key word that makes sense in the context • Test 16: Editing – Identifying and correcting errors in written passages Extended Battery • Test 17: Reading Vocabulary – Reading words and supplying synonyms and antonyms; reading and completing verbal analogies • Test 18: Quantitative Concepts – Identifying math terms and formulae; identifying number patterns • Test 19: Academic Knowledge – Responding to questions about science, social studies, and humanities • Test 20: Spelling of Sounds – Spelling letter combinations that are regular patterns in written English Extended Battery • Test 21: Sound Awareness – Providing rhyming words; deleting, substituting, and reversing parts of words to make new words. • Test 22: Punctuation & Capitalization – Applying punctuation and capitalization rules “Sometimes when I'm in a bookstore or a library, I am overwhelmed by all the things that I do not know.” - Arthur C. Clarke Basal and Ceiling Rules • For almost all tests (exceptions are noted in the scoring record sheets), to obtain a valid score on any subtest, you need a basal level of 6 items correct and a ceiling level of 6 items incorrect. • Add up the total number of items correct (including items BELOW basal) = Raw Score. • Enter Raw Scores in the WJ-III Scoring Assistant (hand scoring not really an option). Scoring Output • • • W-scores – Equal-interval scale where 500 is set to approximate the average performance of a 10-0 child. – Used to measure growth. Age Equivalents / Grade Equivalents – Raw score cross-referenced to the median score within the age/grade band. – Highly variable. RPI – Relative Proficiency Index – Predicts a student’s level of proficiency on tasks that typical age- or grade-level peers would perform with 90% proficiency. – RPI of 55/90 indicates impairment – they did 55%, a peer would do 90% – RPI of 98/90 indicates strength – they did 98%, a peer would do 90% Scoring Output • Percentile Ranks • Standard Scores – Mean 100; SD 15 – Used in determining ability-achievement discrepancies. • Z Scores “Your mind should be a place where you work things out, not store a bunch of stuff.” - Albert Einstein 2. Reading Fluency • Rapidly read simple sentences and mark whether it is true or false • Start with Item 1 • 3 minute time limit • Skipped items are scored “0” • If fewer than 3 correct on Practice Exercises C-F, discontinue and score “0” • Enter Number Correct and Number Incorrect • Used in Broad Reading, Academic Fluency, Total Achievement 3. Story Recall • Use tape • Student listens to story on tape, then retells it • Must repeat Bold elements exactly; other parts may be paraphrased (in any order) • Use suggested starting points • Follow continuation rules • Pause tape player after each story • Do not repeat or replay any stories • Used in Oral Language (Std & Ext), Oral Expression 4. Understanding Directions • Use Tape; pause tape if needed • Student points to items on picture according to taped instructions • Allow student 10 seconds to view each picture before starting tape • Use suggested starting points; follow discontinuation instructions • Lay Test Manual flat • Tip: Only watch for correct answers – don’t try to figure it out • Used in Oral Language (Std & Ext), Listening Comprehension 6. Math Fluency • • • • • • Timed Math facts Begin with Item 1 3 minutes timed Do not point out signs Lay booklet open Discontinue if 3 or fewer correct in 1 minute • Used in Broad Math, Math Calculation Skills, Academic Fluency, Total Achievement 7. Spelling () • Spelling (not usage & punctuation) • Basal/Ceiling = 6/6 • Used in Broad Written Language, Basic Writing Skills, Academic Skills, Total Achievement 8. Writing Fluency • Student quickly writes simple sentences using prompt words • Begin with Item 1 • 7 Minutes timed • Discontinue if “0” correct on Samples or fewer than 3 correct in first 2 minutes • Read stimulus words as needed • Stimulus words cannot be altered • Used in Broad Written Language, Written Expression, Academic Fluency, Total Achievement Tests of Achievement: Extended Battery Word Attack Picture Vocabulary Oral Comprehension Editing Reading Vocabulary Quantitative Concepts Academic Knowledge Spelling of Sounds Sound Awareness Punctuation/Capitalization Preliminary Testing (at BC) Special Instructions Item Intake Booklet & Consent Form 3 Minutes Timed Test 2: Reading Fluency Test 5: Calculation 3 Minutes Timed Test 6: Math Fluency Use Tape Test 7: Spelling 7 Minutes Timed Test 8: Writing Fluency Use Tape Test 11: Writing Samples Other: Administered By: Date & Initials “To avoid criticism, do nothing, say nothing, be nothing.” - Elbert Hubbard Scoring and Reporting Normative Update Normative Updates (Nu) • Check accuracy of all scores • Launch Normative Update Program – – – – Set up program once: Click <Options> Click <Program Options> Under “Additional Score Column” choose CALP – For “Confidence Band” choose 68% – For “Discrepancy Cutoff” choose 1.30 Normative Update Continued • For Verbal & Numeric Data: • Enter scores for Cognitive & Achievement, Standard & Extended Batteries • Enter observations • Click <Report> <Summary & Score Report> • Choose Age • Select “Include GIA/BIA” • For “Ability/Ach” choose “GIA-Extended”, then also run “Oral Language” or “GIA-Standard” • For “Intra-Ability” choose “Intra-Cog and/or Intra-Ach” • Send to “Print Preview” • Click <OK> Normative Update Report • To see next page in report, press PAGE DOWN key • Click icon to Print the report • Click icon to Close Preview (or press ESC) Normative Update Continued • For Graphical Data: • Click <Report>, <Age/Grade Profiles> • Choose “Age” • Choose “Include GIA/BIA” • For “Data to Include” choose “Standard Score” • Send report to “Print Preview” • Click <OK> “Machines increase the number of things we can do without thinking. Things we do without thinking there's the real danger.” - Frank Herbert