Transition Academy @ Shadd
October 5, 2010
Woodcock Johnson III
Tests of Achievement
(Adapted from Riverside Publishing Materials)
Purpose of Academic Testing
• Identify areas of academic strength and
weakness
• Diagnose specific learning disabilities
• Develop recommendations to address
academic deficiencies
Learning Disabilities
•
Two operational definitions:
–
–
•
Significant discrepancy between ability and
achievement
Response to intervention
Different from DSM-IV-TR Learning
Disorder, yet related.
Issues with Academic Testing
• Age norms vs. Grade norms
– Kids enter school at different ages
– Children can be accelerated or retained in grade
– Which normative sample to use?
• If you are computing ability-achievement
discrepancies, you will need to use age norms.
• However, if the student has been retained or
accelerated in grade, should also re-compute
scores using grade norms.
Psychometrics of WJ-III ACH
• Major strength of the WJ-III ACH test battery –
co-normed with the WJ-III COG test battery.
• Standardization sample: 8,818 subjects
• 100+ geographic areas (including Moorpark!)
• Reliability:
– For individual subtests across all ages: .70 to .96
– For clusters across all ages: .93 to .99
– Alternative form reliabilities for Passage
Comprehension range between .85 and .96 over all
ages.
Psychometrics of WJ-III ACH
• Validity
– Differential correlational patterns between different
cluster scores suggest discriminant validity.
– Factor structure aligns with the narrow Cattell-HornCarroll abilities
• Validity studies with the KTEA and WIAT show
modest agreement
– Total Achievement on WJ-III with KTEA: .79
– Total Achievement on WJ-III with WIAT: .65
– Convergent and Discriminant validity coefficients are
a little higher e.g.:
• WJ-III Broad Reading with WIAT Reading Composite: .82
• WJ-III Broad Reading with WIAT Math Composite: .50
Psychometrics of WJ-III ACH
• Discriminant validity with WJ-III COG,
WAIS-III, and Kaufman Adolescent &
Adult Intelligence testing produces lower
coefficients (most below .50).
Organization
• Achievement (BLUE)
– Standard Battery
– 12 Tests
– 10 Clusters
– Extended Battery
– 10 Tests
– 9 Clusters
– (Form A & B)
Clusters
•
•
•
•
•
•
•
•
•
•
•
Broad Reading
Basic Reading Skills
Broad Math
Math Calculation Skills
Math Reasoning
Broad Written Language
Written Expression
Academic Knowledge
Academic Skills
Academic Fluency
Oral Language
Cluster and Subtest Structure
• Broad Reading
– Letter-Word Identification
– Reading Fluency
– Passage Comprehension
• Basic Reading Skills
– Letter-Word Identification
– Word Attack (extended battery, commonly
administered)
• Broad Math
– Calculation
– Math Fluency
– Applied Problems
Cluster and Subtest Structure
• Math Calculation Skills
– Calculation
– Math Fluency
• Math Reasoning
– Applied Problems
– Quantitative Concepts (extended battery)
• Broad Written Language
– Spelling
– Writing Fluency
– Writing Samples
Cluster and Subtest Structure
• Written Expression
– Writing Fluency
– Writing Samples
• Academic Knowledge
– Academic knowledge (3 extended subtests: Sci, SS,
Hum)
• Academic Skills
– Letter-Word Identification
– Spelling
– Calculation
Cluster and Subtest Structure
• Academic Fluency
– Reading Fluency
– Writing Fluency
– Math Fluency
• Oral Language
– Story Recall
– Understanding Directions
Cluster and Subtest Structure
• Total Achievement
–
–
–
–
–
–
–
–
–
Letter-Word Identification
Reading Fluency
Passage Comprehension
Spelling
Writing Fluency
Writing Samples
Calculation
Math Fluency
Applied Problems
“IQ tests are a poor way to judge
people's abilities, failing as they
do to account for magic, which
has its own importance, both by
itself and as a compliment to
logic.”
- Robert James Waller
Test Administration:
Tools & Techniques
• Strive for exact & brisk administration
• Have necessary materials
–
–
–
–
–
–
–
–
Test books
Test records
Subject response booklet
Tapes
Tape player
Headphones, Y-jack, Earpiece
Pencils
Stopwatch/Timer
Administration Tool:
Test Session Observation Checklist
• Level of conversational
proficiency
• Level of cooperation
• Level of activity
• Attention & concentration
• Self-confidence
• Care in responding
• Response to difficult tasks
Administration Time (ACH)
• Preliminary Testing at BC
– Intake Booklet &
– Ach Tests 2, 5, 6, 7, 8, 11
– Approx. 120 minutes
• Standard Achievement
– Tests 1, 3, 4, 9, 10
– 60 minutes
• Extended Achievement
– Tests 13 - 18
– 60 minutes
Standard Battery
•
•
•
•
•
•
Letter-Word
Identification
Reading Fluency
(timed)
Story Recall (taped)
Understanding
Directions (taped)
Calculation
Math Fluency (timed)
•
•
•
•
•
•
Spelling
Writing Fluency
(timed)
Passage
Comprehension
Applied Problems
Writing Samples
Story Recall –
Delayed
Extended Battery
• Word Attack
• Picture Vocabulary
• Oral Comprehension
(taped)
• Editing
• Reading Vocabulary
•
•
•
•
•
Quantitative Concepts
Academic Knowledge
Spelling of Sounds
Sound Awareness
Punctuation &
Capitalization
Administration Techniques
•
•
•
•
•
•
Know your tools
Follow standardized procedures
Use suggested starting points
Know basal and ceiling rules
Know pronunciations
Don’t penalize mispronunciations due to
speech deficits or regional differences
• Student must smoothly pronounce words
for credit (automaticity is critical)
“If you think you are over
your head with a client, you
probably are.”
- Richard Bolles
Administration Techniques
• Test by complete pages when
subject sees items
• Score LAST response given
• Give benefit of doubt every other
time
• Don’t repeat items unless
directed
• Use pause button if needed
• If interrupted by a sound, repeat
item later
Administration Techniques
• Adhere to time limits
• Record exact finishing times
• On timed written tests, direct
student to cross out errors
• Check questionable responses
later; proceed as if correct
“We never do anything well till
we cease to think about the
manner of doing it.”
- William Hazlitt
“The IQ and the life expectancy
of the average American
recently passed each other
going in the opposite direction.”
- George Carlin
Tests of Achievement
Standard Battery
&
Extended Battery
Tests of Achievement
• Standard Battery
–
–
–
–
–
–
–
–
–
–
–
–
Letter-Word Identification
Reading Fluency
Story Recall
Understanding Directions
Calculation
Math Fluency
Spelling
Writing Fluency
Passage Comprehension
Applied Problems
Writing Samples
Story Recall – Delayed
• Extended Battery
–
–
–
–
–
–
–
–
–
–
Word Attack
Picture Vocabulary
Oral Comprehension
Editing
Reading Vocabulary
Quantitative Concepts
Academic Knowledge
Spelling of Sounds
Sound Awareness
Punctuation &
Capitalization
Standard Battery
• Test 1: Letter-Word Identification
– Identifying printed letters and words
• Test 2: Reading Fluency
– Reading printed statements rapidly and responding true or false
(Y or N)
• Test 3: Story Recall
– Listening to and recalling details of stories
• Test 4: Understanding Directions
– Listening to a sequence of instructions and then following the
directions (by pointing)
• Test 5: Calculation
– Performing various mathematical calculations; retrieving math
facts.
Standard Battery
• Test 6: Math Fluency
– Adding, subtracting, and multiplying rapidly
• Test 7: Spelling
– Spelling orally presented words
• Test 8: Writing Fluency
– Formulating and writing simple sentences rapidly with a visual
cue
• Test 9: Passage Comprehension
– Completing a sentence by giving the missing key word that
makes sense in the context.
• Test 10: Applied Problems
– Performing math calculations in response to orally presented
problems with a visual cue.
Standard Battery
• Test 11: Writing Samples
– Writing meaningful sentences for a given purpose
with a visual and oral prompt.
• Test 12: Story Recall – Delayed
– Recalling previously presented story elements
• Test 13: Word Attack
– EXTENDED Battery test, often included in most
standard battery administrations
– Reading phonically regular non-words
Extended Battery
• Test 13: Word Attack
– Reading phonically regular non-words
• Test 14: Picture Vocabulary
– Naming pictures
• Test 15: Oral Comprehension
– Completing an oral sentence by giving the
missing key word that makes sense in the context
• Test 16: Editing
– Identifying and correcting errors in written
passages
Extended Battery
• Test 17: Reading Vocabulary
– Reading words and supplying synonyms and
antonyms; reading and completing verbal analogies
• Test 18: Quantitative Concepts
– Identifying math terms and formulae; identifying
number patterns
• Test 19: Academic Knowledge
– Responding to questions about science, social studies,
and humanities
• Test 20: Spelling of Sounds
– Spelling letter combinations that are regular patterns
in written English
Extended Battery
• Test 21: Sound Awareness
– Providing rhyming words; deleting,
substituting, and reversing parts of words to
make new words.
• Test 22: Punctuation & Capitalization
– Applying punctuation and capitalization rules
“Sometimes when I'm in a
bookstore or a library, I am
overwhelmed by all the things
that I do not know.”
- Arthur C. Clarke
Basal and Ceiling Rules
• For almost all tests (exceptions are noted in the
scoring record sheets), to obtain a valid score
on any subtest, you need a basal level of 6
items correct and a ceiling level of 6 items
incorrect.
• Add up the total number of items correct
(including items BELOW basal) = Raw Score.
• Enter Raw Scores in the WJ-III Scoring
Assistant (hand scoring not really an option).
Scoring Output
•
•
•
W-scores
– Equal-interval scale where 500 is set to approximate the average
performance of a 10-0 child.
– Used to measure growth.
Age Equivalents / Grade Equivalents
– Raw score cross-referenced to the median score within the age/grade
band.
– Highly variable.
RPI – Relative Proficiency Index
– Predicts a student’s level of proficiency on tasks that typical age- or
grade-level peers would perform with 90% proficiency.
– RPI of 55/90 indicates impairment – they did 55%, a peer would do
90%
– RPI of 98/90 indicates strength – they did 98%, a peer would do
90%
Scoring Output
• Percentile Ranks
• Standard Scores
– Mean 100; SD 15
– Used in determining ability-achievement
discrepancies.
• Z Scores
“Your mind should be a place
where you work things out, not
store a bunch of stuff.”
- Albert Einstein
2. Reading Fluency 
• Rapidly read simple sentences and mark whether
it is true or false
• Start with Item 1
• 3 minute time limit
• Skipped items are scored “0”
• If fewer than 3 correct on Practice Exercises C-F,
discontinue and score “0”
• Enter Number Correct and Number Incorrect
• Used in Broad Reading, Academic Fluency,
Total Achievement
3. Story Recall
• Use tape
• Student listens to story on tape, then retells it
• Must repeat Bold elements exactly; other parts
may be paraphrased (in any order)
• Use suggested starting points
• Follow continuation rules
• Pause tape player after each story
• Do not repeat or replay any stories
• Used in Oral Language (Std & Ext), Oral
Expression
4. Understanding Directions 
• Use Tape; pause tape if needed
• Student points to items on picture according to taped
instructions
• Allow student 10 seconds to view each picture before
starting tape
• Use suggested starting points; follow discontinuation
instructions
• Lay Test Manual flat
• Tip: Only watch for correct answers – don’t try to
figure it out
• Used in Oral Language (Std & Ext), Listening
Comprehension
6. Math Fluency 
•
•
•
•
•
•
Timed Math facts
Begin with Item 1
3 minutes timed
Do not point out signs
Lay booklet open
Discontinue if 3 or fewer correct in 1 minute
• Used in Broad Math, Math Calculation Skills,
Academic Fluency, Total Achievement
7. Spelling ()
• Spelling (not usage & punctuation)
• Basal/Ceiling = 6/6
• Used in Broad Written Language, Basic
Writing Skills, Academic Skills, Total
Achievement
8. Writing Fluency 
• Student quickly writes simple sentences using
prompt words
• Begin with Item 1
• 7 Minutes timed
• Discontinue if “0” correct on Samples or fewer
than 3 correct in first 2 minutes
• Read stimulus words as needed
• Stimulus words cannot be altered
• Used in Broad Written Language, Written
Expression, Academic Fluency, Total Achievement
Tests of Achievement:
Extended Battery
Word Attack
Picture Vocabulary
Oral Comprehension
Editing
Reading Vocabulary
Quantitative Concepts
Academic Knowledge
Spelling of Sounds
Sound Awareness
Punctuation/Capitalization
Preliminary Testing (at BC)
Special Instructions
Item
Intake Booklet &
Consent Form
3 Minutes Timed
Test 2: Reading Fluency
Test 5: Calculation
3 Minutes Timed
Test 6: Math Fluency
Use Tape
Test 7: Spelling
7 Minutes Timed
Test 8: Writing Fluency
Use Tape
Test 11: Writing Samples
Other:
Administered By:
Date & Initials
“To avoid criticism, do nothing,
say nothing, be nothing.”
- Elbert Hubbard
Scoring and Reporting
Normative Update
Normative Updates (Nu)
• Check accuracy of all scores
• Launch Normative Update
Program
–
–
–
–
Set up program once:
Click <Options>
Click <Program Options>
Under “Additional Score
Column” choose CALP
– For “Confidence Band” choose
68%
– For “Discrepancy Cutoff”
choose 1.30
Normative Update Continued
• For Verbal & Numeric Data:
• Enter scores for Cognitive & Achievement, Standard &
Extended Batteries
• Enter observations
• Click <Report> <Summary & Score Report>
• Choose Age
• Select “Include GIA/BIA”
• For “Ability/Ach” choose “GIA-Extended”, then also run
“Oral Language” or “GIA-Standard”
• For “Intra-Ability” choose “Intra-Cog and/or Intra-Ach”
• Send to “Print Preview”
• Click <OK>
Normative Update Report
• To see next page in
report, press PAGE
DOWN key
• Click icon to Print
the report
• Click icon to Close
Preview (or press
ESC)
Normative Update Continued
• For Graphical Data:
• Click <Report>, <Age/Grade
Profiles>
• Choose “Age”
• Choose “Include GIA/BIA”
• For “Data to Include” choose
“Standard Score”
• Send report to “Print Preview”
• Click <OK>
“Machines increase the number of
things we can do without thinking.
Things we do without thinking there's the real danger.”
- Frank Herbert