Chapter 6
Formal Assessment of
Reading: Individualized
Assessment
Introduction
• Development of Formal Reading Measures
Types of Scores
Administration and Scoring of Formal Tests
Determining Chronological Age
Individualized vs Group Tests
What do individualized Norm-referenced Measures of
Reading Look Like?
• Test Bias
• Selecting Formal, Individualized Instruments
• Special Considerations for Formal, Individual Assessment
of Adult and English Language Learners
•
•
•
•
•
Test Development
•
•
•
•
•
Table of specifications (blueprint) (Figure 5.2)
Development version
Pilot or field testing
Standardization version
Norm sample
• Representativeness of sample
Figure 6.1
Types of Scores
• Numerical Scales: Nominal, Ordinal, Interval,
Ratio
• Raw Score: Actual number correct, taking
into account the basal
• Derived Scores
• Standard Scores: Z Scores, IQ Scores, T
Scores, Scale Scores
• Grade Equivalents, Age Equivalents
• Percentiles
Assessment Terms
• Grade Equivalent
• Age Equivalent
• Grade-Based
Norms
• Age-Based Norms
As a Rule, Use AgeBased Norms to
Determine
Standard Scores
• Standard Scorescan Add and
Subtract-Used for
Comparisons
• PercentilesProvides A Rank
Order 1-99
• Stanines-A Rough
Gauge 1-9
Text Box 6.1
Age and Grade Equivalents
Age Equivalent
6-1
6 years, 1 month
Grade Equivalent
1.2
1st grade, 2nd month
Text Box 6.2
Percentile versus Percentage
Percentage
Mastery
Percentile
Misha
50
Jesse
90
16
80
Text Box 6.3
Contrasting Standard Scores for Word Recognition
and Reading Comprehension
Mean = 100; Standard Deviation = 15
Word
Recognition
Shelby
Saimah
85
95
Reading
Comprehension
84
70
Text Box 6.4
Comparing IQ Scores with Reading Achievement
Scores (Mean = 100; Standard Deviation = 15)
Saimah
Standard Score IQ
94
Reading Comprehension
-70
24
Shelby
103
-85
18
Administering and Scoring of
Formal Tests
Test Administration
• Test Manual: Administration Instructions,
Scoring and Norm Information, Reliability and
Validity Data
• Protocol
• Calculation of Chronological Age
• General Guidelines for Test Administration
• Raw Scores
• Establishing Basals and Ceilings
• Derived Scores: Percentiles, Grade Equivalents,
Standard Scores
Test Administration
• Determine
• Do NOT coach,
chronological age
prompt, or give
feedback EXCEPT
• Know start and
as directed in the
stop rules
manual
• Establish rapport
• Obtain raw scoreswith examinee
take
into
account
• Administer subtests
basal and ceiling
according to
• Obtain derived
directions
scores (standard
scores, percentiles,
etc.
Tips for Administering Standardized
Tests
• Text Box 6.5
Word Recognition
Basal Rule: 3 in a row
correct.
Ceiling rule: 3 in row
incorrect.
Figure 6.2. Sample
Reading Subtests
Showing Basal and
Ceiling Rules
1. ___
2. ___
3. ___
→ 4. _1_
5. _1_
6. _1_
7. _1_
8. _1_
9. _1_
10. _0_
11. _0_
12. _0_
Note: Starting point
for this student
indicated by the →
Vocabulary
Basal rule: All items in
a set correct. Ceiling
rule: All items in a set
incorrect.
1. ___
2. ___
3. ___
4. ___
5. ___
→6. _1_
7. _1_
8. _1_
9. _1_
10. _1_
11.
12.
13.
14.
15.
_0_
_0_
_0_
_0_
_0_
Note: Starting point
for this student
indicated by the →
Figure 6.3. Sample Excerpt from a Norm Table for an Individually Administered Reading
Comprehension Test for Students Age 8 years, 9 months. Examiners calculate the raw score for
the test, then look up derived scores (standard scores, percentiles, grade- and age-equivalents)
based on the examinee’s
chronological age.
Raw Score Conversion Table for
Reading Comprehension Test
Age: 8 years, 9 months
Raw
Stand Percen Grade Age
Score ard
tile
Equiva Equivale
Score
lent
nt
↑
15
16
17
18
19
20
↓
95
96
97
98
99
100
37
39
42
45
47
50
2.1
2.4
2.7
2.9
3.2
3.5
7-6
7-9
7-11
8-3
8-6
8-9
Determining Chronological Age
Calculation of Chronological Age
Subtract Date of Birth
from Current Date:
2016
- 2009
06
07
22
14
Months always have
30 days and years
(of course) have 12
months). Always
double check your
work.
Standardized Diagnostic Testing
• When to Use Diagnostic Tests
• Instructional Level
• Informal Instruments
• Probes
• Direct Measurement
• Domains
Individualized vs. Group Tests
Individual versus Group?
Individual
• Special education
eligibility
• Determine goals and
objectives for IFSPs,
IEPs
• Progress monitoring
• May be more reliable
for certain students
Group
• Determine mastery of
curriculum standards
• Determine if teachers
are effective (value
added)
• Controversial but
happening!
• Accountability and
general school
planning
• May be less reliable for
certain students
Test Interpretation
• Analyze test scores
• Is there a pattern of
• Are scores above
strengths and
average, average or
weaknesses?
below average?
• e. g. Areas of learning
disability? basic reading,
• Do the scores yield
reading fluency, reading
any instructional
comprehension?
information?
• Compile with other
data
• Maintain confidentiality
What do Individualized, NormReferenced Measures of
Reading Look Like?
Figure 6.4
Figure 6.5
Figure 6.6
Figure 6.7
Figure 6.8
Figure 6.9
Methods of Assessing
Comprehension
• KTEA-III
• Reading Comprehension subtest
• Picture identification, Answering questions
• WIAT-III
• Reading Comprehension subtest
• Picture identification, Answering questions
• WJ-IV
• Passage Comprehension subtest
• Picture identification, Fill in the blank
• Reading Recall subtest
• Silent reading followed by oral retelling
Comprehension
• Wechsler Individual Achievement Test (WIAT-III)
• Reading Comprehension subtest
• The examinee is required to read short sentences or
passages and respond to comprehension questions
Reading Comprehension
Similar to WIAT-III
Sample Maze Activity
Norm-referenced tests are standardized (without/on/for) a clearly
defined group, termed the (test/sample/norm) group, and scaled
so that each (score/instrument/variance) reflects a rank within the
(evaluation/reliability/norm) group. Psychologists have
developed norm-(standardized/referenced/rated) tests to assess,
for example, (height/attitude/intelligence), reading, mathematics,
writing, etc. Although we (aren’t/are/is) fortunate in having a
choice of (poorly/haphazardly/well)-standardized and
psychometrically sound tests (without/above/with) which to
evaluate children, some (numbers/tests/practitioners) do not meet
acceptable psychometric standards.
(Passage from
Sattler, 2001)
Nelson-Denny Reading Test
• Comprehension
• 7 reading passages
• 38 comprehension questions
• Literal and interpretive question types
• 5 answer choices per question
• 20 minutes
Methods of Assessing
Vocabulary
•WJ-IV: Reading Vocabulary
• Synonyms, Antonyms, Analogies
•Nelson Denny
• Multiple choice
Vocabulary
• Woodcock Johnson - IV Tests of
Achievement (WJ-IV)
• Reading Vocabulary subtest
• The examinee is required to orally
state synonyms and antonyms for
printed words and orally complete
written analogies
Reading Vocabulary
Similar to WJ-IV
Analogies
Antonyms
Synonyms
Nelson-Denny Reading Test
• Vocabulary
• 80 items
• 5 answer choices each
• 15 minute time limit
• Example
• Which word best completes the opening statement?
• A chef works with: A. bricks B. music C.
clothes D. food E. statues
Methods of Assessing Fluency
•KTEA-III
•Decoding Fluency subtest
• Nonsense word list (timed)
•Word Recognition Fluency subtest
• Word list (timed)
• WIAT-III
•Reading Comprehension subtest
• Reading timed passages
•WJ-IV
•Sentence Reading Fluency subtest
• Reading sentences and marking whether statements are true or
false (3 minute time limit)
•Word Reading Fluency
• Choosing two words that go together from an array of choices
Fluency
• The Test of Silent Word Reading Fluency-2
(TOSWRF-2)
• Examinees are required to identify increasingly
difficult words that have no spaces between
them by drawing lines between the boundaries
of as many words as possible within the time
limit (3 minutes)
• The Test of Silent Contextual Word
Reading Fluency (TOSCRF-2)
• Similar to TOSWRF-2 but examinee reads connected text
of increasingly difficult graded passages
TOSWRF-2: Example
Nelson-Denny Reading Test
• Reading Rate
• Part of Comprehension Test
• The examiner calls “Mark” after one minute has
elapsed, and the examinees are asked to record the
number to the right of their current line of text and
continue reading.
Methods of Assessing Phonics
• KTEA-III
• Nonsense Word Decoding subtest
• Nonsense words
• WJ-IV
• Word Attack subtest
• Nonsense words
• WIAT-III
• Pseudoword Decoding subtest
• Nonsense words
Phonics
• Wechsler Individual Achievement Test (WIAT-III)
• Pseudoword Decoding subtest
• Requires the examinee to read aloud a list of nonsense
words designed to mimic the phonetic structure of words
in the English Language.
Pseudoword Decoding
Similar to WIAT-III
•
•
•
•
•
•
•
heb
mib
fum
bim
pon
vun
dreeb
Methods of Assessing Phonemic
Awareness
• KTEA-III
• Phonological Awareness subtest
• Rhyming, Sound Matching, Blending, Segmenting,
Deletion
• WJ-IV
• Spelling of Sounds subtest
• Rhyming, Deletion, Substitution, Reversal
• WIAT-III
• Word Reading subtest
• Rhyming, Sound Matching, Blending,
Phonemic Awareness
• Woodcock Johnson Tests of Achievement (WJ IV)
• Spelling of Sounds subtest
• Requires the examinee to use rhyming, deletion,
substitution, and reversal to manipulate sounds in words.
Spelling of Sounds
Similar to WJ-IV
• Rhyming:
“What rhymes with go?”
• Deletion:
“Say cowboy without saying boy”
• Substitution:
“Change /s/ in sun to /f/.”
• Reversal:
“Listen to the sounds in the word pot - /p/ /o/ /t/. Now
you say the sounds backward.”
Other Measures that Assess
Phonemic Awareness
• CTOPP-2
• Durrell-2
• ERDA-2
• PAL-2 RW
• TOWRE-2
• TERA-3
Methods of Assessing Sight Word
Recognition
• WIAT-II
• Word Reading subtest
• Letter and word identification
• KTEA-II
• Letter and Word Recognition subtest
• Letter and word identification
• WJ-IV
• Letter Word Identification subtest
• Letter and word identification
Sight Word Recognition
• Kaufman Tests of Educational Achievement (KTEA
III)
• Letter and Word Recognition subtest
• The examinee is asked to point to or name various
letters, sounds, and words.
Letter Word Recognition
Similar to KTEA III
Test Bias
• Scores from a test should not provide a
systematic advantage to a particular group or
subgroup based on irregularities in test content,
administration, or interpretation
• If a test produces such an unfair advantage it is
said to be biased against the group that is
disadvantaged (Payne, 2003)
Evidence of Test Bias
• a) Does the test assess the same content for all
the groups using it?
• (b) Are the constructs assessed by the test
measured consistently for all groups using it?
• (c) Does the test predict (later achievement) as
well for all the groups using it?
• (d) Does the test provide equally useful
instructional information for all groups using it?
Avoiding Bias
• Expert judgment
• Statistical analyses: Are the constructs assessed
the same across different groups (e.g., gender,
race, SES level?)
• Remember tests that are not inherently biased
may be misused (e.g., use of test in English for
students with limited English skills)
Selecting Formal,
Individualized Instruments
• Consider test characteristics
• Consider psychometric properties
• Consider needs of children, classrooms, schools,
or systems
• Determine whether a group or individualized test
is needed
Special Considerations for
Formal, Individual Assessment
of Adult and English Language
Learners
Tips for Formal, Individualized
Assessment of Adult Learners
• Text Box 6.7
• Assess adult learners’ educational histories, background
experiences, and interests as well as specific reading skills.
• Multiple assessments will likely be needed to provide complete
information about adult learners’ reading strengths and
weaknesses.
• Ensure that the test used with adults is standardized on that
population.
• Put adult learners at ease; establish rapport and a purpose before
launching into formal testing.
• Using grade equivalent scores, and to a lesser extent age
equivalent scores, is problematic due to ease of misinterpretation
of these score, and their psychometric limitations.
Tips for Adults cont’d
• Tests should include item content appropriate for
adults, rather than young students, even though
the difficulty level of the content may be low in
order to achieve a reasonable test floor.
• Consider entering your test data and using the
instructional skills profile available at the Adult
Reading Components Study website:
https://lincs.ed.gov/readingprofiles/resources.ht
m
Tips for Formal, Individualized Assessment of
English Language Learners
• Text Box 6.8
• Users should follow standards set for test administration,
scoring and interpretation in general and for ELL and other
diverse learners as developed by experts. Many of these
standards are reproduced in Chapter 6; some of the most
relevant are reproduced in this Text Box in abbreviated
form.
• Be aware that an ELL student’s formal test scores will be
compared to students in the U.S., who may not resemble
the ELL in background, culture, knowledge, etc.
• Related to Number 2 above, an ELL’s scores on formal tests
may be depressed because of cultural, linguistic, and
knowledge differences.
Tips for ELLs cont’d
• Do not administer a language-loaded test for ELL
examinees unless the language content in the test is the
focus of assessment.
• Related to Number 4 above, if a student’s English language
proficiency is the target of assessment English languageloaded tests (i.e., English reading tests) are appropriate
• Ensure that the test does not contain item content that may
be offensive in the ELL’s home country or culture.
• Ensure that reliability and validity of the test have been
established for ELLs.
• Ensure that the items are not biased against ELL learners
by evaluating recommendations from expert panels.
Tips for ELLs cont’d
• Ensure that items are not biased against ELL learners by
examining evidence from statistical techniques designed to
determine bias.
• When possible, use an examiner who can communicate in the
ELL’s native language to establish rapport and aid in administering
the test.
• Be aware that an ELL’s progress in learning a second language
and ability to understand test demands may be limited by
negative transfer from the native language.
• Reduce importance of speeded performance on scores; some
cultures do not value speed.
• Ensure that the test only measures what you intend to measure
(i.e., specific types of reading skills) and not some irrelevant
content.
Tips for ELLs cont’d
• If your ELL students score below the mean,
ensure that if mean differences occur, the
differences are not the result of bias.
• Use ample teaching items to ensure that ELL
learners understand the task demands.
• Using grade equivalent scores, and to a lesser
extent age equivalent scores, is problematic due
to each of misinterpretation of these score, and
psychometric limitations.
Assessment at a Glance: Formal,
Individualized Assessment
• Tables 6.1 and 6.2
• Characteristics of Formal, Individualized, NormReferenced Assessments of Reading
• Psychometric Properties of Formal, Individualized,
Norm-Referenced Assessments of Reading
Summary
• Development of Formal Reading Measures
Types of Scores
Administration and Scoring of Formal Tests
Determining Chronological Age
Individualized vs Group Tests
What do individualized Norm-referenced Measures of
Reading Look Like?
• Test Bias
• Selecting Formal, Individualized Instruments
• Special Considerations for Formal, Individual Assessment
of Adult and English Language Learners
•
•
•
•
•