Reliability and Validity of the Reynolds Intellectual Assessment Scale:
A Brief Literature Review
Guy M. McBride, Ph.D.
School Psychologist
May, 2006
Description:
"The RIAS is an individually administered test of intelligence, which has been normed and
standardized on an age population from 3 to 94 years. Utilizing the Cattell-Horn Model of
Intelligence (Horn and Cattell, 1966), the authors have developed a reliable measure of general
intelligence and its two primary components, fluid and crystallized intelligence. The RIAS includes
a measure of verbal and nonverbal intelligence and a resultant Composite Intelligence Index
(CIX). Verbal intelligence is assessed with tasks involving verbal problem solving and verbal
reasoning tasks. Nonverbal intelligence is assessed by visual and spatial ability tasks. the authors
have eliminated reliance on motor coordination and visual-motor speed for the measurement of
intelligence. A Composite Memory Index (CMX) can be derived from two supplementary subtests
which assess verbal and nonverbal memory. The basic RIAS can be administered by a trained
examiner in 20 to 25 minutes. The CMX adds another 10 minutes of administration time." (Elliot,
2004)
Some Advantages of RIAS:
1. The RIAS provides a "nice option as compared to the majority of extant language
loaded intelligence tests that seem to have grown longer every year." (Bracken, 2006)
2. The RIAS employs "well tested and proven measures of intelligence." (Bracken,
2006.)
3. The RIAS presents strong evidence for reliability. (Bracken, 2006).
4. "Concurrent validity studies with the WISC III and WAIS III yielded comparable
mean scores and overall positive correlations." (Bracken, 2006)
5. "Correlations ranged from .61 to .79 using the VIX, NIX, and CMX composite scores
from the RIAS and three composite scores from the WAIS III." (Schraw, 2006)
Correlations between CIX and FSIQ on the WISC III ranged from .60 to .78 (Edwards,
2006)
6 The RIAS substantially reduces or eliminates "dependence on motor coordination and
visual motor speed in the measurement of intelligence." (Reynolds and Kamphaus, 2003)
7. The RIAS provides a "practical measurement device in terms of efficacies of time,
direct costs, and information needed from a measure of intelligence." (Reynolds and
Kamphaus, 2003).
8. The RIAS allows for continuity of measurement across all developmental levels from
ages 3 through 94 years for both clinical and research purposes. (Reynolds and
Kamphaus, 2003) (The Wechsler series employs three separate tests across those age
ranges, making comparisons of performance on different tests impractical; and even the
DAS uses substantially different subtests for different age ranges, making comparisons of
performance from the preschool age range to school age age range more problematic.
GMM)
9. The RIAS accurately predicts basic academic achievement comparable to IQ tests
twice its length. (Reynolds and Kamphaus, 2003).
10. "The RIAS substantially lessens the time to assess intelligence without
compromising statistical integrity." (Elliott, 2004)
11. "Reliability and validity are comparable, if not stronger, than reliability and validity
measures on similar measures of intelligence that are substantially longer and more
complex." (Elliott, 2004).
12. "It is likely that the RIAS will become a very popular measure of intelligence for
school districts that are under serious pressure to provide measures of intelligence for
more rapid fashion. The RIAS can satsitify that need and should become part of every
school psychologist's battery of tests." (Elliott, 2004)
Discussion:
Although test reviewers talk about construct, concurrent, and predictive validity,
ultimately ALL forms of validity studies reflect on construct validity . . . "Does the test
do what it is supposed to do?" (Joint Standards; Office for Civil Rights, 2000).
In validating the inferences of the test results, it is important to consider the consequences
of the test's interpretation and use.
With respect to the RIAS, the mean and distribution of scores are reportedly similar for
both the RIAS and Wechsler series. Nevertheless, different tests WILL give different
scores. (A child will not score the same on a Woodcock Johnson Tests of Achievement
that he would on the End of Grade test; some children will score higher, some lower.)
A test must be validated for the purpose for which it is being used in order to be in
compliance with the federal regulations. When the Wechsler was last revised, it doubled
the loading on working memory and processing speed, both psychological processing
skills are typically deficient in children with learning disabilities. In essence, this meant
that children who were having difficulty with achievement because of deficits in these
areas were also being penalized on some tests of ability (such as the Wechsler), in
essence creating a case of double jeopardy.
"Do children with learning disabilities score lower on WMI and PSI?
Studies reported in the [WISC IV] Manual suggest that children with
learning and/or attention disorders tend to perform lower on tasks that
measure working memory and processing speed."
(Frequently Asked Questions, Harcourt Brace website)
The Mark Penalty occurs in psychological testing for LD based on a discrepancy
model when the examiner uses a test that is biased by the same psychological
processing deficits that formed the basis, in whole or part, for the child's academic
problems.
Psychological Corporation responded to these criticisms by issuing Technical
Report No. 4, which allows the psychologist, after conducting a post hoc analysis,
to calculate a General Abilities Index to be used in lieu of FSIQ. The GAI,
which calculates a cognitive score based on a combination of the six Verbal and
Performance subtests (excluding WM and PS), will result in a higher score for
children who have been found, after the testing, to have significant problems in
either area.
Indeed, one limited (N=34) study of referred children tested using the WISC IV
and the RIAS confirmed that trend. The children tested had a RIAS CIX that was
on average about 8 to 9 points higher than the FSIQ; but when GAIs were
calculated, that difference fell to three to four points.
Others, like John Willis and Ron Dumont, have argued that it is more appropriate
to eliminate the "contamination" of working memory, processing speed, and
psychomotor skills before the fact, not (as with the Wechsler) after the fact when
testing children suspected of learning disabilities or mental impairment.
"If a student has weaknesses in basic sensory, motor, or psychological processes
(e.g., visual impairment, hearing loss, cerebral palsy, oral language disorder,
word-finding impairment, auditory perception, visual perception, processing
speed, working memory, etc.), we think it is only reasonable to seek a measure of
intelligence that is not contaminated by weaknesses that have been documented
by testing (other than the cognitive ability test itself), observation, classroom
performance, and other sources of information. In fact, Ron Dumont has said that
the intelligence test for a child should not be selected until other assessment is
completed. If you are trying to assess thinking, reasoning, and problem-solving
ability, you do not want to use intelligence tests that require a blind child to copy
block designs or a child with an oral language disorder to define words. You
want an intelligence measure that is not contaminated by the documented
weaknesses, even – or especially – if those weaknesses are in abilities important
to intellectual functioning."
Since the RIAS eliminates many of those potentially contaminating factors up
front, however, the amount of analysis required to determine the tasks'
appropriateness is clearly reduced. (For example, the RIAS would in most cases
not be the best test to use with a blind or deaf child or an ESL child.)
Reliability: A brief comparison
CIX reliability is reported to be about .97; the reliability for the FSIQ on the
WISC IV was reported to be about .98; reliability for GAI on the WISC IV was
reported to be .96. The DAS reports reliabilities of .94 to .95 for preschool
children, and .95 to .96 for school age children.
Validity: A brief review
The DAS reports predictive reliabilities (correlations with achievement tests)
ranging from .52 (spelling) to .60 (Basic Number Skills, Word Reading) when
correlated with scores obtained by children in the normative sample of the DAS
Achievement tests (Colin Elliott, 1983) When compared with other tests,
correlations ranged from .46 (spelling at age 7 and reading age at 11) to .66
(reading at age 7.)
Reported correlations between CIX and WIAT Composites ranged from .60 to
.69. (Reynolds and Kamphaus, 2003). [Verbal Intelligence Index reportedly
correlated .73 (Elliott, 2003).]
Brief Summary of Correlations with Other Cognitive Measures
By way of the comparison, the Differential Ability Scales, the GCA on the DAS
reportedly correlated .75 with the Kaufman Assessment Battery for Children 5-7
years old. It correlated .88 with the Stanford Binet IV for 9 and 10 year olds. It
correlated .84 with the WISC III. For 8 to 10 year olds. (Colin Elliott, 1983)
The RIAS (based on a study of 54 children) correlated .76 with FSIQ on the old
WISC III.
In his study, Edwards (2006) found a correlation of .86 between CIX and FSIQ on
the WISC IV, despite differences between the mean scores.
Criticisms: No review would be complete without criticisms. The most
frequent criticism levied at the RIAS in 2006 was that in the best of all possible
worlds, there would be more studies of predictive validity than were included in
the test manual. No one, however, has suggested that the predictive validity
studies done by Reynolds, et. al, were flawed or that the data reported was less
than satisfactory. Test validation is an on going process, not a finished task
once the first technical manual is issued. See some of the references cited above
for more current opinions.
Summary: The RIAS offers significant advantages over other measures of
cognitive ability when assessing children who are suspected of being SLD and
one reviewer for Buros thought it best for assessing children with cognitive
impairments.
Overall, comparable scores for the general population would be expected.
However, the RIAS may provide higher scores for children who have significant
psychomotor problems or specific problems with working memory or processing
speed (underlying psychological processing deficits oft times associated with
SLD children.) This is not a flaw.
Working memory, processing speed, and psychomotor skill deficits may all
impede academic progress. (The Bender, for example, has a .6 correlation with
some measures of achievement.) They are not unimportant, and they should be
documented. We just shouldn't call them "IQ" or "intelligence."
Guy’s Conclusion: While the RIAS may not be the best tool for assessing
children who are blind, deaf, or who have limited English proficiency, the RIAS
is an appropriate assessment tool for use both in initial and reevaluations. As
Elliot suggested in his review, this reviewer also believes the RIAS should be in
every school psychologist’s assessment toolbox.
References:
Bracken, Bruce, and Gregory Shaw. Review of the Reynolds Intellectual Assessment
Scales. Buros Mental Measurements Yearbook. 16th Edition.
Edwards, Oliver. Referred Students' Scores on the RIAS and WISC IV:
Implications for Test Selection. PowerPoint Presentation, NASP, 2006.
Elliott, Colin. Differential Ability Scales. Introductory and Technical Handbook.
Harcourt Brace. 1983
Elliott, Robert. Reynolds Intellectual Assessment Scales. Archives of Clinical NeuroPsychology. 2003.
http://www3.parinc.com/uploads/pdfs/RIASTestReview.pdf
Reynolds, Cecil. Re: [Nasp Listserv] Rias v. Wechsler. Personal Communication by
email published on NASP Listserv. April, 2006.
Reynolds, Cecil, and Randy Kamphaus. Reynolds Intellectual Assessment Scales.
Professional Manual. Psychological Assessment Resources, Inc. 2003.
Standards for Educational and Psychological Testing. American Educational
Research Association, American Psychological Association, Nationa Council onf
Measurement in Education. 1999.
The Use of Tests as Part of High Stakes Decision Making for Students. A Resource
Guide for Educators and Policy Makers. "Chapter One. Test Measurement
Principles." United States Department of Education. Office for Civil Rights. 2000.
Willis, John. Using the DWI or the GAI. Internet web page. Undated.