The Role of Nonverbal Ability Tests
in Identifying Academically Gifted
Students: An Aptitude Perspective
David Lohman
The University of Iowa
http://faculty.education.uiowa.edu/dlohman
Overview
• Background
• Nonverbal tests
– Advantages
– Disadvantages
• Understanding abilities
• Aptitude perspective
– For minority students
• Recommendations
Background
Why use nonverbal tests?
1. Measure abilities in ways that are fair to
all students
2. Increase the diversity in programs for
academically gifted and talented
3. Actively assist children who have not had
the advantages of wealth or who have not
from birth been immersed in English
Other factors to consider
• Get the right kids, not just the right number
• Especially critical for minority students
– Next generation of writers, scientists,
mathematicians
• Crafting policy for the identification and
development of a diversity of academic
talents
Nonverbal Tests
• Present visual stimuli (objects, line
drawings) and
• Require a nonverbal response (assemble a
puzzle, point, fill in a circle)
• “Nonverbal” describes the test, not the
cognitive processes used to solve items
• Involvement of verbal processes
– Explicit (UNIT Analogic reasoning subtest)
– Implicit (Figural Reasoning tests)
UNIT Analogy
Advantages
• Reduced oral/written language load
• Verbal knowledge, verbally mediated
strategies can be in any language
• Reduced mean differences between
monolingual and bilingual students
Disadvantages
•
Pictorial tests
1. Deciphering line drawings
2. Shorter directions are not necessarily better
directions
3. Unforeseen linguistic confusions
UNIT Analogy
CogAT Figure Analogy
J
:
K
L
M
N
Disadvantages
• Figural reasoning tests
– Task specificity greater than for V or Q
Task Specificity 1
Test 1
Task Specificity
Test 1
+
Test 2
=
Task Specificity
Figural
Verbal
Disadvantages
• Figural reasoning tests
– Task specificity greater than for V or Q
– Large practice effects
– Largest Flynn effect
Example
Flynn
Effect Effect
of Flynn
Example of
105
100
IQ Score
95
90
85
80
75
70
1910
1920
1930
1940
1960
1950
1970
1980
Year
Gains in Wechsler-Binet IQ for the U.S. White population.
Sources J. Horgan (1995) and D. Schildlovsky.
1990
2000
Disadvantages
• Figural reasoning tests
–
–
–
–
Task specificity greater than for V or Q
Large practice effects
Largest Flynn effect
Appearance of measuring something innate
Fluid-Crystallized Continuum (1)
Fluid
Cognitive
abilities
Physical
skills
General fluid
ability ( Gf)
General
physic al
fitness
Crystallized
Science
achievement
Math
achievement
Basketball
Social s tudies
achievement
Knowledge of
literature
Football
Specific
factual
knowledge
Volleyball
Swimming
Wrestling
Field hockey
Cycli ng
Fluid-Crystallized Continuum (2)
Fluid
Cognitive
abilities
Physical
skills
General fluid
ability ( Gf)
General
physic al
fitness
Crystallized
Science
achievement
Math
achievement
Basketball
Social s tudies
achievement
Knowledge of
literature
Football
Specific
factual
knowledge
Volleyball
Swimming
Wrestling
Field hockey
Cycli ng
Disadvantages
• Figural reasoning tests
–
–
–
–
–
Task specificity greater than for V or Q
Large practice effects
Largest Flynn effect
Appearance of measuring something innate
Appearance of being culture fair
Culture fair?
• Intuitively plausible but long discredited
idea
• Anastasi & Urbina (1997) Psychological
Testing (7th ed.)
– “no test can be equally fair to all cultures”
– “nonlanguage tests may be more culturally
loaded than language tests”
Cronbach quote
• Cronbach (1990) Essentials of
Psychological Testing (5th ed).
– “no behavioral evidence is culture free.”
– “the term ‘culture fair’ makes a dubious claim”
Scarr quote
• Scarr (1994) In Sternberg’s Encyclopedia of
Intelligence
– “intelligence and ability tests sample human
cultural knowledge, acquired (through)
development.”
– “Although tests such as the Raven Matrices
may seem fair… puzzle-like tasks turn out to
have their own limitations.”
CogAT Figure Analogy
J
:
K
L
M
N
Disadvantages
• Figural reasoning tests
–
–
–
–
–
–
Task specificity greater than for V or Q
Large practice effects
Largest Flynn effect
Appearance of measuring something innate
Appearance of being culture fair
Distal predictors of academic success
Example
r =PR.6
using
Example r = .6of
100
90
Ach.
Mathematics
Mathematics
Achievement
80
70
60
50
40
30
20
10
0
0
10
20
30
40
50
60
Nonverbal Reasoning
70
80
90
100
Example
r = PR
.6
Example
r = .6 using
100
90
Mathematics Ach.
Mathematics Achievement
80
70
60
50
40
30
20
10
0
0
10
20
30
40
50
60
Nonverbal Reasoning
70
80
90
100
Example
r = PR
.6
r = .6 using
Example
100
90
Ach.
Mathematics
Mathematics Achievement
80
70
60
50
40
30
20
10
0
0
10
20
30
40
50
60
Nonverbal Reasoning
70
80
90
100
Mathematics Ach.
Example r = .6
Construct Representation
Verbal
g
Quantitative
Nonverbal
What predicts academic achievement?
Academic
Achievement
Nonverbal
Reasoning
Verbal
Reasoning
Quantitative Reasoning
Selecting students on the basis of a test
of nonverbal reasoning ability would:
• admit many students who are unprepared
for advanced instruction in mathematics or
science or other content-rich domains.
• exclude many students who either have
already demonstrated high levels of
accomplishment in these domains OR
whose high verbal or quantitative reasoning
abilities make them much more likely to
succeed in such programs.
Figural reasoning as an inaptitude?
1
40
50
60
99
Verbal
Quantitative
Nonverbal
N+
1
Verbal
Quantitative
Nonverbal
40
50
60
99
N-
Figural reasoning as an inaptitude?
• Students with an N+ profile do less well in
school than students with an N- profile
• Gohm, Humphreys & Yao (1998) find high
spatial students do poorly on a wide range
of academic outcomes.
• High N, high Q = engineer profile
• Cannot look at Nonverbal test alone
Extravagant Claims, Unlikely
Promises
• NNAT claims
– culture fair
– a “very good” predictor of school achievement
– small and inconsequential difference in mean
scores for White, Black, & Hispanic students
– identify equal proportions of high-scoring
White, Black, & Hispanic students
Predict achievement?
• r (NNAT, Reading) = .56
• r (NNAT, Reading in Spanish) = .32
• r (NNAT, Math) = .6
• r (CogAT, Reading) = .80
• r (CogAT, Math) = .81
Small mean differences between ethnic
groups? equal proportions of high scorers?
• Exceedingly implausible
– e.g. NAEP differences 1 SD
– Matrix format much studied
• Sample is small and unrepresentative
– 5.6 % Urban school children
– More high SES Hispanics & Blacks
• Numbers do not add up
– W-B and W-H differences inconsistent
– means < 100 for all three groups
– SD’s all greater than 15
Demographics: Urbanicity
Black
Urbanicity
NNAT
U.S.

Urban
11.0
54.9
-43.9
Suburban
56.1
31.7
24.4
Rural
32.8
13.3
19.5
Note. Urbanicity is defined as in the U.S. Census.
Demographics: SES
NNAT
SES
White
Black
Hispanic
Low
19.2
20.8
42.0
Low middle
20.1
26.2
29.3
Middle
20.4
8.4
3.0
High middle
23.7
19.5
6.2
High
16.6
25.2
19.5
Note. SES was a composite of median family income in
the community and the percent of adults with high school
diplomas.
Mean W-B, W-H differences
W-B
• Naglieri & Ronning (2000) 4.2
• Naglieri & Ford (2003)
3.2
W-H
2.8
2.0
Aptitude Perspective
• Aptitude is
– the degree of readiness to perform well in a
particular situation or fixed domain.
• Examples
– Ability to comprehend instructions
– To use previously acquired knowledge and skill
appropriately
– To make good inferences and generalizations
– To manage one’s emotions
Academic accomplishment 1
Learning
Context
1
Learning
Context
2
Person characteristics
Academic accomplishment 2
Academic
Accomplishment/
Expertise
Performance
Assessments
Teacher
grades/
evaluations
On-grade and
above-grade
achievement tests
Predicting Math Achievement in
Grades 1-12 from CogAT 6
CogAT
Verbal
CogAT
Quantitative
CogAT
Nonverbal
Mathematics
Achievement
Predicting Math Achievement in
Grades 1-12 from CogAT 6
CogAT
Verbal
.23
CogAT
Quantitative
.50
.15
CogAT
Nonverbal
Multiple R = .80
Mathematics
Achievement
Predicting Reading Comprehension/
Vocabulary in Grades 1-12 from CogAT 6
CogAT
Verbal
CogAT
Quantitative
CogAT
Nonverbal
Reading
Comprehension/
Vocabulary
Predicting Reading Comprehension/
Vocabulary in Grades 1-12 from CogAT 6
CogAT
Verbal
.63
CogAT
Quantitative
.15
.07
CogAT
Nonverbal
Multiple R = .80
Reading
Comprehension/
Vocabulary
Predicting Reading Comp/ Vocab for All
Students (Hispanics) grades 1-6
CogAT
Verbal
.66 (.72)
CogAT
Quantitative
CogAT
Nonverbal
.14 (.12)
Reading
Comprehension/
Vocabulary
.06 (.04)
Multiple R = .81 (.80)
Predictors of Achievement
• The regression equations that best predict
achievement in Reading, Mathematics,
Social Studies, & Science from CogAT
Verbal, Quantitative, and Nonverbal
reasoning are the same for White, Black,
Hispanic, and Asian-American students
• Other investigators find the same (Keith)
Predicting Future Achievement
Grade 4
Grade 9
Achievement
Predicting Future Achievement
Grade 4
Reading
Achievement
Grade 9
.35
Reading
Achievement
Verbal
Reasoning
.30
Predicting Future Achievement
Grade 4
Math
Achievement
Quant.
Reasoning
Nonverbal
Reasoning
Grade 9
.27
.19
.19
Math
Achievement
Recap
1. Structure of abilities the same within ethnic
groups
2. Predictors of concurrent achievement are the
same in White, Black, Hispanic, & AsianAmericans
3. Best predictors of future achievement in a
domain are current achievement in that domain
and the ability to reason in the symbol system(s)
used to communicate new knowledge in the
domain
4. Therefore….
Bilingual students
Verbal achievement in both L1 and L2 depend on a common
set of verbal processes
– Phonemic awareness in Spanish predicts reading in
English (Lindsey et al. 2003)
– Grades in English are more strongly related to
(Swedish) verbal abilities than are grades in Swedish
(Gustafsson & Balke, 1993).
– Best predictors of learning French are verbal abilities
and achievements in English -- not mathematical or
figural reasoning abilities (Carroll, 1981)
Aptitude versus Achievement
• Estimates of academic aptitude must always
be judged relative to circumstances.
• Estimates of academic attainment, on the
other hand, must be made on a scale that is
similar for all.
Common Cut Scores?
• Current
Accomplishment
Common standards more
reasonable
• Potential for future
accomplishment
Common standards not
defensible
Lead to the search for
aptitude tests that predict
achievement but not group
differences in achievement
Get more kids, but more of
the wrong kids
Distinguishing Present Accomplishment from
Predicted Accomplishment
99
Accomplishment
Accelerate
95
?
90
One year later
99
95
90
Improve
Same
Decline
Who is most likely to improve?
• Strongest reasoning abilities in the symbol
systems used to communicate knowledge in
the domain
• Best, most appropriately challenging
instruction
• Motivation and persistence
Guidelines
1. Except for very young children, academic
giftedness should be defined primarily by
measures of academic accomplishment.
2. The primary cognitive aptitudes for future
academic accomplishment are domainspecific achievement and the ability to
reason in the symbol systems used to
communicate new knowledge.
3. The predictors of achievement are the
same in different ethnic groups.
4. Use the nonverbal test as a helpful adjunct, but as
a measure of last resort.
5. Provide different levels of challenge to those who
have already exhibited academic excellence and
those who are working to attain it
6. Use common aptitude measures but uncommon
cut scores (e.g., rank within group) when
identifying minority students most likely to profit
from intensive instruction.
The Role of Nonverbal Ability Tests
in Identifying Academically Gifted
Students: An Aptitude Perspective
David Lohman
The University of Iowa
http://faculty.education.uiowa.edu/dlohman