Size and Depth of Vocabulary:
What Does the Research Show?
Norbert Schmitt
1
History
•
•
•
•
•
Two year review of vocabulary studies
pertinent to size vs. depth
Results reported at AAAL 2012
Manuscript submitted to Language
Learning
3 reviewers gave plenty of feedback
Latest version: focus on conclusions
2
Size and Depth
•
•
•
Vocabulary has often been characterized in
terms of size vs. depth of knowledge
The distinction is widespread, but one
depth is not easy to pin down
One reviewer states that depth is “the
wooliest, least definable, and least
operationalisable construct in the entirety
of cognitive science past or present”
3
Size and Depth
•
It is time to start thinking about this
distinction more rigorously
The various conceptualizations and
measurements of depth make it difficult to
start from a theoretical framework
So start from an empirical perspective to
inform the debate:
•
•
–
Review all studies that have a measurement of
size and at least one measurement of depth
4
Vocabulary Size
•
Size = the number of lexical items ‘known’
5
Vocabulary Size
•
Size = the number of lexical items ‘known’
(to some criterion of mastery, i.e. depth)
6
Vocabulary Size
•
Size = the number of lexical items ‘known’
(to some criterion of mastery, i.e. depth)
•
Every size test is also a depth test in the
sense that a certain criterion of mastery
must be met
7
Vocabulary Depth
• Depth / Quality = How well do you know
those items?
• What can you do with those items?
• Very broad: can be conceptualized and
operationalized in a variety of ways
8
Size and Depth
• The relationship between size and depth
depends on:
• How both are conceptualized
• How both are measured
9
Conceptualizing Depth
•
Receptive vs. Productive Knowledge
•
Usually connected with the 4 skills
10
Reading, Writing,
Speaking, Listening
•
Depth could be seen as how well word can be
employed in the four skills
•
Vocabulary size correlates with all four skills
•
But little research which shows how well individual
lexical items are employed in the skills
11
What is Involved in Knowing a Word
Nation (2001)
Form
Meaning
Use
spoken
receptive/productive
written

word parts
form and meaning
concept and referents
associations
grammatical functions
collocations
constraints on use (register, frequency …)
12
Size and Depth
• Knowledge of all of the word knowledge
aspects taken together can be
conceptualized as a relatively
comprehensive depth of knowledge
• But each aspect can be known to various
degrees of mastery
13
Degree of Knowledge
Schmitt, 2010a: 38
14
Vocabulary Depth
• So, improving knowledge of any individual word
knowledge aspect can be considered as adding
to depth
• Not an all-or-nothing concept
• Anything that improves mastery can be
considered additional depth
15
Vocabulary Depth
Depth =
•
•
•
•
Degree of mastery of the form-meaning link
Polysemous word meanings
Derivations (word family members)
Collocations
• Other word knowledge aspects but were not
found in research in conjunction with a size
measure
16
Vocabulary Depth
• Depth in this conceptualization concerns
individual lexical items
• Only a small number of items can ever be
measured, so unclear how generalizable the
depth measures can be
17
Conceptualizing Size and Depth
Meara and Wolter, 2004: 89
18
Lexical Organization
•
Concerns lexicon as a whole rather than
individual lexical items
Depth could be seen as any of the word
knowledge connections between items
But how to measure it?
Word associations
•
•
•
–
–
–
–
Difficult to interpret
Idiosyncratic to individuals
Good measure of organization?
WAF main measure, but ?
19
Lexical Fluency
•
Daller, Milton, and
Treffers-Daller (2007)
see fluency as a
separate dimension
•
Can see fluency as depth (i.e. depth does
not have to be knowledge, but can be seen
as employability (skills, automaticity)
20
Recognition and Recall of the
Form-Meaning Link
•
•
•
•
Laufer and Paribakht (1998)
VLT (form recognition)
PVLT (form recall)
Recall-recognition r=.89 (EFL: Israeli high school)
.72 (ESL: Canadian university)
PVLT ÷ VLT ratio
EFL%
ESL%
Combined
77
62
2,000
3,000
5,000
10,000
94
76
62
46
84
58
63
44
21
Depth of Form-Meaning Link
• As vocabulary size increases (and frequency
level decreases), the recall/recognition gap
increases
• Learners are more likely to have both form
recognition and form recall mastery at the
higher frequencies (i.e. smaller gap)
• Less likely to have form recall mastery at the
lower frequency levels (i.e. form recognition
mastery only)
• Form recall lags both form recognition and
meaning recall
22
Knowledge of Written vs.
Spoken Word Forms
• van Zeeland (2013) used a meaning recall interview to
measure the written and spoken vocabulary knowledge of
advanced L2 learners
• Results showed a stronger correlation between written and
spoken word knowledge than found by Milton and Hopkins
(2006) (r = .85 vs. .68).
• The relationship between learners’ knowledge of written and
spoken vocabulary furthermore remained constant as
overall scores increased
• These results suggest that knowledge of vocabulary in the
two modes may be more closely related than suggested by
checklist test results (Milton and colleagues)
23
Depth of Spoken-Written Mastery
• Some evidence that very small vocabularies
might be mainly known phonologically
• Somewhat larger vocabulary sizes shift to
being known predominately orthographically
? Advanced learners tend to have relatively
balanced spoken/written vocabularies, while
lower-level students are prone to the type of
imbalanced vocabularies found by Milton and
24
colleagues?
Knowledge of Derivatives
• Correlations between size and derivation/suffix knowledge
• Schmitt and Meara (1997)
– recall of suffix derivation
– Recognition of suffix derivation
r
.27 and .35
.37 and .41
• Kieffer and Lesaux (2008)
– recall of derivation
.53 and .46
• Kieffer and Lesaux (2012)
– recall of derivation
.50 -.57
• Noro (2002)
– recall function of affix
.42, .54, .69
• Mochizuki and Aizawa (2000)
– recall function of affix
.54 - .65
25
Knowledge of Derivatives
• Milton (2009) reviews Mochizuki and
Aizawa’s results and concludes:
• A vocabulary size of 3,000-5,000 families is
necessary for affixes to be mastered
• But even at 5,000 families, some affixes may
not be known well
26
Depth of Derivative Knowledge
• Size is only modestly related to knowledge of
affixes and derivatives
(system learned before items?)
• Milton suggests that a threshold might exist
(3,000-5,000 families?)
27
Knowledge of Collocation
• Gyllstad (2007)
high-proficiency Swedish ESL
• VLT
• 2 collocation tests
– COLLEX 5: a 3-option form recognition test
– COLLMATCH 3: a yes/no collocation judgement
• Size - collocation (r=.90)
• 10,000 families  >90% on both collocation tests
• 5,000 families
≈ 85%
• 3,000 families
≈ 70%
• With larger vocabulary sizes, it is possible to recognize collocations well
28
Knowledge of Collocation
• Laufer and Waldman (2011) review the literature
and conclude:
– receptive knowledge is related to general vocabulary
knowledge
– productive knowledge of collocation lags behind
knowledge of individual words
– the problem with collocations is not recognition, but in
using them properly, i.e. productive mastery
29
Lexical Fluency
• Laufer and Nation (2001)
Israeli university
• VORST (computerized, timed, modified VLT)
• There was an increase in lexical access speed at a
vocabulary size of around 5,000 word families
• The larger the vocabulary size, the faster the access speed
2,000 – speed
3,000 – speed
5,000 – speed
10,000 – speed
(r = -.38)
(r = -.40)
(r = -.50)
(r = -.67)
30
Depth of Automaticity
• The larger the vocabulary size, the faster the
access speed
• The larger the vocabulary size (and the lower the
frequency level) the stronger the relationship
between size and fluency
• Hint of a threshold: there is an increase in lexical
access speed at a vocabulary size of around 5,000
word families
31
Lexical Organization
• Henriksen (2008)
Danish high school
Grades 7
• VLT – association recognition
• VLT – association recall
.72
.85
10
13
n.s.
.69
n.s.
.55
• The relationship between size and association is stronger at
lower grades than more advanced ones
• Since the students had increasing vocabulary sizes at all
three grades, we can also interpret the results to show a
stronger size-association relationship for smaller vocabulary
sizes than larger ones
32
Lexical Organization
•
•
•
•
Greidanus, et al. (2004)
Advanced Dutch learners of French
Form recognition and form recall
New WAF (paradigmatic, syntagmatic, and analytic)
• Form recognition – association (r= .70)
• Form recall – association
(r= .81)
33
Lexical Organization
• Size – association correlations
.18 .36 .39 .43 .49 .51 .55 .61 .62 .69 .69
.70 .70 .72 .78 .81 .81 .82 .85 .85 .86 .89
• Overall, fairly strong correlations
34
Lexical Organization
• Size – association correlations
.18 .36 .39 .43 .49 .51 .55 .61 .62 .69 .69
.70 .70 .72 .78 .81 .81 .82 .85 .85 .86 .89
Association recall
Association recognition (WAF)
35
Lexical Organization
• Size – association correlations
.18 .36 .39 .43 .49 .51 .55 .61 .62 .69 .69
.70 .70 .72 .78 .81 .81 .82 .85 .85 .86 .89
Lower vocabulary size
Larger vocabulary size
36
Lexical Organization
• 3 studies showed a trend for larger vocabulary sizes having
stronger size – association correlations
• 2 studies showed a trend for larger vocabulary sizes having
weaker size – association correlations
37
Lexical Organization
• 3 studies showed a trend for larger vocabulary sizes having
stronger size – association correlations
• 2 studies showed a trend for larger vocabulary sizes having
weaker size – association correlations
• Does larger size relate to better lexical organization?
• Evidence seems mixed at this point
(Problems with measuring organization?)
38
Tentative Conclusions
• Does greater vocabulary size relate to
greater depth of vocabulary knowledge?
• Yes, generally
• But how strongly depends on what ‘depth’ is
39
Tentative Conclusions
• How one views the size-depth relationship
should depend on one’s purpose of use
• If one wishes to discuss the nature of
vocabulary in general, particularly with
practitioners, then the distinction is useful
40
Tentative Conclusions
• If one’s purpose is to characterize vocabulary
knowledge in more precise terms:
– theorizing
– designing and interpreting research
– assessment
• Depth is probably too vague a term to be
useful
• Need to state lexical aspect addressed and
focus on that
41
Tentative Conclusions
• Virtually all aspects of vocabulary knowledge
seem interrelated
• This makes it difficult to discuss any
particular conceptualization of depth in
isolation
• This makes it difficult to conceptualize overall
depth as anything but the combined
interrelationships between word knowledge
aspects
42
Tentative Conclusions
• The most widely-used vocabulary tests are
size tests, and they typically describe their
results as the number of words ‘known’
• But they do not define what this actually
entails
• Test developers need to explicitly state what
correct answers on their tests entail, and
what degree of depth they represent
43
Tentative Conclusions
• There can be no clear conceptual distinction
between size and depth
• Size by definition is the number of lexical
items ‘known’ to some criterion level of
mastery
• But the criterion will always be some
measure of depth, and so the two will always
be confounded
44
Questions / Comments
Comments to help me understand
size/depth better?
?
Size?
Depth?
www.norbertschmitt.co.uk
45