Vocabulary Knowledge and Speaking Proficiency
1
Supplementary materials for Koizumi and In’nami (in press)
Koizumi, R., & In’nami, Y. (in press). Vocabulary knowledge and speaking proficiency
among second language learners from novice to intermediate levels. Journal of Language
Teaching and Research.
Table 1
Note. Table 1 did not include Uenishi (2005), because his study included many affective and
linguistic factors as independent variables in the regression and SEM analyses, and only
reported regression coefficients, on account of which we found it difficult to interpret his
results.
Current study
Vocabulary size and depth represent declarative knowledge (knowledge skills, according to
the classification of De Jong et al. 2012, in press), and involve, for example, the
form-meaning link, while speed is procedural knowledge (processing skills, according to De
Jong et al. 2012, in press), enabling learners to access and retrieve word form and meaning
quickly (see Wood, 2010, for details). Based on De Jong et al. (2012), we considered
speaking proficiency as an individual, rather than a socially constructed trait. We also
followed Housen and Kuiken (2009) in regarding speaking proficiency as consisting
primarily of fluency, accuracy, and SC, which are often abbreviated as CAF (complexity,
accuracy, and fluency). They called for research into how CAF are associated with internal
variables, such as learners’ language knowledge, and the current study responds to their call.
Finally, our approach is similar to De Jong et al. (in press), in terms of inspecting associations
between cognitive fluency (reflected in vocabulary knowledge) and utterance fluency
(reflected in fluency measures).
Study 1: Instruments
In the derivation test, 20 derivational suffixes were selected from Levels 2 to 4 in Bauer and
Nation (1993). All words used as stimuli and answers were selected from among the
3,000 most frequent lemmas in the JACET8000. Word class information (i.e., noun, verb,
or adjective) was provided in the prompt, which may have elicited metalinguistic
knowledge as well as derivation knowledge.
In the vocabulary tests, relatively high frequency vocabulary was targeted because, as Milton
et al. (2010) argued, words with higher frequency are used more often in speech than in
written texts (Adolphs & Schmitt, 2003), and thus high frequency vocabulary should be
tested when correlating with speaking.
Study 1: Procedures and Analyses
After solving questions in the size test, test takers waited until the test administrator
instructed them to move to the derivation test. In answering subsequent tests, test takers were
not allowed to go back to the size test. These procedures were followed because some items
in the size test could be answered using information in the other tests.
The speaking test had two versions with different orders of tasks. The order of one version
was Tasks 1, 2, 3, 4, and 5, whereas that of the other was Tasks 1, 5, 4, 2, and 3. This was
done to partially counterbalance order effects and decrease the chances of cheating by
copying other test-takers’ utterances.
Vocabulary Knowledge and Speaking Proficiency
2
References
Adolphs, S., & Schmitt, N. (2003). Lexical coverage of spoken discourse. Applied Linguistics,
24, 425–438.
Albrechtsen, D., Haastrup, K., & Henriksen, B. (2008). Vocabulary and writing in a first and
second language: Processes and development. New York: Palgrave MacMillan.
Anderson, R. C., & Freebody, P. (1981). Vocabulary knowledge. In J. T. Guthrie (Ed.),
Comprehension and teaching: Research reviews (pp. 77–117). Newark, DE: International
Reading Association.
Bauer, L., & Nation, P. (1993). Word families. International Journal of Lexicography, 6,
254–279.
Bonk, W. J. (2000). Second language lexical knowledge and listening comprehension.
International Journal of Listening, 14, 14–31.
Council of Europe. (2001). Common European Framework of Reference for Languages:
Learning, teaching, assessment. Cambridge University Press.
Crossley, S. A., Salsbury, T., McNamara, D. S., & Jarvis, S. (2011). What is lexical
proficiency? Some answers from computational models of speech data. TESOL Quarterly,
45, 182–193. doi: 10.5054/tq.2010.244019
Daller, H., Milton, J., & Treffers-Daller, J. (2007). Editors’ introduction: Conventions,
terminology and an overview of the book. In H. Daller, J. Milton, & J. Treffers-Daller
(Eds.), Modelling and assessing vocabulary knowledge (pp. 1–32). Cambridge University
Press.
Henriksen, B. (1999). Three dimensions of vocabulary development. Studies in Second
Language Acquisition, 21, 303–317.
Higgs, T. V.,  Clifford, R. (1982). The push toward communication. In T. V. Higgs (Ed.),
Curriculum, competence, and the foreign language teacher (pp. 57–79). Lincolnwood, IL:
National Textbook.
Hulstijn, J. H. (2011). Language proficiency in native and nonnative speakers: An agenda for
research and suggestions for second-language assessment. Language Assessment Quarterly,
8, 229–249. doi: 10.1080/15434303.2011.565844
In’nami, Y., & Koizumi, R. (2011). Structural equation modeling in language testing and
learning research: A review. Language Assessment Quarterly, 8,
250–276.doi:10.1080/15434303.2011.565844
Ishii, T., & Schmitt, N. (2009). Developing an integrated diagnostic test of vocabulary size
and depth. RELC Journal, 40, 5–22. doi:10.1177/0033688208101452
Koizumi, R. (2011). Test-taking processes of the Lexical Organisation Test: Comparing it
with the Word Associates Test. ARELE (Annual Review of English Language Education in
Japan), 22, 153–168.
Koizumi, R., & In’nami, Y. (2012). Modeling fluency, accuracy, and syntactic
complexity of speaking performance: A structural equation modeling approach.
Manuscript submitted for publication.
Kormos, J., & Dénes, M. (2004). Exploring measures and perceptions of fluency in the
speech of second language learners. System, 32, 145–164.
doi:10.1016/j.system.2004.01.001
Laufer, B., & Nation, P. (1999). A vocabulary-size test of controlled productive ability.
Language Testing, 16, 33–51. doi: 10.1177/026553229901600103
Meara, P. (1996). The dimensions of lexical competence. In G. Brown, K. Malmkjaer, & J.
Williams (Eds.), Performance & competence in second language acquisition (pp.
35–53). Cambridge University Press.
Meara, P., & Fitzpatrick, T. (2000). Lex 30: An improved method of assessing productive
Vocabulary Knowledge and Speaking Proficiency
3
vocabulary in an L2. System, 28, 19–30. doi:
http://dx.doi.org/10.1016/S0346-251X(99)00058-5
Mehnert, U. (1998). The effects of different length of time for planning on second language
performance. Studies in Second Language Acquisition, 20, 83–108.
Moinzadeh, A., & Moslehpour, R. (2012). Depth and breadth of vocabulary knowledge:
Which really matters in reading comprehension of Iranian EFL learners? Journal of
Language Teaching and Research, 3, 1015–1026. doi:10.4304/jltr.3.5.1015-1026
Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge University
Press.
Read, J. (2004). Plumbing the depths: How should the construct of vocabulary be defined? In
P. Bogaards & B. Laufer (Eds.), Vocabulary in a second language: Selection, acquisition,
and testing (pp. 209–227). Amsterdam, the Netherlands: John Benjamins.
Segalowitz, N. (2010). Cognitive bases of second language fluency. New York: Routledge.
Snellings, P., van Gelderen, A., & de Glopper, K. (2002). Lexical retrieval: An aspect of
fluent second-language production that can be enhanced. Language Learning, 52,
723–754. doi: 10.1111/1467-9922.00202
Uenishi, K. (2005). An empirical study on factors predicting the speaking ability of Japanese
EFL learners. Unpublished Ph.D. dissertation, Hiroshima University, Japan.
Yuan, F., & Ellis, R. (2003). The effects of pre-task planning and on-line planning on fluency,
complexity and accuracy in L2 monologic oral production. Applied Linguistics, 24,
1–27.doi: 10.1093/applin/24.1.1
APPENDIX 1: TASKS IN THE SPEAKING TEST IN STUDY 1
Task
Speaking type;
Instructions (originally written in Japanese)
Content
1
Description;
Please introduce yourself to Ms. Smith. Please state your name and
Self-introduction talk about your family and friends first. If you do not know what to
say, you can talk about anything you want (e.g., your school and
likes and dislikes).
2
Picture
Differences exist between the two pictures. Please locate these
comparison;
differences. Please talk about the marked objects first.
Comparing
pictures on the
left and the right
3
Picture
Describe the picture in as much detail as possible so that Ms. Smith,
description;
who is not looking at the picture, can understand what is in it.
Picture
of Please talk about the marked behaviors first.
washing dishes
4
Picture
Same as Task 3
description;
Picture of riding
bicycles
5
Picture
There are pictures above and below. Your brother (Jiro) is
comparison;
mischievous. While you were away at school, he scattered your
Comparing
belongings in your room. Describe what in the room has changed
Taro’s
rooms and how by saying, “something was something before, but now
before and after
something is something else.”
Vocabulary Knowledge and Speaking Proficiency
4
APPENDIX 2: DESCRIPTIVE STATISTICS IN STUDY 1
M
SD
Minimum Maximum Skewness Kurtosis
Size (k = 78)
29.21 10.21 8.00
54.00
0.22
–0.52
Derivation (k = 20)
7.92
3.67
0.00
19.00
0.07
–0.52
Antonym (k = 17)
6.72
2.92
0.00
14.00
0.14
–0.47
Collocation (k = 18) 10.63 2.84
2.00
18.00
–0.36
0.57
T1 Speed F
49.52 18.18 8.00
106.67
0.30
–0.47
T1 Repair F
5.02
4.97
0.00
22.67
1.12
0.84
T1 Accuracy
.75
.20
.00
1.00
–.81
.76
T1 SC
1.10
0.16
0.60
1.63
0.79
0.76
T2 Speed F
39.64 17.29 9.33
102.67
0.66
0.36
T2 Repair F
6.19
5.76
0.00
26.67
1.21
1.13
T2 Accuracy
.26
.25
.00
1.00
.88
.41
T2 SC
0.89
0.21
0.20
1.25
–1.82
2.50
T3 Speed F
33.86 14.64 9.33
82.67
0.60
0.10
T3 Repair F
5.45
5.55
0.00
29.33
1.40
1.94
T3 Accuracy
.59
.28
.00
1.00
–.15
–.59
T3 SC
1.07
0.21
0.33
2.00
1.58
6.26
T4 Speed F
37.55 15.41 5.33
82.67
0.48
–0.26
T4 Repair F
4.67
4.86
0.00
29.33
1.67
3.85
T4 Accuracy
.54
.26
.00
1.00
–.35
–.45
T4 SC
0.98
0.18
0.20
1.50
–1.30
4.59
T5 Speed F
40.18 15.92 10.67
89.33
0.69
0.42
T5 Repair F
6.45
5.66
0.00
28.00
1.08
1.22
T5 Accuracy
.34
.28
.00
1.00
.46
–.51
T5 SC
0.93
0.29
0.17
2.00
0.33
3.10
Note. T = Task. F = Fluency. SC = Syntactic complexity. These also apply to other
appendixes.
Vocabulary Knowledge and Speaking Proficiency
5
APPENDIX 3: CORRELATIONS USED FOR MODELS 1 TO 4
2 3 4 5
6
7
8
9 10 11 12 13
14
15
16
17
18
19
20
21
22 23 24
.77 .80 .66 .30 .15 .35 .19 .42 .31 .10 .36 .48 .18 .34 .12 .50 .26 .31 .28 .41 .29 .32 .22
1. Size
-- .68 .63 .25 .16 .19 .13 .37 .24 .04 .32 .44 .15 .27 .15 .41 .26 .25 .28 .34 .21 .19 .16
2. Derivation
-- .61 .32 .14 .26 .13 .43 .31 .07 .26 .47 .13 .32 .11 .53 .26 .30 .25 .44 .28 .27 .21
3. Antonym
-- .22 .17 .22 .13 .34 .23 .07 .29 .40 .15 .26 .09 .37 .21 .23 .22 .29 .20 .18 .15
4. Collocation
5. T1 Speed F
-- .28 .07 .26 .62 .36 –.12 .14 .62 .31 .19 .09 .69 .34 .11 .28 .64 .40 .15 .18
6. T1 Repair F
-- –.02 –.06 .38 .43 –.02 .15 .35 .47 –.07 –.08 .43 .53 –.03 .06 .34 .45 .11 .16
7. T1 Accuracy
-- .23 .16 .09 .22 .29 .20 –.02 .33 .08 .21 .02 .27 .19 .07 .10 .15 .14
8. T1 SC
-- .10 .08 .08 .13 .15 .03 .17 .08 .16 .05 .13 .13 .05 .05 .13 .10
9. T2 Speed F
-- .42 –.13 .30 .70 .41 .17 .12 .71 .40 .11 .28 .71 .46 .16 .27
10. T2 Repair F
-- –.01 .18 .46 .50 .11 .00 .45 .51 .11 .16 .48 .50 .19 .21
11. T2 Accuracy
-- .02 –.12 –.07 .23 .00 –.07 –.13 .13 –.08 –.13 –.09 .01 .07
12. T2 SC
-- .23 .08 .13 .09 .22 .05 .23 .20 .19 .15 .11 .18
13. T3 Speed F
-- .32 .16 .09 .74 .43 .18 .25 .63 .40 .13 .24
14. T3 Repair F
-- .00 .02 .34 .57 .00 .12 .33 .61 .07 .18
15. T3 Accuracy
-- .18 .22 –.02 .32 .26 .16 .12 .25 .04
16. T3 SC
-- .04 .04 .08 .17 .09 .01 .09 .02
17. T4 Speed F
-- .38 .23 .25 .69 .50 .25 .31
18. T4 Repair F
-- –.02 .21 .38 .52 .12 .16
19. T4 Accuracy
-- .36 .11 .15 .27 .10
20. T4 SC
-- .26 .25 .28 .23
21. T5 Speed F
-- .42 .13 .22
22. T5 Repair F
-- .15 .24
23. T5 Accuracy
-- .37
24. T5 SC
--
Note. *p< .05: from .14 to .17. **p< .01: from .18 or more.
Vocabulary Knowledge and Speaking Proficiency
6
APPENDIX 4: MULTIVARIATE NORMALITY AND MODEL FITS IN STUDY 1 (N =
224)
Mardia’s
coefficient
Criteria
Models 1 & 2: Vocabulary
nonnormal: >
5.00
–0.63
Satorra-Bentl
er scaled χ2
(df) p
p > .05
CFI
TLI
RMSEA
[90%CI]
SRMR
> .90
< 0.08
< .08
4.20 (2) .12
1.00
>
0.90
0.99
0.07
[0.00,
0.07]
0.05
[0.04,
0.06]
0.04
[0.03,
0.05]
.01
Model 3: Speaking
9.25
262.37 (164)
< .01
.93
0.92
Model 4: Vocabulary and
speaking
7.15
346.70 (242)
< .01
.95
0.94
.06
.06
Note. CFI = comparative fit index; TLI = Tucker-Lewis index; RMSEA = root mean square
error of approximation; CI = confidence interval; SRMR = standardized root mean square
residual. This also applies to Table 6.
Vocabulary Knowledge and Speaking Proficiency
7
APPENDIX 5: Mplus Input for the Monte Carlo Analysis to Determine the Precision and
Power of Parameters for Study 2
TITLE:
EFA Three-FACTOR, NORMAL DATA, NO MISSING
MONTECARLO:
NAMES ARE X1-X8;
NOBSERVATIONS = 87;
! SAMPLE SIZE OF
INTEREST
NREPS = 1000;
SEED = 53567;
MODEL POPULATION:
Voc BY X1@1 X2*.34 X3*-11.97;
Speak BY X4@1 X5*.83 X6*.11 X7*.01 X8*.05;
X1*88.80; X2*12.34 X3*76348.76;
X4*27.73; X5*28.87; X6*7.48; X7*0.08; X8*1.21; Speak*20.70;
Speak ON Voc*.65
MODEL:
Voc BY X1@1 X2*.34 X3*-11.97;
Speak BY X4@1 X5*.83 X6*.11 X7*.01 X8*.05;
X1*88.80; X2*12.34 X3*76348.76;
X4*27.73; X5*28.87; X6*7.48; X7*0.08; X8*1.21; Speak*20.70;
Speak ON Voc*.65
ANALYSIS:
ESTIMATOR = ML;
OUTPUT:
TECH9;
APPENDIX 6: Mplus Output for the Monte Carlo Analysis to Determine the Precision and
Power of Parameters for Study 2
Population
parameter
Criteria
Sample
parameters
averaged
SD of
sample
parameters
Standard
error of
sample
parameters
Mean
square
error of
parameters
95%
Coverage
.91-.98 is
OK
Power
(% sig
coefficient)
0.80
or
more
VOC By
X1
16.22
16.07
1.66
1.65
2.79
0.94 1.00
X2
5.55
5.50
0.59
0.58
0.35
0.94 1.00
X3
5.55
5.50
0.59
0.58
0.35
0.94 1.00
SPEAK By
X4
1.00
1.00
0.00
0.00
0.00
1.00 0.00
X5
0.83
0.83
0.08
0.08
0.01
0.95 1.00
X6
0.11
0.11
0.03
0.03
0.00
0.95 0.98
X7
0.01
0.01
0.00
0.00
0.00
0.94 0.94
X8
0.05
0.05
0.01
0.01
0.00
0.95 0.99
SPEAK On
VOC
10.52
10.43
1.15
1.13
1.32
0.95 1.00
Note. The column labels were partially changed from original Mplus outputs to enhance
clarity. VOC By X1 refers to a path from the Vocabulary factor to the X1 variable.
Vocabulary Knowledge and Speaking Proficiency
APPENDIX 7: ESTIMATES OF PATH COEFFICIENTS IN MODEL 4
Parameter
B
Standard error β
Vocabulary -->
Size
1.00a
-.94
Derivation
0.31*
0.02
.82
Antonym
0.26*
0.01
.86
Collocation
0.21*
0.02
.72
Vocabulary -->
Speed fluency
0.83*
0.10
.57
Repair fluency
0.12*
0.03
.36
Accuracy
0.07*
0.001
.63
SC
0.003* 0.001
.66
Speed fluency -->
T1 Speed fluency
1.00a
-.76
T2 Speed fluency
1.05*
0.08
.84
T3 Speed fluency
0.88*
0.07
.83
T4 Speed fluency
0.98*
0.07
.88
T5 Speed fluency
0.92*
0.07
.80
a
Repair fluency --> T1 Repair fluency
1.00
-.65
T2 Repair fluency
1.22*
0.14
.68
T3 Repair fluency
1.28*
0.15
.74
T4 Repair fluency
1.12*
0.15
.75
T5 Repair fluency
1.31*
0.16
.75
a
Accuracy -->
T1 Accuracy
1.00
-.53
T2 Accuracy
0.51*
0.20
.22
T3 Accuracy
1.49*
0.25
.56
T4 Accuracy
1.42*
0.25
.56
T5 Accuracy
1.19*
0.25
.45
SC -->
T1 SC
1.00a
-.30
T2 SC
1.90*
0.60
.43
T3 SC
0.93*
0.43
.21
T4 SC
2.02*
0.66
.53
T5 SC
2.32*
0.78
.39
a
Note. Fixed to 1.00 for scale identification. *p< .05. **p< .01. These also
appendixes.
8
R2
.88
.67
.73
.51
.32
.13
.40
.44
.58
.70
.69
.77
.64
.42
.47
.55
.56
.56
.28
.05
.32
.32
.20
.09
.19
.04
.28
.15
apply to other
APPENDIX 8: COVARIANCES BETWEEN MEASUREMENT ERRORS IN MODEL 4
Parameter
Covariance Standard error r
D1 (Speed fluency) <-->
D2 (Repair fluency)
20.46*
3.15
.60
D3 (Accuracy)
–0.02
0.09
–.02
D4 (SC)
0.17*
0.07
.41
D2 (Repair fluency) <-->
D3 (Accuracy)
–0.03
0.03
–.14
D4 (SC)
0.03*
0.02
.27
D3 (Accuracy) <-->
D4 (SC)
0.002*
0.001
.79
Vocabulary Knowledge and Speaking Proficiency
9
APPENDIX 9: Vocabulary tests used in Study 2

Note. The upper left column shows one item in the J8VST. The target word is mention,
meaning of which is provided in L1 as “write or speak about something; refer to something
briefly.” One correct option, three distractors, and one option labeled “I don’t know” are
provided. The upper right column shows the LOT, in which test takers select one link with
the strongest connection between two words out of the three links; to select a link, test takers
need to move the ball in the center to the slot on the link. The bottom left and right columns
show the two tasks of the LEXATT; on the left, test takers start pushing the button in the
bottom center and the target word trip appears; they release the button when they recognize
the word form and meaning. The response time, from when the test takers begin to push the
button to when they release it, is measured. In the bottom right column, they choose one
option out of two to demonstrate their understanding of the word meaning (trap and trip).
APPENDIX 10: DESCRIPTIVE STATISTICS IN STUDY 2 (N = 87)
M
SD
Minimum Maximum Skewness
Size (k = 125)
Depth (k = 50)
Speed (k = 40)
Processing efficiency
Speed fluency
Repair fluency
Accuracy
SC
100.26
24.28
755.08
40.59
16.10
3.95
0.62
2.43
18.76
6.57
337.71
12.61
10.95
3.04
0.32
1.27
42.00
9.00
274.00
20.00
1.00
0.00
0.00
1.00
123.00
41.00
2233.00
69.00
40.50
14.50
1.00
8.00
–1.04
0.24
1.22
0.41
0.55
0.80
–0.68
1.60
Kurtosi
s
0.50
–0.23
3.04
–0.77
–0.75
0.49
–0.52
3.80
Vocabulary Knowledge and Speaking Proficiency
10
APPENDIX 11: CORRELATIONS USED FOR MODELS 5 TO 7
Size
Depth
Speed
Processing efficiency
Speed fluency
Repair fluency
Accuracy
SC
Depth
.74**
--
Speed
–.51**
–.44**
--
Efficiency Speed F
.71**
.65**
.71**
.69**
–.47**
–.49**
-.80**
--
Repair F
.34**
.30**
–.25*
.38**
.46**
--
Accuracy
.56**
.36**
–.43**
.43**
.36**
.07
--
SC
.41**
.40**
–.33**
.46**
.41**
.12
.26*
--
APPENDIX 12: MULTIVARIATE NORMALITY AND MODEL FITS IN STUDY 2 (N =
87)
Model
Vocabulary
Model
Speaking
5:
Mardia’s
coefficient
nonnormal:
> 5.00
4.51
6:
Model
Vocabulary
speaking
7:
and
Criteria
Satorra-Bentler scaled χ2
(df) p
p> .05
CFI
TLI
> .90
--
1.24
-- (Not computed because
of df = 0)
7.22 (5) .20
>
0.90
--
.99
0.97
2.79
28.80 (19) .05
.97
0.95
RMSEA
[90%CI]
< 0.08
SRMR
--
--
0.07
[0.00,
0.18]
0.08
[0.00,
0.13]
.05
APPENDIX 13: ESTIMATES OF PATH COEFFICIENTS IN MODEL 7
Parameter
B
Standard error β
a
Vocabulary --> Size
1.00
-.87
Depth
0.34*
.04
.85
Speed
–11.97*
2.24
–.58
Vocabulary --> Speaking
0.65*
.07
.92
a
Speaking -->
Processing efficiency 1.00
-.91
Speed fluency
0.83*
.08
.87
Repair fluency
0.12*
.03
.43
Accuracy
0.01*
.003
.48
SC
0.06*
.01
.50
R2
.75
.71
.33
.84
.83
.76
.19
.23
.25
< .08
.06