The Possibility of the Application
of Neural Test Theory to SLA
Research
Kenichi YAMAKAWA*1, Naoki, SUGINO*2, Yuko SHIMIZU*2,
Hiromasa OHBA*3, & Michiko NAKANO*4
*1: Yasuda Women’s University, *2: Ritsumeikan University,
*3: Joetsu University of Education, *4: Waseda University
AAAL 2010 Annual Conference: March 6, 2010, Sheraton Atlanta Hotel, Atlanta, GA
*The present study is supported by Grants-in-Aid for Scientific Research 1348064
(2001-2003), 16320078 (2004-2006) and 19320091 (2007-2009) from the Japan
Society for Promotion of Science.
Abstract summary
This poster explores a possibility of using Neural Test Theory in SLA
research. A grammaticality judgment task aiming to explore the
acquisition of unaccusative and unergative verbs was administered
to 369 Japanese EFL learners. It was found that the learners could
be categorized into four latent ranks.
Continuous ability scale
87 pts
85pts
30pts
Test scores
Students’ language ability on
a continuous scale
(= test scores)
Problems！
•Tests do not have enough resolution to continuously
evaluate human ability. Tests are at best capable of
ranking test takers into 5 to 20 ranks.
• It is difficult to clarify the relationship between
scores on a continuous scale and ability specification.
Based on http://antlers.rd.dnc.ac.jp/~shojima/ntt/index.htm
What is NTT (Neural Test Theory) ?
NTT is a test theory designed for evaluating
achievements on an ordinal scale. Each level
is called a “latent rank”.
Latent ranks are based on the response
patterns of the test takers.
The number of latent ranks is determined by
the data analyst. Goodness-of-fit indices are
used in this determination. (Note 1)
Based on http://antlers.rd.dnc.ac.jp/~shojima/ntt/index.htm
Ordinal scale based on NTT
Test scores
Latent
rank 5
Ability to do X5
Ability to do X4
Latent
rank 2
Ability to do X3
Latent
rank 1
Ability to do X2
Ability to do X1
• It is easy to explain the relationship
between scores and abilities because
the individual abilities also change in
stages.
• This increases the test’s accountability,
and helps to draw a can-do chart.
Based on http://antlers.rd.dnc.ac.jp/~shojima/ntt/index.htm
Purpose
 To explore a possibility of using NTT in SLA research (Note 2)
To divide learners into different latent ranks and to
attempt to draw a can- do chart
Method
1) Subjects --- university-level 369 Japanese EFL learners
2) Materials --- a grammaticality judgment task on unaccusative /
unergative verbs (Note 3) (36 items: see below) (Note 4)
3) Procedures --- judgments made on a 5-point scale (1-5) converted
into 1-4 points
if ungrammatical
0 point <--1 point <--2 points <--3 points <--4 points <---
if grammatical
5 (definitely possible)
--->
4 points
4 (probably possible)
--->
3 points
3 (not sure)
--->
2 points
2 (probably impossible) --->
1 point
1 (definitely impossible) --->
0 point
Six
categories
Unaccusative
verbs (A-F) in the judgment task
Unaccusative verbs (appear, arrive, die, exist, fall, happen)
Category A (NP+V: 6 items)
e.g. Your letter arrived yesterday.
Category C (*NP+be+p.p.: 6 items)
e.g. *Because of the rain, the train was arrived late.
Category E (*NP+V+NP: 6 items)
e.g. *Finally the waitress arrived the salad to us.
Unergative verbs (cry, dance, laugh, play, sing, work)
Category B (NP+V: 6 items)
e.g. Her father cried at her wedding ceremony.
Category D (*NP+be+p.p.: 6 items)
e.g. *He was cried when he heard of his mother’s death.
Category F (*NP+V+NP: 6 items)
e.g. *The boy hit his little sister and cried her.
Conclusion
 The learners were best distinguished
on an ordinal scale based on NTT
Rank 1
when they were ranked into four levels.
Rank 4
Rank 3
Rank 2
 Characteristics of each latent rank were described with
regard to their acquisition of unaccusative / unergative
Rank 4: Ability to correctly judge Category D
verbs.
Rank 3: Ability to correctly judge Categories E and F
Rank 2: Ability to correctly judge Category A
Rank 1: Ability to correctly judge Category B
 Some learners in higher ranks seem to regress on an
acquisition path when they judge incorrect passive
sentences in Categories C and D.
??
Rank 4
Rank 3
C) *…,the accident was happened.
Grammatical ?
D) *They were laughed when …
(Note 1) More about NTT
• Developed by Dr. Kojiro SHOJIMA and his colleagues
• NTT is a nonparametric test theory based on
the self-organizing map (SOM) and generative
topographic map (GTM).
• First presented at the International Meeting of the
Psychometric Society in 2007 (IMPS2007) in Tokyo
• The statistical software：
(Shojima, 2008)
available at http://www.rd.dnc.ac.jp/~shojima/exmk/index.htm
• Selected publications on NTT:
1) About NTT: http://www.rd.dnc.ac.jp/~shojima/ntt/index.htm
2) Shojima, K. (2009). Neural test theory. In K. Shigemasu et al. (Eds.), New Trends in Psychometrics (pp.407416). Universal Academy Press.
3) Shojima, K. (2008). Neural test theory: A latent rank theory for analyzing test data. DNC Research Note, 08-01.
(Note 2) Summary of the background of our research
The present joint research has been continuing for eight years with the purpose of
exploring the overall development of grammatical competence of Japanese EFL
learners. We developed several grammaticality judgment tasks on unaccusative/
unergative verbs, dative alternation, psych verbs, relative clauses, wh-questions,
articles, in addition to three sets of standardized test of the English grammar and a
test on cue dependency, all of which were conducted on university-level Japanese
learners of English. The research findings on each task/test have been published
elsewhere (Yamakawa et al., 2003; Ohba et al., 2006; Shimizu et al., 2006; Sugino et
al., 2006; Nakano et al., 2007; Ohba et al., 2008). Based on the data we obtained, we
employed IRT (Item Response Theory), and equated the data of 1,200 learners from
different tasks/tests (Yamakawa et al., 2008). We obtained the distribution of the
average values of all the grammatical categories on the b-parameter (difficulty
parameter) scale. To our knowledge, this was a first attempt to apply the IRT over a
range of grammatical features, and revealed many characteristics of the development
of their grammatical competence. In addition, we also attempted to compare the
results of grammaticality judgment tasks (unaccusative/unergative verbs and relative
clauses) conducted on Japanese and Korean EFL learners (Ohba et al., 2009;
Yamakawa et al., 2009). Although we have obtained substantial information on the
order of the difficulty of grammatical categories, we need to explore the relationship
between learners’ acquisition stages and their mastery of grammatical categories,
which is the focus of the present study.
Unaccusat
(Note 3) Unaccusative verbs and unergative verbs
The Unaccusative Hypothesis (Perlmutter, 1978, Burzio, 1986)
1) Unacusative verbs ( e.g., happen; break)
The accident happened 15 years ago.
[ empty [ VP happen the accident ] ] The accident (THEME)
2) Unergative verbs (e.g., dance; cry )
Mary danced.
[ Mary [ VP danced] ] Thematic role: Mary (AGENT)
L2 literature: overpassivization errors of unaccusatives
*Most of people are fallen in love and marry with someone. (Japanese L1)
(Zobl, 1989: 204)
Two Major Accounts of Passivized Unaccusative Verbs
1) NP movement account: Analogy to the passive
*The accident was happened 15 years ago. [ empty [ VP happen the accident ] ]
2)
Lexical causativization account: Creation of a temporary causer
*The driver happened the accident 15 years ago.
(Note 5) The order of difficulty of the categories
 Our previous studies utilizing ANOVA and IRT also found that the order of
difficulty of the categories were B<A<F≦E<D<C. This means that, if used in the
same construction, learners always find unaccusatives more difficult than
unergatives, and that they start to be able to judge grammatical sentences
(NP+V) as “grammatical” first, and then judge ungrammatical ones (NP+V+NP)
as “ungrammatical”. Unaccusatives, if used in NP+be+p.p., are the most difficult
for them. All the learners in Ranks 1-4 judged in the same order.
(Note 6) U-shaped development:
Unaccusative Trap Hypothesis (Oshita, 2001)
 Some learners in higher ranks may seem to regress on an acquisition path,
exhibiting a U-shaped curve. Oshita (2001) argues that a U-shaped
development can be observed in the acquisition of unaccusative verbs,
especially when learners progress from beginners’ level to intermediate level.
He claims that this phenomenon is caused by learners’ reanalysis of
unaccusative verbs analogous to the passive formation rule.
References
Nakano, M., Sugino, N., Yamakawa, K., Ohba, H., & Shimizu, Y. (2007). A Study of grammar development among Japanese university
students: Intransitive verbs, transitive verbs, ditransitive verbs and logical subjects in Xcomps - Part (1). Proceedings of the 12th
Conference of Pan-Pacific Association of Applied Linguistics (pp. 264-267).
Ohba, H., Sugino, N., Nakano, M., Yamakawa, K., Shimizu, Y., & Kimura, S. (2006). The Development of Grammatical Competence of
Japanese EFL Learners: Focusing on Relative Clause Constructions. Journal of Pan-Pacific Association of Applied Linguistics, 10,
73–87.
Ohba, H., Sugino, N., Yamakawa, K., Shimizu, Y., & Nakano, M. (2008). Exploring the acquisition of English articles by Japanese EFL
learners using on-line tasks. Paper presented at The Third CLS International Conference (National University of Singapore,
Singapore).
Ohba, H., Yamakawa, K., Sugino, N., Shimizu, Y., & Nakano, M. (2009). The acquisition of restrictive relative clauses by Japanese and
Korean learners of English. Proceedings of the 14th Conference of Pan-Pacific Association of Applied Linguistics (pp. 467-470).
Shimizu, Y., Yamakawa, K., Sugino, N., Ohba, H., & Nakano, M. (2006). Developing a sample-free grammatical proficiency test for SLA
research. Proceedings of the 10th Conference of Pan-Pacific Association of Applied Linguistics (pp. 227–235).
Sugino, N., Yamakawa, K., Ohba, H., Nakano, M., & Shimizu, Y. (2006). The acquisition of psych verbs by Japanese EFL learners: From
the perspective of cue dependency. Paper presented at the 11th Conference of Pan-Pacific Association of Applied Linguistics
(Kangwon University, Korea).
Yamakawa, K., Sugino, N., Kimura, S., Nakano, M., Ohba, H., & Shimizu, Y. (2003). The development of grammatical competence of
Japanese EFL learners: Focusing on unaccusative/unergative verb. Annual Re-view of English Language Education in Japan, 14, 1–
10.
Yamakawa, K., Sugino, N., Ohba, H., Nakano, M., & Shimizu, Y. (2008). Acquisition of English grammatical features by adult Japanese
EFL learners: The application of Item Response Theory in SLA research. Electronic Journal of Foreign Language Teaching (e-FLT),
Vol.5, No.1: pp.13-40.
Yamakawa, K., Sugino, N., Shimizu, Y., Nakano, M., & Ohba, H. (2009). The acquisition of unaccusative verbs by Japanese and Korean
learners of English. Proceedings of the 14th Conference of Pan-Pacific Association of Applied Linguistics (pp. 9-14).
Acknowledgments
*The present study is supported by Grants-in-Aid for Scientific Research 1348064 (2001-2003), 16320078 (2004-2006) and
19320091 (2007-2009) from the Japan Society for Promotion of Science.
Acknowledgments
*The present study is supported by Grants-in-Aid for Scientific Research 1348064 (2001-2003),
16320078 (2004-2006) and 19320091 (2007-2009) from the Japan Society for Promotion of Science.