Neuroscience and Multilingualism
CEFR Proficiency Testing in the Context of
Cognitive Neurolinguistics
Dr. Edna Andrews
Duke University
Defining human language
Where language is NOT equal to speech
and
There is NO language in the ONE
Human language is a …
 Dynamic, learned, hierarchical, relatively autonomous
system of meaning-generating paradigmatic and
syntagmatic signs that signify and communicate to self
and others via speech communities and communities of
practice throughout the life cycle.
Understanding variation
Language is not a monolith in the human brain.
 ...культура представляет собой коллективный интеллект и
коллективную память, т.е. надындивидуальный механизм хранения
и передачи некоторых сообщений (текстов) и выработки новых....
 Память культуры не только едина, но и внутренне разнообразна.
Это означает, что ее единство существует лишь на некотором
уровне и подразумевает наличие частных «диалектов памяти»,
соответствующих внутренней организации коллективов,
составляющих мир данной культуры.... (Память в
культурологическом освещении, Из. ст. I, с. 200, 1992)
4
Central controversies of the field of
Brain and Language
Learned/innate
(predispositions)
Degree of localization
(modularity and
connectivity)
Appropriate definitions
of critical periods
(specificity, plasticity,
complexity)
Central disciplines of the field of
Brain and Language
 Neuroanatomy

Ojemann, Dowling, Rose, Parnavelas, Huttenlocher
 Neurophysiology

Ojemann, Rose, Huttenlocher
 Biochemistry

Rose, Ojemann, Huttenlocher
 Neurofunctionality

Poeppel, Fabbro, Rosenfield
 Neurolinguistics

Lieberman, Poeppel, Fabbro, Andrews, Jakobson
 Neuroimaging

Huettel, Poeppel, Ojemann
Central topics of the field of
Brain and Language
 Linguistic theory

Jakobson, Bolinger, Lieberman, Poeppel, Vygotsky
 Lesion-deficit tradition

Fabbro, Rosenfield, Damasio,
Skotko/Andrews/Einstein, Sacks, Luria
 Cultural anthropology

Donald
 Language, culture & identity

Jakobson, Donald
 Language and Memory (&
collective memory)

Rose, Jakobson, Donald, Squire, Rosenfield
 Development neurobiology

Dowling, Rose, Parnavelas, Ojemann, Huttenlocher
 Evolutionary biology

Lieberman, Donald
 Imaging technology (EEG, ERP,
fMRI, PET, MEG, TMS, CSM)

Ojemann, Poeppel, Huettel
 Developmental cognitive
psychology (consciousness/
metaconsciousness)

Vygotsky
 Phonology

Jakobson, Bolinger, Rosenfield, Fabbro,
Lieberman
 Multilingualism

Jakobson, Fabbro, Ojemann, deBot, Hernandez,
Perani
Poeppel/Hickok 2004, “Towards a new functional
anatomy of language”
“1. Broca’s area and Wernicke’s area are no longer viewed as monolithic or homogeneous
pieces of tissue. Rather, there are attempts to define, subdivide, and functionally
interpret both of these cortical regions. It is particularly noteworthy that no paper in the
present collection focuses on or attributes any special role to Broca’s area or
Wernicke’s area.
2. The fractionation of STG and its functional role is a very active area of imaging
research, with a major proposal being that functionally and anatomically distinct
parallel dorsal and ventral pathways originate in the STG.
3. There is a dramatic increase in attention to cortical areas outside the traditional
perisylvian language zone. Some of these regions include the middle and interior
sectors of the temporal lobe for its role in word-level processes, the anterior STG for its
role in the construction of phrases as well as intelligibility, and subcortical structures
(basal ganglia, cerebellum) for their role in linguistic computation.
4. There is increasing interest in the relation between perception/comprehension and
production and the potential role of posterior temporal and inferior parietal cortex in
D. Poeppel, G. Hickok / Cognition 92 (2004) 1–12 9
the auditory–motor interface. There is a hypothesis that a Sylvian parieto-temporal
area (Spt) drives an auditory–motor interface, as well as proposals that areas 7 and 40
perform subroutines of verbal working memory.
5. The right hemisphere, the ugly step-hemisphere in brain-language models, is being
rehabilitated. There is broad consensus that, at least in speech perception, the right
temporal lobe plays an important role, and, more generally, one of the main
consequences of imaging research has been to highlight the extensive activation of the
right hemisphere in language tasks. On balance, a modification of the virulent left hemisphere
imperialism characteristic of the field is in order.”
*importance of ecological validity
Hickok & Poeppel Model of Brain & Language
(2007)
9
•Revelations of language function from
CSM results (Cortical stimulation mappings)
from Corina et al. 2010)
“Semantic paraphasias are errors in which the patient substitutes
a semantically related or associated word for the target word. Iconic
examples of semantic paraphasias include producing the word
‘horse’ for the target image cow, or producing the word ‘car’ when
the correct target is train.”
“Performance errors include form-based distortions that are
slurred, stuttered, or imprecisely articulated.”
(Corina, 2010)
from Corina et al. 2010)
from Corina et al. 2010)
“Phonological paraphasias are characterized by unintended phonemic
epenthesis, omission, substitution, metathesis, and repetition. In
production models of language, phonological paraphasias
are believed to involve phoneme selection. Unlike neologisms,
phonological paraphasias bear a resemblance to the intended target.
[…]
Neologisms (e.g., fish ? ‘herp’) are form-based errors which are
‘‘possible but nonexistent words” that generally follow the
phonotactics of the language”
(Corina 2010)
from Corina et al. 2010)
from Corina et al. 2010)
“Circumlocution errors are responses in which the subject talks
about or ‘around’ the target in lieu of naming it. The subject may
describe attributes of the target, describe its use (e.g., chair ? ‘sit
down’), or talk about the target in a roundabout manner (e.g., shoe
? ‘cover foot’).”
“No-response errors are cases in which stimulation leads to the
lack of naming response. We differentiate these no-response errors
from errors of speech arrest that occur due to stimulation of ventral
motor regions associated with speech articulation. In clinical
practice, these latter areas are determined by having the patient
perform an automatic counting task.”
(Corina 2010)
from Corina et al. 2010)
Subcortical areas identified as relevant to
neurological representations of human language
























1. Fabbro (1999)
Basal ganglia – caudate nucleus, putamen, globus pallidus
Thalamus - ventral anterior nucleus (VA), ventral lateral nucleus (VL),
Pulvinar (P) and dorsomedial nucleus (DM)
Substantia nigra
Cortico-striato-thalamo-cortical loop (inner putamen-pallidus pathway)
Cortico-striato-subthalamo-cortical loop (outer putamen-pallidus pathway)
2. Lieberman (2006)
Basal ganglia – caudate nucleus, putamen, globus pallidus
Cerebellum
Hippocampus
3. Poeppel/Hickok (2004), Hickok/Poeppel (2004) and Shalom/Poeppel (2008)
Anterior superior temporal lobe
Middle temporal gyrus
Basal ganglia
Many right-hemisphere homologues
Ventral stream – posterior middle temporal gyrus, superior temporal gyrus (bilaterally)
(from STS (superior temporal sulcus) to
pITL (posterior inferior temporal lobe)
Dorsal stream – posterior Sylvian fissure (area Spt – Sylvian parietal temporal) (toward the parietal lobe and on to frontal regions)
4. Duffau (2008: 927-934) and Menjot de Champfleur et al. (2013: 151-157) –
Subcortical white matter fiber tracts important in language processing:
[measured using diffusion tensor imaging (DTI), a non-invasive approach to study white matter, can only provide anatomical [not functional] information;
however, when pre-operative and post-operative DTI are combined with intraoperative subcortical mapping, the results are “reliable anatomo-functional
correlations” (Duffau 2008: 928).]
IFOF - inferior occipito-frontal fasciculus (also called inferior frontal occipital fasciculus)
AF – arcuate fasciculus
Lateral SLF – superior longitudinal fasciculus
MdLF – middle longitudinal fasciculus (connecting the angular gyrus (AG) and superior temporal gyrus (STG)
17
Multilingual aphasia recovery data
 Fabbro (1999: 115): “With the exclusion of cases of
individuals bilingual since infancy, the mother tongue
is often the most familiar and the most automatized
language preferred for …. By analyzing all clinical
cases of bilingual and polyglot aphasics published so
far, I have calculated that around 40% of patients
present parallel recovery of all languages, 32% present
a better recovery of the mother tongue, and the
remaining 28% present a better recovery of the
second language.”
18
Language and Culture: There is no language in the one
 The important difference between grammatical and lexical categories in
language and the meanings produced by these categories in human
language is a focal point of Jakobson’s analysis ([1967]1985: 110):
 Grammatically, languages do not differ in what they can and cannot convey.
Any language is able to convey everything. However, they differ in what a
language must convey. If I say in English (or correspondingly in Japanese)
that “I spent last evening with a neighbor”, you may ask whether my
companion was a male or a female, and I have the factual right to give you
the impolite reply, “It is none of your business.” But if we speak French or
German or Russian, I am obliged to avoid ambiguity and to say: voisin or
voisine; Nachbar or Nachbarin; sosed or sosedka. I am compelled to inform
you about the sex of my companion not by virtue of a higher frankness,
openness, and informativeness of the given languages, but only because of
a different distribution of the focal points imparting information in the
verbal codes of diverse languages.
20
Signification & Memory




Donald identifies the key to understanding human language as a collective phenomenon when he notes that:
“[t]he isolated brain does not come up with external symbols. Human brains collectively invent symbols in a
creative and dynamic process” (2004: 43). And symbols are invented, according to Donald, by means of
executive skills “that created a nervous system that invented representation out of necessity” (ibid.). It is the
human ability to collectively invent innovative and dynamic external symbols that the field of linguistics calls
signification. Without signification as the initial and primary ability that underlies human language and all of
human cognition, there can be no nonhereditary collective memory. Signification always requires the translation
from one system into another, and the process is potentially infinite and unbounded (Jakobson 1985: 206, Peirce
4.127). This fact will become especially relevant in understanding speech acts and the construction of linguistic
meaning.
Donald singles out autocueing, the uniquely human ability to voluntarily control memory recall, that
provides freedom from the hic et nunc. It would have been a prerequisite to the development of human language,
which requires volitional actions, including retrieval of linguistic forms and their modification (2004: 45). Also,
the many different living systems are able to communicate with other living beings within and beyond their
species and the environment, but signification and autocueing are the critical pieces for human language. With
these two primary abilities—signification/invention of creative and dynamic external symbols and voluntary
control of memory retrieval—the evolution of human language becomes possible.
Tomasello’s insights about linguistic reference as a “social action” is an important corollary to the phenomenon
of signification (1999: 97). At the point at which children begin participating in the signification process as
learners of linguistic symbols, they can tap into not only the richness of “preexisting” knowledge, but also
participate in the uniqueness of linguistic symbols and their inherent polysemic nature, where one can
cognitively embrace an event or object at multiple levels simultaneously (cf. “a rose, a flower, and a gift”) (1999:
107).
21
Signification and Communication in Action: Building Blocks for a Theory
of Language and Brain via Modeling Speech Acts (Jakobson, Lotman,
Searle and Tomasello)













Jakobsonian speech act model (all factors/functions doubled in Lotman’s rendering)
Factors
Context
Contact
Addresser
Addressee
Code
Message
Functions
Referential
Phatic
Emotive
Conative
Metalingual
Poetic
22
What is important to know about
speech acts?













1. All speech acts are multifaceted events embedded in the cultural context.
2. Meanings of speech acts are always negotiated (they are never given as a priori categories).
Meanings are negotiations, not a priori categories (cf. It’s cool—meaning (1) the weather is chilly, (2) there is no
problem, everything is fine, (3) something is interesting, neat, etc.). Given this fact, it may be more productive to
imagine that the neural image of a word is multisensory and obligatorily involves cross-modal effects (cf. MarslenWilson 2007, Watkins et al. 2003, Massaro and Cohen 1995, McGurk/Madonald 1976, Lieberman 2006). By suggesting
that word meanings are multisensory, I am implying an approach that is deeply informed by the cognitive linguistic
paradigm, but the result is more like that found in Mahon and Caramazza (2008) but with important synergies with
the “embodied” approach given in Gallese and Lakoff (2005).
Meaning is indeed the fundamental condition that any unit on any level must fulfill in order to obtain linguistic
status. We repeat: on any level. - Émile Benveniste (1971: 103)
3. It matters who is talking and who is listening.
4. Speakers and hearers are always members of multiple and changing speech communities and communities of
practice.
5. All speech acts require translation between and among the factors and functions that define them.
6. There is no such thing as a single speech act. They always occur as multiplicities.
7. All speech acts (and all language) are ambiguous and redundant to varying degrees.
8. The Jakobsonian model of speech acts gives the minimum number of factor and functions. There may be more in
a given instance, but there are never less than 6.
9. Speech acts may be the ecologically-valid minimal unit of human language.
10. Human language is not the product of a single brain but rather a product of multiple brains in sync with each
other and embedded in the cultural context.
11. Any language can say anything, but some languages make you say certain things.
23
Skotko, Andrews, Einstein 2005
24
H.M.’s Discourse
Skotko, Andrews, Einstein 2005, Andrews 2014: 90








When looking at H.M.’s usage of language from a more broadly based discourse perspective, it becomes clear (1)
where H.M. demonstrated higher competency than his peer group and (2) where he demonstrates deficits. If we
now implement Jakobson’s speech act model as discussed previously, we can characterize H.M.’s speech and writing
in terms of the six functions (metalingual, conative, poetic, emotive, referential, phatic) in the following manner:
1. H.M.’s metalingual function is highly developed and exceeds expectations for healthy subjects of his educational
background and age group;
2. H.M.’s conative responses, whether they be verbal answers to questions or subsequent actions responding to
requests, are robust;
3. H.M.’s use of the poetic function is developed, especially in punning and humorous turns of phrase, and merits
special attention;
4. H.M.’s emotive function is appropriate in terms of his affect, sense of humor, laughter, eye contact with
interlocuters, body language, gestures accompanying speech, but his desire to share verbally is more reactive than
initiatory;
5. H.M. makes limited reference to the extralinguistic context surrounding his discourse, but on occasion did make
direct reference to persons and things; thus, his referential function was operative;
6. The weakest area of discourse for H.M. in terms of the speech act model is his lack of use of the phatic function,
including his reluctance to initiate or continue conversation, to reinitiate a previously given topic of conversation, or
to interact with his interlocutor’s narrative if a question to him is not involved. (Note that there are exceptions to
this characterization, but they are quite infrequent.)
25
LfMRI SLAM
•Longitudinal fMRI Study of SecondLanguage Acquisition and Multilingualism
• Unique longitudinal fMRI study where
scan data is correlated with subject-based
proficiency testing data and behavioral data
using a multivariate analysis of covariance
(MANCOVA)
• experimental design structured to follow
the acquisition of Russian language in a
study that combines, for the first time,
BOLD fMRI data with empirical language
proficiency data (CEFR) and course-based
behavioral data
• Six central issues relevant to appropriate presentation of
data on multilingualism: (1) intensity of contact, (2)
motivation to learn the new language, (3) language
aptitude, (4) attitudes by learners toward the L1, L2, L3,
etc., (5) other languages learned by subjects prior to the
study, and (6) degree of literacy of subjects (summarized
from de Bot 2008:118-120)
Critical Periods – fiction and fact
 For neuroscientists, the notion of critical periods is much more complex and nuanced
than the primitive rendition we often see through the prism of linguistics. In fact,
any neuroscientist recognizes that different cortical areas may require different
definitions of sensitive or critical periods. For example, the visual cortex has a very
well-defined critical period as seen in studies of the cat eye and light deprivation, but
even in visual cortices, the environment can modify the critical period itself (Dowling
2004: 46-51). In contrast, there are other cortical areas that do not demonstrate any
clear beginning or end of what Dowling calls “periods of more susceptibility” (ibid.:
51).
 As Dowling states: “The general notion of critical periods in cortical development
has been questioned, because often there is neither a sharp start nor a sharp end of
such sensitive periods. Some investigators believe, rather, that cortical modifiability
is a continuum, with, at most, periods of more susceptibility….In addition, critical
periods can be modified by environment” (ibid.: 50-51). He explains, “It is clear that a
variety of mechanisms can alter synaptic strength and circuitry in the adult brain –
from simple synaptic excitation and inhibition, to strengthening or weakening of
synaptic strengths by neuromodulatory mechanisms, to neurons sprouting new
branches and forming new synapses by mechanisms such as LTP” (ibid.: 106).
30
Regions of interest in our fMRI SLAM were selected based on the following
principles: (1) regions that are frequently mentioned in the fMRI literature for
sentence comprehension (Price 2010: 68) or CSM literature (Corina et al. 2010:
107-111) and (2) regions that showed significant change across the language
acquisition scans.
MTG – medial temporal gyrus
STG – superior temporal gyrus
MFG – middle frontal gyrus
IFG – inferior frontal gyrus
PrG – precentral gyrus
PoG – postcentral gyrus
HeG - Heschl’s gyrus
SMG – supramarginal gyrus
AnG – angular gyrus
(adapted from Corina et al. 2010)
LfMRI SLAM at a glance
Subjects for LfMRI SLAM are 7 subjects (3 males and 4 females). The subjects
began the study at the ages of 19 and 20 years of age. Handedness was not a
criterion for elimination from the study, although all longitudinal participants
are right-handed. Andrews et al. 2013 covers scans conducted between April
2011 and April 2012.
All LfMRI SLAM subjects have participated in multiple testing sessions using
the official Russian language proficiency exam for the Russian Federation
Ministry of Education (TRKI). TRKI is an instrument that meets the
requirements of the language proficiency exams developed through the
Council of Europe, CEFR (Breiner-Sanders et al 2001, North 2000). They have
also been involved in course work and class examinations at both Duke
University and St. Petersburg State University (Russia).
Time table of fMRI scans and
proficiency testing.
Scan 1
Apr-Aug, 2011 (pre-program)
Scan 2
Dec, 2011
Scan 3
Mar-Apr, 2012
Scan 4
Oct, 2012
Scan 5
Mar, 2013
Scan 6
Mar-Apr, 2014
34
35
CEFR/TRKI Proficiency Testing
 Levels: A2, B1 (different versions)
 Types of tests: listening comprehension, grammar,
reading, writing, speaking
 Published study: Focus on first 3 scans (2011/2012)
36
Stimuli and Procedure
This experiment uses the CIGAL (Voyvodic, 1999; Voyvodic et al., 2011)
software package for auditory and visual stimulation as well as real-time
recording of subject responses, cardiac and respiratory physiological
oscillations, and eye-tracking behavior.
Subjects listen to no fewer than 24 ~30-second digitized auditory segments in
four languages during a 30 to 60 minute session. Subjects are asked to close
their eyes and not use any sub-vocalizations during the sessions. Directly
following the sessions, subjects participate in a debriefing session including a
list of questions (cf. number 9 in design section iii).
Imaging parameters
Imaging is performed using a General Electric LX 3T Signa scanner using head
restraints to reduce motion. All scans discussed herein were conducted on the
same BIAC3 scanner at Duke University Hospital. The subject's head is
positioned along the canthomeatal line. For functional scans, a total of 30
slices with a 5mm thickness are obtained using a TR of 1500ms, a TE of 35ms, a
64x64 matrix, and a spiral pulse sequence. Slices are axial, taken parallel to the
plane of the anterior and posterior commissures with the most inferior image
level with the top of the pons. High resolution dual echo proton density and
T2-weighted anatomical images are acquired in the same slice planes as the
functional scans. A high resolution 3D fast spoiled GRASS T1-weighted scan
covering the whole brain with isotropic 1mm cubic voxels is acquired to allow
structural visualization in any orientation (256x256x166 voxels).
In each run, the audio stimulus files are played in 30-second blocks followed by 10-second rest periods for four
minutes. Run 1 is English-Russian, Run 2 is Spanish-English, Run 3 is Russian-Spanish, and Run 4 is EnglishGeorgian. Languages alternate within each run.
There are samples in four different languages, three of which will be known (to varying degrees) to the subject
while the fourth will be entirely unknown and generally unidentifiable to the subject. The audio stimulus
consists of short narratives of no less than 50 sentences in four languages.
The voices for each language will be different (including male and female voices at an indeterminable age but
with native pronunciation.
In the longitudinal study, subjects hear the same protocol across scans. Some recognition of voice is probably
by the fourth longitudinal scan.
The ordering of auditory segments will vary, with 3 segments of two languages alternating in each segment
with a total of 6 segments.
Participants will not be excluded based on handedness.
This section of the functional scan will involve only auditory comprehension. Reading comprehension, which
was recently added to the protocol, is given at the end of the scan.
Each 30-second audio segment consists of unique utterances — no repetitions of content between or among
languages. There is a 10 second rest period following each audio segment.
There is a post-scan interview with a free-response questionnaire.
Subject 1, Russ-rest
Subject 1, Russ-rest, visit 2
Subject 1, Russ-rest, visit 3
Sample Data
Subject 1 – Eng-Rest
Scan 1
Sample Data
Subject 1 – Eng-Rest
Scan 2
Sample Data
Subject 1 – Eng-Rest
Scan 3
Sample Data
Subject 1 – Russ-Rest
Scan 1
Sample Data
Subject 1 – Russ-Rest
Scan 2
Sample Data
Subject 1 – Russ-Rest
Scan 3
AMPLE Normalization
Voyvodic, J. T. (2006). Magnetic Resonance Imaging, 24(9), 1249-1261.
same subject doing the same English sentence comprehension
task under different scanning conditions over a 6-year period
MANCOVA analysis of LfMRI SLAM study (2013)

The multivariate analysis of covariance used in our analysis is based on proficiency testing measurements and fMRI
ROI measurements (12 regions of interest) generated under two pairs of conditions: (1) English-rest and (2) Russianrest. The raw data are the percentage of voxels in each region whose activity levels are above threshold. Additionally,
to provide an internal benchmark, we also used fMRI readings from the Middle Occipital Gyrus, a region which would
not be relevant in a listening comprehension condition.

The primary model used in this analysis was MANCOVA, where the vector of responses (the percents of non-zero
voxels by time, region, hemisphere, and subject) is modeled as the additive effects of two categorical variables
(hemisphere and region) and one continuous covariate (proficiency score):
Y = mean + region effect + hemisphere effect + score effect



The analysis treated the two pairs of conditions (English-rest, Russian-rest) separately. Although all four conditions
could be handled simultaneously in one MANCOVA model with more complicated repeated measures structure, we
analyzed the two conditions by fitting the same form of the MANCOVA model each time. This simplifies the
interpretation of the results, and reduces the risk of misleading results due to failure of model assumptions (such as
non-normal residuals) or some aberrant situation in one of the test conditions.
In reporting the results from these two MANCOVA models, we use Pillai's trace to test all hypotheses. Alternative
statistics, including Wilks’ lambda, Hotelling’s trace and Roy’s largest root, found nearly identical results. We also note
that tests of sphericity or specific pattern matrices for the correlation structure were generally rejected, implying that
the covariance matrix is complex.
50
Primary Results of the MANCOVA Analysis
 The primary results included the following:
 1. The score effect for the repeated measure is significant for the
condition Russian-rest. Pillai’s trace has p-values where p = 0.01.
 As expected, the score effect is not significant for English-rest,
where the p-value equals 0.47. This supports the research
hypothesis that language acquisition is associated with
 characteristic activations found in the Russian conditions.
Furthermore, the fact that it was insignificant for the English-rest
condition strongly supports the belief that non-normal residuals are
not distorting the analysis in any important way.
 2. The time effect is significant; activation levels change across the
three different sets of measurement.
51
Secondary Results of the MANCOVA Analysis
 Secondary results included the following:
 1.
The time effect is significant for Russian-rest; average
activation levels change across the different sets of
measurements.
 2.
The time effect is not significant for English-rest; average
activation levels do not change across the different sets.
 3.
There is a significant hemisphere effect.
 4.
The Middle Occipital Gyrus, used as an internal statistical
standard, shows a lack of effect
as expected.
 5.
Different regions show variation in activation patterns.
52
Preliminary Statistical Data on Reading Scans

In parallel with results obtained from the auditory scans under the Russianrest and English-rest conditions, a MANCOVA analysis was run based on five regions of
interest (IFG, STG, MTG, Lingual Gyrus and Precuneus) across three time points for
Russian-rest and English-rest during a reading task using proficiency results for each
subject as before. The score effect for the repeated measure is significant for the
condition Russian-rest. Pillai’s trace has p-value 0.017. Here again, as expected, the
score effect is not significant for English-rest, where the p-value equals 0.22. This
supports the research hypothesis that language acquisition is associated with
characteristic activations found in the Russian conditions for reading as well as
auditory comprehension.
 In addition to the MANCOVA analysis, we added a regression analysis to determine if
faster reading speeds by subjects in the English condition would correlate with their
proficiency achievements in the acquisition of Russian. In fact, as anticipated, the
regression finds a very significant and positive relationship between these two factors
(p = 0.027, R=0.804 [where R is the correlation between the speed of reading and the
proficiency scores], explaining 65% of the variation
53
Profile summary for longitudinal subjects (prior to April, 2011)
SUBJECT
L1/proficiency
L2/proficiency
Subject 1:
Eng/C2
Fr/C1, AP 5, SAT II-800, It/B1
Subject 2:
Eng/C2
Ger/B2/C1, AP 5, IB HL-6,
Arab/B1
Subject 3:
Eng/C2
Fr/B1, Rus/B1
Subject 4:
Eng/C2
Latin/A2, Heb/A1
Subject 5:
Eng/C2
Fr/A1, It/A1
Subject 6:
Eng/C2
Sp/A2
Subject 7:
Eng/C2
Sp/SAT II, AP 5
Conclusions and Future Directions




Conclusions and Future Directions
In our study, we set out to collect a robust set of data acquired longitudinally using both fMRI and behavioral and
proficiency data on subjects who begin their intensive study of a second language during the study. By coordinating
proficiency testing and fMRI scanning, we could analyze the degree to which fMRI can track language acquisition
within subjects. From the behavioral and proficiency data we could derive empirically valid information about the
achievements of the subjects in a range of measurements that are available by task (listening comprehension,
reading, grammar/lexicon) as a component of the analysis of the fMRI scan data for a listening comprehension task.
Using a multivariate analysis of covariance allows us to determine if there is a significant relationship between the
changes in activations in the ROIs for each subject across the 3 scans/time points by comparing those activation
changes to changes in proficiency for each subject. The result produced a p value = 0.01, which supports the
fundamental research hypothesis that language acquisition is associated with characteristic activations found in the
Russian conditions. Furthermore, the lack of significance for the English-rest condition (where p = 0.47) strongly
supports the belief that non-normal residuals are not distorting the analysis in any important way. Finally, time
effect is significant for the Russian conditions and shows activation levels changing across the three sets of
longitudinal measurements.
The importance of understanding invariance in variation has been one of the central concerns of theoretical
linguistics of the twentieth and twenty-first centuries. The construction and conducting of imaging studies that
include protocols of language that are not only ecologically valid and coupled with behavioral and proficiency data,
but also allow for multiple comparisons across and within subjects longitudinally, may provide a new perspective on
how to answer some of the most challenging issues about brain and language, as well as formulate new questions
that can deepen the research paradigms in cognitive and neurolinguistics.
55
Contribution of longitudinal fMRI analyses of
language acquisition with general linear models
 While our study is only one contribution to the
longitudinal study of multilingualism and second
language acquisition, it serves as one more important
component leading to a prescient model that more
deeply integrates behavioral information, empirical
testing and proficiency data with computational data
provided from fMRI studies embedded in appropriate
statistical analysis.
56
Reassembling the Pieces: Languages
and Brains
 1. The importance of culture in the evolution of human cognition and
language
 2. Key to understanding human language as a collective phenomenon:
“the isolated brain does not come up with external symbols. Human
brains collectively invent symbols in a creative & dynamic process.”
(Donald 2004: 43)
 3. Reading as a game changer (cf. Bolinger’s “visual morphemes”)
 4. Multilingualism through the life cycle: change as essential, not
essentialist
 5. Reuniting lesion-deficit studies with healthy subject research
 6. Utility of imaging research in the field of cognitive neurolinguistics
57
Abrams, A. (1973). Minimal auditory cues for distinguishing Black from White talkers.
(Unpublished doctoral dissertation), City University New York, New York.
Bibliography
Abutalebi, J. (2008). Neural aspects of second language representation and language control. Acta Psychologica, 128(3), 466-478.
Abutalebi, J., & Green, D. (May 01, 2007). Bilingual language production: The neurocognition of language representation and control. Journal of
Neurolinguistics, 20(3), 242-275.
Abutalebi, J., Tettamanti, M., & Perani, D. (May 01, 2009). The bilingual brain: Linguistic and non-linguistic skills. Brain and Language, 109, 51-54.
Aggleton, J. P. (1992). The Amygdala: Neurobiological aspects of emotion, memory, and mental dysfunction. New York: Wiley-Liss.
Ahlsén, E. (2006). Introduction to neurolinguistics. Amsterdam: John Benjamins.
Akhutina, T., Kurgansk, A., Kurganskaya, M., Polinsky, M., Polonskaya, N., Larina, O., Bates, E., ... Appelbaum, M. (January 01, 2001). Processing
of grammatical gender in normal and aphasic speakers of Russian. Cortex, 37(3), 295-326.
Amunts, K., Schleicher, A., Bürgel, U., Mohlberg, H., Uylings, H. B., & Zilles, K. (January 01, 1999). Broca's region revisited: cytoarchitecture and
intersubject variability. The Journal of Comparative Neurology, 412(2), 319-341.
Andrews, E. (2014). Neuroscience and Multilingualism. Cambridge: Cambridge University Press.
Andrews, E., L. Frigau, C. Voyvodic-Casabo, J. Voyvodic, J. Wright. (2013). “Multilingualism and fMRI: Longitudinal Study of Second Language
Acquisition.” (2013). Brain Sciences, 3(2), 849-876.
Andrews, E. (1990). Markedness theory: The union of asymmetry and semiosis in language. Durham: Duke University Press.
Andrews, E. (1994). The interface of iconicity and interpretants. In M. Shapiro & M. Haley (Eds.), Peirce Seminar Papers, II (pp. 9–28).
Providence, RI: Berghahn Books
Andrews, E. (1996a). The semantics of suffixation. München: Lincom Europa.
Andrews, E. (1996b). Gender and declension shifts in contemporary standard Russian:
Markedness as a semiotic principle. In E. Andrews & Y. Tobin (Eds.), Towards a calculus of meaning: Studies in markedness, distinctive features,
and deixis (pp. 109–140). Amsterdam: John Benjamins.
Andrews, E. (2003). Conversations with Lotman: Cultural Semiotics in Language, Literature, and Cognition. Toronto: University of Toronto
Press. (Translated into Japanese in 2006, reprinted as an electronic book for University of Toronto Press in 2007.)
Andrews, E. (2011). Language and brain: Recasting meaning in the definition of human language. Semiotica, 184, 11-32
Andrews, E. (2014). Neuroscience and Multilingualism. Cambridge: Cambridge University Press.
Ben, S. D., & Poeppel, D. (October 02, 2007). Functional Anatomic Models of Language: Assembling the Pieces. The Neuroscientist, 14(1), 119127.
Benveniste, E., & Meek, M. E. (1971). Problems in general linguistics. Coral Gables, Fla: University of Miami Press.
Berman, R. A., Colby, C. L., Genovese, C. R., Voyvodic, J. T., Luna, B., Thulborn, K. R., & Sweeney, J. A. (January 01, 1999). Cortical networks
subserving pursuit and saccadic eye movements in humans: an FMRI study. Human Brain Mapping, 8(4), 209-225.
Berndt, R. S., & Caramazza, A. (August 01, 1980). A Redefinition of the Syndrome of Broca's Aphasia: Implications for a Neurological Model of
Language. Applied Psycholinguistics, 1(3), 225-278.
Berndt, R. S., Caramazza, A., & Zuril, E. (1983). Language functions: Syntax and semantics. In S.J. Segalowitz (Ed.), Language Functions and
Brain Organization, Academic Press, New York.
Bhatia, T. K. (2006). The handbook of bilingualism. Malden, MA: Blackwell Pub.
Bialystok, E. (January 01, 2010). Global-Local and Trail-Making Tasks by Monolingual and Bilingual Children: Beyond Inhibition. Developmental
Psychology, 46(1), 93-105.
Bialystok, E. (July 01, 2010). Bilingualism. Wiley Interdisciplinary Reviews: Cognitive Science, 1(4), 559-572.
Bialystok, E. (January 01, 2011a). Reshaping the mind: the benefits of bilingualism. Canadian Journal of Experimental Psychology = Revue
Canadienne De Psychologie Expérimentale, 65(4), 229-235.
Bialystok, E. (January 01, 2011b). Coordination of executive functions in monolingual and bilingual children. Journal of Experimental Child
Psychology, 110(3), 461-468.
Bialystok, E. (January 01, 2011c). How does experience change cognition? Evaluating the evidence. British Journal of Psychology, 102(3), 303-305.
Bialystok, E., Barac, R., Blaye, A., & Poulin-Dubois, D. (October 01, 2010). Word mapping and executive functioning in young monolingual and
bilingual children. Journal of Cognition and Development, 11(4), 485-508.
Diaz, M.T. & G. McCarthy. 2009. A comparison of brain activity evoked by single content and
function words: An fMRI investigation of implicit word processing. Brain Res. 1282: 38–49.
Friederici, A. D. (1998a). Language comprehension: A biological perspective. Berlin: Springer.
Friederici, A. D. (1998b). The neurobiology of language comprehension. In A. D. Friederici (Ed.), Language Comprehension: A Biological
Perspective (pp. 263-301). Berlin-Heidelberg-New York
Friederici, A. D. (June 01, 1999). Language related brain potentials in patients with cortical and subcortical left hemisphere lesions. Brain, 122(6),
1033-1047.
Friederici, A. D., Hahne, A. & Mecklinger, A. (1996). The temporal structure of syntactic parsing: Early and late effects elicited by syntactic
anomalies. Journal of Experimental Psychology: Learning, Memory and Cognition, 5, 1-31.
Friederici, A. D., Meyer, M. & von Cramon, D.Y. (in press). Auditory language processing: Brain images evoked by syntax, semantics and
phonology. Journal of Cognitive Neuroscience
Friederici, A. D., Pfeifer, E., & Hahne, A. (October 01, 1993). Event-related brain potentials during natural speech processing: effects of
semantic, morphological and syntactic violations. Cognitive Brain Research, 1(3), 183-192.
Galaburda, A., & Sanides, F. (January 01, 1980). Cytoarchitectonic organization of the human auditory cortex. The Journal of Comparative
Neurology, 190(3), 597-610.
Kandel, E. R., & Schwartz, J. H. (1991). Principles of neural science (3rd ed.). Norwalk: Appleton and Lange.
Kapur, S., Rose, R., Liddle, P. F., Zipursky, R. B., Brown, G. M., Stuss, D., Houle, S., ... Tulving, E. (January 01, 1994). The role of the left prefrontal
cortex in verbal processing: semantic processing or willed action?. Neuroreport, 5(16), 2193-2196.
Karbe, H., Wurker, M., Herholz, K., Ghaemi, M., Pietrzyk, U., Kessler, J., & Heiss, W. D. (September 01, 1995). Planum Temporale and
Brodmann's Area 22: Magnetic Resonance Imaging and High-Resolution Positron Emission Tomography Demonstrate Functional Left-Right
Asymmetry. Archives of Neurology, 52(9), 869-874.
Keller, T. A., Carpenter, P. A., & Just, M. A. (January 01, 2001). The neural bases of sentence comprehension: a fMRI examination of syntactic
and lexical processing. Cerebral Cortex, 11(3), 223-37.
Lucas, T. H., McKhann, G. M., & Ojemann, G. A. (September 01, 2004). Functional separation of languages in the bilingual brain: a comparison
of electrical stimulation language mapping in 25 bilingual patients and 117 monolingual control patients. Journal of Neurosurgery, 101(3), 449457.
Luk, G., Anderson, J. A. E., Craik, F. I. M., Grady, C., & Bialystok, E. (December 01, 2010). Distinct neural correlates for two types of inhibition in
bilinguals: Response inhibition versus interference suppression. Brain and Cognition, 74(3), 347-357.
Luk, G., Bialystok, E., Craik, F. I. M., & Grady, C. L. (November 16, 2011). Lifelong Bilingualism Maintains White Matter Integrity in Older Adults.
Journal of Neuroscience, 31(46), 16808-16813.
Luk, G., de, S. E., & Bialystok, E. (October 04, 2011). Is there a relation between onset age of bilingualism and enhancement of cognitive
control?. Bilingualism: Language and Cognition, 14(4), 588-595.
Osterhout, L. & Holcomb, P. J. (1995). Event-related potentials and language comprehension. In M.D. Rugg & M.G.H. Coles (Eds.),
Electrophysiology of Mind. Oxford (pp. 171-209). New York: Oxford University Press.
Osterhout, L., MacLaughlin, J., Pitkänen, I., French-Mestre, C., & Molinaro, N. (2006). Novice learners, longitudinal designs and event-related
potentials: a means for exploring the neurocognition of second language processing. In M. Gullberg & P. Indefrey (Eds.), The cognitive
neuroscience of second language acquisition (pp. 199-230). Blackwell: Malden, MA.
Page, M. P. A. (January 01, 2006). What Can't Functional Neuroimaging Tell the Cognitive Psychologist?. Cortex, 42(3), 428-443.
Palmer, G. B. (1996). Toward a theory of cultural linguistics. Austin: University of Texas Press.
Paradis, M. (January 01, 2003). The bilingual Loch Ness Monster raises its non-asymmetric head again-or, why bother with such cumbersome
notions as validity and reliability? Comments on Evans et al. (2000). Brain and Language, 87(3), 441-8.
Rosenblum, L. D., Yakel, D. A., Baseer, N., Panchal, A., Nodarse, B. C., & Niehus, R. P. (February 01, 2002). Visual speech information for face
recognition. Perception & Psychophysics, 64(2), 220-229.
Rosenfield, I. (1988). The invention of memory: A new view of the brain. New York: Basic Books.
Rugg, M. D., & Coles, M. G. H. (1995). Electrophysiology of mind: Event-related brain potentials and cognition. Oxford: Oxford University Press.
Sachs, J. S. (1967). Recognition memory for syntactic and semantic aspects of connected
discourse. Perception & Psychophysics, 2, 437–442.
Sachs, J. S. (January 01, 1974). Memory in reading and listening to discourse. Memory & Cognition, 2(1), 95-100.
Skotko, B. G., Kensinger, E. A., Locascio, J. J., Einstein, G., Rubin, D. C., Tupler, L. A., Krendl, A., ... Corkin, S. (January 01, 2004). Puzzling
Thoughts for H. M.: Can New Semantic Information Be Anchored to Old Semantic Memories?. Neuropsychology, 18(4), 756-769.
Skotko, B. G., Andrews, E., & Einstein, G. (September 01, 2005). Language and the Medial Temporal Lobe: Evidence from H. M.'s Spontaneous
Discourse. Journal of Memory and Language, 53(3), 397-415.
Verkuyten, M., De Jong, W., & Masson, C. N. (1994). Similarities in anti-racist and racist discourse: Dutch local residents talking about ethnic
minorities. New Community, 20, 253-268.
Voyvodic, J. T. (January 01, 1999). Real-Time fMRI Paradigm Control, Physiology, and Behavior Combined with Near Real-Time Statistical
Analysis. Neuroimage, 10, 2, 91-106.
Voyvodic, J. T. (November 01, 2006). Activation mapping as a percentage of local excitation: fMRI stability within scans, between scans and
across field strengths. Magnetic Resonance Imaging, 24(9), 1249-1261.
Voyvodic, J. T., Petrella, J. R., & Friedman, A. H. (January 01, 2009). fMRI activation mapping as a percentage of local excitation: Consistent
presurgical motor maps without threshold adjustment. Magnetic Resonance Imaging, 29(4), 751-759.