Cross-modal language activation.
Does processing a sign (L1) also activate its corresponding
written translation (L2)?
(English)
The question how mental lexicon entries of a first (L1) and second language (L2) are crosslinked is still of major interest in psycholinguistic research, especially since sign languages
entered the field. Thierry & Wu (2007) show for Chinese-English bilinguals that their first
language (Chinese) is also activated when participants read sentences in their second
language English. Morford et al. (2011) show the same connection between L1 activation
(American Sign Language) during L2 processing (English) for Deaf signers.
We extended this question and asked whether a persons L2 is also activated by its L1, even
across modalities and during sentence processing. Does a visual-manual representation of a
sign (for example of MOUSE) activate the orthographic representation of the written word
“mouse”?
On this poster, we will present the design and the methodological procedure of an event
related potential (ERP) study, the underlying assumptions regarding mental representations
of lexicon entries, and the consequences of the highly interesting results of our study for the
modeling of the mental lexicon.
In our study, we investigated whether or not deaf native signers of German Sign Language
(DGS) activate their second language (written German) when processing sentences in DGS.
In contrast to Morford et al. (2011), participants watched unmodified real time videos with
signed DGS sentences, while their EEG signal (electroencephalogram) was recorded. We
used a priming design with two kinds of priming conditions to investigate the processing
costs of a target sign: first, an intermodal priming, where prime and target sign where
minimal pairs in DGS (ROOM – FAMILY) as in (1a); and second, a cross-modal priming, where
prime and target signs where minimal pairs in their German translation (HOUSE ‘Haus’ –
MOUSE ‘Maus’) as in (2a). Both conditions where compared to the corresponding unprimed
sentences like in (1b) and (2b). Stimuli signs (both prime and target) were produced without
mouthing to avoid a cross activation originated in “spoken” articulation.
(1) Intermodal priming (minimal pairs in DGS):
Primed sentence:
a. NEXT WEEK ROOM THERE FAMILY PARTY ORGANIZE
‘Next week, a family will organize a party in the room.’
Unprimed sentence:
b. NEXT WEEK RESTAURANT THERE FAMILY PARTY ORGANIZE
‘Next week, a family will organize a party in the restaurant.’
(2) Cross-modal priming (minimal pairs in written German):
Primed sentence:
a. LAST WEEK MY HOUSE THERE MOUSE HIDE
‘Last week, a mouse hid in my house.’
Unprimed sentence:
b. LAST WEEK MY KITCHEN THERE MOUSE HIDE
‘Last week, a mouse hid in my kitchen.’
The so far existing analysis of the EEG data shows that a priming effect (less processing
costs for the target sign if it is a minimal pair of the prime) is not only found for DGS minimal
pairs (condition 1) but also – and this is new – for the corresponding orthographic minimal
pairs in German (condition 2). The results support the conclusion of L2 activation by L1
processing across modalities.
Pictures of prime and target sign of example (1) and (2):
(1)
related prime: ROOM
target: FAMILY
unrelated prime: RESTAURANT
(2)
related prime: HOUSE
target: MOUSE
unrelated prime: KITCHEN
References:
Morford, J.P., E. Wilkinson, A. Villwock, P. Piñar & J.F. Kroll (2011). When deaf signers read English:
Do written words activate their sign translations? Cognition 118: 286-292.
Thierry, G., & Y.J. Wu (2007). Brain potentials reveal unconscious translation during foreign language
comprehension. Proceedings of the National Academy of Sciences 104(30): 12530-12535.