Eliciation Techniques and Data Collection
Sonja Eisenbeiss
seisen@essex.ac.uk
February 2013
Criteria for Method Selection: Reliability and Validity
•
Reliability: The consistency of a measurement/test
•
Test-re-test reliability
•
Inter-rater reliability
 Strict control of variables, test situation, etc.
•
Validity: the degree to which the test measures what it intends to measure (e.g.
children’s knowledge of genitives or possessives vs. their ability to focus)
•
Ecological validity: capturing the use of an ability as closely to real-life
use as possible
 Natural test situation with minimal extra demands
Types of Methods
•
spontaneous speech / naturalistic data
•
"classic" elicitation
•
stimulus-based elicitation
•
experiments
NOTE: Depending on the sub-field of linguistics (fieldwork, language acquisition
research in different contexts,…) , the term “elicitation” is used in VERY different ways.
The common denominator is: some effort has been made to get people to talk (in a
particular way or about a particular topic).
Data can be collected cross-sectionally, longitudinally, or at fixed intervalls
1
Spontaneous Speech
•
other terms:
•
naturalistic data
•
observed communicative events
•
External interference is limited to the fact that the ongoing communicative event
(discussion, play situation,…) is being observed and/or recorded.
•
This may be known or unknown to the participants
•
PRO
•
•
most “natural” data type
•
analysis not limited to target phenomena
•
no age limitations
CONTRA
•
lack of data for low-frequency phenomena
•
lack of negative evidence (i.e. based on such data one does not know which
structures are ungrammatical for the speaker)
•
hardly any control about form and content
"Classic" Elicitation
•
Participants who have some general awareness of the researchers aims are asked to
provide information about their language
•
Types:
•
•
Translation from a lingua franca into the target language
•
Back-translating utterances provided by other speakers
•
Manipulating utterances (e.g. turning them into questions, negating them)
•
Grammaticality/acceptability judgements
•
very controlled data type
•
provides data for low frequency phenomena
•
can provide negative evidence (grammaticality judgments) and meta-linguistic
information
CONTRA
•
less “natural” than observed or staged communicative events or elicitation games
•
analysis often limited to target phenomenon (dependent on type of elicitation)
•
age limitations
2
•
often limited to one or a few speaker and no/few controls for inter-speaker
variation and cross-situational variation (vs. experiments)
Stimulus-Based Elicitation (Games)
•
creation of contexts for low-frequency phenomena using stimuli (pictures, real life
objects, videos, etc.), often in a game-like setting
•
e.g. eliciting complete sentences with to give in a " feeding game": participants have
to feed toy animals and explain which food they would like to give to which animals
•
often used with children that are too young for experiments or as supplements to
(quasi-)naturalistic data or experiments
•
PRO
•
•
more controlled than other com. events.
•
more “natural” than experiments
•
provide data for low frequency phenomena
•
analysis not limited to target phenomena
CONTRA
•
age limitations, depending on the task
•
lack of negative evidence
Experiments
•
controlled manipulation of one or more variables (e.g. syntactic construction: active
vs. passive sentence) in a fixed setting and with a fixed procedure
•
e.g. elicited production experiments: participants are asked to provide a specific
morphological form of a given word (e.g. the plural forms cars and mice of the nouns
car and mouse)
•
PRO
•
•
very controlled data type
•
provide data for low frequency phenomena
•
can provide negative evidence (grammaticality judgement experiments)
CONTRA
•
less “natural” than observed or staged communicative events or elicitation games
•
analysis limited to target phenomenon
•
age limitations
3
Cross-sectional vs. Longitudinal vs. Interval Designs
•
longitudinal:
period of time
a series of recordings of one or more speakers over an extended
PRO
•
appropriate for studies of development
•
representative for the individual speaker
CONTRA
•
•
less representative for the variety
•
long recording period
cross-sectional: recordings of several speakers one point in time per speaker
PRO
•
short recording period
•
representative for the variety under study
CONTRA
•
•
less representative for the individual speaker
•
not direct investigation of development
interval/mixed: several blocks or continuous series of recordings, for several
individual learners
or for several groups of learners
example I (age in yrs;ms): 1 recording per year
Group I (3 children)
1;0
2;0
3;0
4;0
Group II (3 children)
5;0
6;0
7;0
8;0
Group III (3 children)
9;0
10;0
11;0
12;0
example II: 1 recording per week, 3 children
I: 1;0-2;0 II: 2;0-3;0
III: 3;0-4;0
PRO
•
large age range in limited recording time
•
representative for speaker AND variety
CONTRA
•
no analysis of individual development across the entire time span
4
Study Design and Method
•
Naturalistic sampling and semi-structured elicitation can be used in longitudinal
studies.
•
Experiments can lead to the development of task-specific or stimulus-specific
strategies (low validity). They can thus not be used in longitudinal designs (or only at
the end).
•
Experiments require larger groups of participants (typically at least about 20
participants).
Combining Methods
•
It is advisable to provide converging evidence, using a combination of different data
collection methods.
•
If little is known about the acquisition of a phenomenon, one can start with one or
two sessions with naturalistic sampling and semi-structured elicitation:
•
•
to build up a relationship with participants
•
to test knowledge of words in experimental stimuli
•
to use information about spontaneous “errors” to fine-tune experiments (e.g.
grammaticality judgment).
Naturalistic samples can be collected “around” blocks of elicitation/experiments, e.g.
while (un)packing toys.
Using Elicitation Games to overcome the Lack of Data in Spontaneous Speech
Constructions in spontaneous speech can differ widely in their frequency and
distribution:
Table 1: A German First Language (L1) corpus (Eisenbeiss 2003)
child
age
files (w. elicit.)
utterances
AND
2;1
1
1.4500
ANN
2;4-2;9
6
1.977
CAR
3;6
1
1.795
HAN
2;0-2;8
8
1.399
LEO
1;11-2;11
15 (all)
4.383 (4.383)
MAT
2;3-3;6
18
1.978
SVE
2;9-3;3
15 (10)
3.811 (2814)
5
total
1;11-3;6
64 (20)
16.793 (7.197)
Table 2: Contexts for Different Types of Noun Phrase Elements
(Eisenbeiss 2003)
Noun phrase with
context for
number
% of utterances
correlation with mean length of
utterance, in words (MLUw)
Article
2.646
28
0.489; sign.
adjective + article
249
3
0.274; n.s.
possessive ‘s
19
<1
-0.097; n.s.
•
In Table 2, you can see that some NP types were found to be comparatively frequent
and became more frequent with increasing utterance length (MLU).
•
BUT: Some noun phrase types (e.g. with contexts for adjective+article or ‘s) :
•
•
were rare
•
occurred in some files, but not in others
•
did not become more frequent over time, even though children’s utterances
got longer over time: there was no correlation between the relative frequency
of a particular utterance type and the mean length of utterance for the
respective recording.
In recordings with semi-structured elicitation games (represented by black filled
symbols in Figure 1), a higher percentage of utterances involve a context for a
possessive s-marker than in recordings with spontaneous speech (represented by
unfilled symbols). For Leonie, elicited and non-elicited data for the same child in the
same recording period and MLU-range (Leonie) were available. Spontaneous speech
data contained NO s-contexts, but such contexts were found in the elicited data from
the same period.
Figure 1: The Frequency of Possessive s-Contexts in Naturalistic and Elicited Data
6
KORP US
6
% of analysable utterances
sve
5
sve
mat
4
leo
3
leo
han
2
car
1
ann
0
and
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
MLU
Stimulus-Based Elicitation: Interactional Setting
•
Director/matcher (or “confederate description”):
A “director” describes a scene/object etc. and a “matcher” who is not able to see this
scene/object, has to recreate it.
E.g.: The matcher has to build a toy house identical to the one created by the director.
•
Speaker/Listener:
A speaker provides information for someone who does not have access to this
information. E.g.: The speaker retells a story (s)he heard/read while the listener was
not in the room.
•
Co-Players:
All participants are involved in a game and provide each other with information to coordinate their actions.
E.g.: The players are involved in a construction or puzzle game where not everyone
has access to all pieces.
Stimulus-Based Elicitation: Target Type
•
broad-spectrum: generally encouraging participants to speak
7
•
•
•
frog story: a picture book w/o words used
to elicit narratives (Berman & Slobin, 1994)
•
interview techniques
•
re-telling of videos or stories
form-focused: the use of a particular form or construction
e.g. a game where children have to find the possessions of animals and describe the
combination using an adnominal possessive construction like the horse's saddle
meaning-focused: the linguistic encoding of a particular function or meaning (which
can be encoded in different ways)
•
e.g. “circle of dirt”: a picture book w/owords used to elicit descriptions of partwhole relationships and actions affecting (body) parts (Eisenbeiss, McGregor, &
Schmidt 1999)
•
“cut-and-break”: video stimulus created for cross-linguistic studies of “separation
and material destruction” events (Bohnemeyer, Bowerman and Brown 2001)
• see p. 13 for References and download information.
8
3 Tasks (Eisenbeiss 2009)
The Bag Task: broad-spectrum
This co-player task involves a bag with pockets for blocks and animals of
different sizes and colours. The bag has pockets that match the animals in
colour and have coloured buttons, ties, etc.; and children frequently refer to
colours, sizes and locations when they ask other players to help them hide or
find animals in the pockets or to attach the pockets to the Velcro-strips on
the bag.
As an alternative, one can use sets of small bags that are not attached as
pockets to a big bag, but can be carried in a big bag. This requires fewer
sewing skills and allows for a more flexible use of different bags.
The Picture-Pairing Task: form-focused or meaning-focused
This director-matcher task is similar to the game “memory”, but the pictures are
matched - not identical. For very young children, memory load can be reduced by
visible pictures.
Variants
•
contrast-oriented:
The child has to describe what is on the two pictures and to say whether they match
(e.g. small green bananas vs. large yellow bananas)
•
combination-oriented:
The child is asked to form a phrase or an utterance with components that are depicted
on the matching pictures (e.g. the hairdresser’s sheep or the sheep is helping the
hairdresser).
•
Form vs. meaning-based
•
The task can be used to focus on particular forms (e.g. constructions with the
verb help or noun phrases with adjectives).
•
It can be used to investigate how children encode particular meanings. For
instance, picture pairs of owners and their possessions can be used to study
how children talk about possessions (e.g. the queen's crown vs. the crown of
the queen?).
9
Figure 2: Sample Pictures for the Picture-Pairing Task
The Puzzle-Task: form-focused or meaning-focused
•
In this co-player task, children have to describe contrasting pictures on a puzzle
board, adults find the matching pieces, and the children then put them into the
matching cut-out on the puzzle-board.
•
The puzzle board involves exchangeable pictures and puzzle pieces (see Figure 3)
•
It can be used to elicit particular forms (e.g. or to elicit the linguistic encoding of
particular meanings (e.g. actions on body-parts, see Figure 4).
10
Figure 3: The Picture-Pairing Task
11
Figure 4: Some Sample Pictures for the Puzzle Task
12
Some Freely Available Elicitation Stimuli
The BOWPED Stimulus
Bowerman, Melissa & Eric Pederson. 1992. Topological relations picture series.
In Stephen C. Levinson (ed.), Space stimuli kit 1.2: November 1992, 51.
Nijmegen: Max Planck Institute for Psycholinguistics.
http://fieldmanuals.mpi.nl/volumes/1992/bowped/
The Caused Positions Stimulus
Hellwig, Birgit & Friederike Lüpke. 2001. Caused positions. In Stephen C.
Levinson & N.J. Enfield (eds.), Manual for the field season 2001, 126-128.
Nijmegen: Max Planck Institute for Psycholinguistics.
http://fieldmanuals.mpi.nl/volumes/2001/caused-positions/
The Circle of Dirt Stimulus
Eisenbeiss, S., McGregor, B. & Schmidt, C. (1999). Story book stimulus for the
elicitation of external possessor constructions and dative constructions ('the circle
of dirt'). In David Wilkins (ed.), Manual for the 1999 Field Season, 140-144.
Nijmegen: Max Planck Institute for Psycholinguistics.
http://fieldmanuals.mpi.nl/volumes/1999/story-book-stimulus-possessor-dativeconstructions
The Cut-and-Break Stimulus
Bohnemeyer, J., Bowerman, M. & Brown, P. (2001). Cut and break clips.
Manual’for the field season 2001, 90--96.
http://fieldmanuals.mpi.nl/volumes/2001/cut-and-break-clips/
See the following webpages for more downloads of stimuli:
http://fieldmanuals.mpi.nl
http://experimentalfieldlinguistics.wordpress.com/experimental-materials/
http://www.iris-database.org/iris/app/home/index
13
References
Berman, R. A. & Slobin, D. I. (1994). Relating events in narrative: A crosslinguistic
developmental study, Vol. 1. Psychology Press.
Eisenbeiss, S. (2003). Merkmalsgesteuerter Grammatikerwerb. .PhD-dissertation,
University of Duesseldorf, Germany.
Eisenbeiss, S. (2009) Contrast is the Name of the Game: Contrast-Based Semi-Structured
Elicitation Techniques for Studies on Children’s Language Acquisition. Essex
Research Reports in Linguistics 57.
http://www.essex.ac.uk/linguistics/publications/errl/errl57-7.pdf
14