Computer-supported intervention
for children with language and
literacy problems
Ludo Verhoeven
In collaboration with Eliane Segers and Rosemarie Irausquin
University of Nijmegen
Possibilities computer
•
•
•
•
•
•
•
Connecting speech, orthography, semantics
Virtual reality
Hypertekst
Communication
Adaptivity
Responsivity
Motivational value
Digital learning environments
• Computer-assisted instruction
– Drills & practice
– Games
• Open learning environments
– (Automatic) feedback
– Communication
Computer-supported intervention
• Computer-assisted instruction
– Early language intervention (kindergarten)
– Intervention for poor readers
• Open learning environment
– Language in the content areas
Early intervention software
Schatkist met de muis
• Storybook reading
–
–
–
–
Story telling
Story illustrations
Semantic meaning
Word blending
• Language play
–
–
–
–
–
Multimedial songs
Book making
Postcard writing
Rhyme and word segmenting
Word making
Overview of a CDROM
(bold arrows show adaptivity)
Postcard writing
Rhyme and word synthesis: b-el
•Attention to word sounds
•Isolating beginning sound
•Word blending
Grapheme booklet: computer keeps track of individual progress
Word maker
Effects
• Positive effects
– Vocabulary
– Text comprehension
– Phonological awareness
• Effects equally strong for L1 and L2 learners
• Effects for children with SLI
– Rhyme
– Phonemic awareness
– Role of speech manipulation???
Research Tallal, Merzenich & associates
• Children with SLI:
– Problems with fast formant transitions in synthetic speech
(1974, 1981)
– Lengthening signal > better discrimination (1973)
– Lengthening formant transition > better discrimination
(1975, 1980)
– New algorithm: lengthening whole signal + amplifying
fast formant transitions up to 20dB
– Commercial program: Fast Forword (1996: Effectful after
100 hours in four weeks of training)
Difficulties with fast formant transitions
year
authors
subject group
speech material
conclusion
1974
Tallal & Piercy
12 aphasic children
12 controls
age 6.9-9.3 years old
/ba/ - /da/; formant transition
43 msec
aphasics have more
problems than controls
1980a
Tallal, Stark,
Kallmann &
Mellits
35 developmental
dysphasics
38 controls
average age: 6.8 years
old
/ba/ - /da/ as in Tallal and
Piercy 1974.
Synthetic copies of natural
utterances /ba/ - /be/ /bi/ /dae/, /de/ - /di/; formant
transition varying
naturally
dysphasics have more
problems than controls
1981
Tallal & Stark
same as above
/ba/ - /da/, /da/ - /ta/, /e/ - /ae/,
/dab/ - /daeb/, /sa/ - /sta/,
/sa/ - /fa/
dysphasics have more
problems than controls
in /ba/-/da/, /da/-/ta/ and
also /sa/-/fa/.
1989
Reed
23 reading disabled
23 'normals'
average age: 8.9
/ba/ - /da/, /e/ - /ae/,
reading disabled have
problems in /ba/ - /da/
as opposed to controls.
1992
Leonard,
McGregor
& Allen
8 SLI, 8 normals.
4.5-5.6 years old
/ba/ - /da/ , /dab/ - /daeb/, /i/ /u/, /dab-i-ba/ - /dab-uba/, /das/ - /daf/
SLI's have more problems
than controls not in
/dab/-/daeb/ and /i/-/u/
Benefits of speech manipulation
1975
Tallal & Piercy
12 aphasic children
12 controls (6.8-9.3)
/ba/ - /da/; formant transition extended
from 40 to 80msec
positive effects
1980
Frumkin & Rapin
20 dysphasic children
9 control children
average age 9.6
/ba/ - /da/ and /a/ - /u/
Synthetic speech, 2nd and 3rd formant
40msec or 80 msec; total length 250
msec
positive effects for subgroup
1980b
Tallal, Stark,
Kallman & Mellits
14 developmental
dysphasics
23 normally developing
age 5-9
/ba/ - /da/
Synthetic copies of natural utterances
/be/ - /be/, /bi/ - /dae/, /d/ - /di/;
formant transition varying naturally
results aphasics dependent on
syllable speed and ISI time
1982
Alexander & Frost
24 children with
language/speech problems
(7.2-11.6)
/ba/ - /da/. Formant transitions: 80, 70,
60 and 40 msec.
positive effects
1984
Blumstein, Tartter,
Nigro & Statlender
16 aphasics, 6 controls
2 synthetic /ba/ - /da/ - /ga/ continua
with formant transition 65 and 85 msec.
no effects
1985
Riedel & StuddertKennedy
12 adult aphasics.
average age 55
/ba/ - /da/
Formant transition 30 and 82 msec.
no effects
1985
Tallal, Stark &
Mellits
26 developmental
dysphasics
same as Tallal and Stark 1981
unclear
1996
Stark & Heinz
11 children with output
disorders
21 children with SLI 22
controls (6-10 yrs)
/ba/ /da/ Klatt synthesis. Formant
transition 30 to 80 msec. in steps of 10.
positive effect for children
with expressive and receptive
problems
1999
Bradlow et al.
32 children with learning
problems
72 controls (6 –16 yrs)
two /da/ - /ga/ continua
40 msec and 80 msec formant transition
no effects
Training studies using speech
manipulation
1996
Tallal et al.
22 SLI
average age 7.4
Computer games with or without
speech manipulation (Fast Forword)
positive effects
1999
Habib, et al.
12 dyslexics
age 10-12
Listening exercises with or without
speech manipulation
positive effects
2001
Gillam et al.
4 SLI
average age 7 ;3
Fast Forword vs Laureate Learning
software
no differential
effects
2003
Troia &
Whitney
37 LD: 25 FFW/12 C
average age 9;4
Fast Forword
overall effect on
expressive language
only
Study 1: Speech manipulation
• Participants
– 21 children with SLI vs 24 NLA controls (5 yrs)
• Stimulus set
– Five contrasts: b-p, d-t, v-w, h-g, b-d
– 60 word pairs, e.g., buik-duik (12x60 items)
• Speech manipulation:
– Normal speech
– Amplifying fast formant transitions
– Slowing down speech signal
– Amplification + slowing down
Oscillogram normal vs amplified speech
(enhanced fast transitional elements): /buik/ - /duik/
Study 1 (continued)
• Results
– Normal children perform better than SLI children
– +/- voice is more complex than place contrasts
– No effect of speech manipulation
• Conclusion
– No replication of effects reported by Tallal &
Piercy
Study 2: Training with natural speech
manipulation
• Participants (kindergarten)
– 36 SLI children: 24 experimental vs 12 control
• Training: rhyme and word blending
• Normal speech (N1=12)
• Manipulated speech (N2=12)
• Procedure
– Pretest-posttest-retention test
Results study 2
0,3
0,2
0,1
progress at
posttest
progress at
retention
0
-0,1
-0,2
-0,3
control group
normal speech
manipulated
speech
Conclusions Study 2
• Program is effective for children with SLI
• Significant retention effects
• No effect for speech manipulation
Study 3: Training with synthetic speech
manipulation
• Participants (kindergarten)
– 19 SLI children and 24 NLA children
• Stimulus set: minimal word pairs
– Normal
– Slowing down entire speech signal (like in FFW)
– Slowing down just formant transitions
Results study 3
100
90
normal speech
80
slowing down
entire speech
signal
slowing down
formant transitions
70
60
50
40
NLA
SLI
Conclusions Study 3
• Program is effective for NLA and SLI
• Positive effect speech manipulation for SLI
• No difference between slowing down
entire signal or formant transitions
General conclusions
• SLI children have difficulties in phonological tasks
• Phonological training is helpful for SLI children
• Role of speech manipulation is inconclusive
• Limitations: small N, limited hours of training
Dyslexia
• Phonological deficit
– Phonological memory
– Phonemic segmentation
– Access to phonology
• Sensory defects
– Processing brief sensory cues
– Processing rapidly changing sequences
– Limited use of temporal information
LEESLADDER:
Adaptive computer program for poor readers
• cd-rom 1: alphabetic principle, phonological awareness,
decoding of CVC words
• cd-rom 2: automatisation of reading and spelling of simple
word structures + basic reading comprehension
• cd-rom 3: reading/spelling monosyllabic words with
consonant clusters and specific orthographic patterns (eeuw, -ooi, -ng, etc.) + reading comprehension
• cd-rom 4: decoding multisyllabic words + advanced reading
comprehension
Sources
database
graphemes
words
Sentences, etc..
Adaptive
navigation
module
Planning
abilities
Parameters
Pupil part
Play types
(Sub)abilities
Pupil
database
Pupil ID
Learning history
Teacher
part
Basic 1
Grapheme
discovery
Basic 2
Type &
copy
Reading
and
Grapheme order
correct
Sound
discovery
Word
closure
Grapheme- and
phoneme knowledge
Auditory
synthesis
Simple spelling
Letter
flashing
Word reading
correct
spelling 1
Reading
and
Picture-word
Grapheme
order fast
spelling 2
Reading
and
spelling 3
Visual word
dictation
Auditory
word
dictation
Word-picture
Word reading
fast
Word rows 1
Word rows 2
Flash words
Letter test
Basic 1
Grapheme
discovery
Phoneme
discovery
Basic 2
Type and
copy
Word
closure
Reading
and
Grapheme order
correct
Grapheme and
phoneme knowledge
Auditory
synthesis
Elementary
spelling
Flash letters
Word reading
correct
spelling 1
Reading
and
spelling 2
Reading
and
spelling 3
Picture-word
Grapheme
order fast
Visual word
dictation
Auditory
word
dictation
Word-picture
Worden
reading fast
Word rows 1
Word rows 2
Flash words
Mean Odd-one-out alliteration score
Effects on phonological awareness
10
9
8
7
6
5
4
3
2
1
0
Training
Control
Pretest
Posttest
Mean Score on Pseudoword Test
Effects on word decoding
16
14
12
10
8
6
Training
Control
4
2
0
Pretest
Posttest
Correlations between training
intensity and reading gains
Exercises per week
Days per week
CVC word reading test
.53*
.56*
Word reading test
.54*
.58*
Pseudoword reading test
.62*
.59*
Speed vs comprehension training
50
45
40
35
30
CVC
CCVCC
Bisyllabic
25
20
15
10
5
0
Speed1
Speed2
Compreh1
Compreh2
General discussion
•
•
•
•
•
•
Age: plasticity of the brain
Contents of training
Synthetic vs natural speech
N of hours of training (minimum 100?)
Effects: phonology vs information processing
Necessity of control groups
Open Learning Environment: facilitating
language and knowledge construction
Communication
Text sample
Communication: reaction types
30
25
20
reaction type
15
10
5
0
1
2
3
4
5
6
7
Reaction types: (1) (dis)approval, (2) question about the text, (3) personal,
(4) reasoning , (5) suggestion (6) hello/goodbye, (7) rest category
Online spelling feedback
14
12
10
8
num be r of
e r r or s
6
4
2
0
H
T
C
S
Conditions: Handwriting (H), Typing (T), Spellingchecker (C), Spelling suggestions (S)
Perspective
• Children need to learn how to communicate
in open learning environment
• Spelling development can be supported by
means of online feedback
• Children with learning problems equally
benefit from the environment
• CIA can be integrated in open learning
environments