A comparison of elicitation procedures for connected
language analyses in person with aphasia:
Concurrent validation of the
Story Retell Procedure
Malcolm R. McNeil
Jee Eun Sung
Stacey Pavelko
Dorothy Yang
Brenda Smolky
Sheila R. Pratt
Tepanta R.D. Fossett
Patrick J. Doyle
Speech Motor, Aphasia, Cognitive Laboratory
VA Pittsburgh Healthcare System
AND
Department of Communication Science and Disorders
University of Pittsburgh
Correspondence:
Malcolm R. McNeil, Ph.D.
Professor and Chair
Department of Communication Science and Disorders
University of Pittsburgh
4033 Forbes Tower
Pittsburgh, PA 15260
412-383-6541 (phone)
412-383-6555 (fax)
mcneil@pitt.edu
BACKGROUND
Among the elicited and observed procedures used to describe, classify, diagnose,
measure change, quantify severity, and plan intervention for persons with aphasia, the
measurement of connected spoken language has become a stable and valued procedure
for many of these purposes. Although recognized, the most valid, reliable, and efficient
methods for sampling connected language has received relatively little experimental
attention from clinical and experimental aphasiologists. The recently developed Story
Retell Procedure (SRP) (Doyle, et al, 2000) has the unique measurement advantage of
predetermined targets for the retold stories; however, it relies on the comprehension as
well as the production of the individual for the samples from which linguistic
measurements are performed. Its concurrent validation has not been established.
METHOD
To assess the concurrent validity of the SRP it was compared to two other
published procedures for eliciting spoken language in persons with aphasia; the
“Cinderella Story” (Berndt, R. S., Wayland, S., Rochon, E. Saffran, E. & Schwartz, M.,
2000) and the five elicitation tasks of Nicholas, L. E. & Brookshire, R. H. (1993) (WAB
& BDAE picture, two sequenced picture; two novel picture, two procedural language,
and two personal information).
To date, eleven persons with aphasia (defined by their performance on two
standardized aphasia tests) have completed the seven language elicitation procedures
describe above. The experimental tasks were administered in random order across
participants and later transcribed and analyzed using SALT software. Although
additional analyses (e.g. number of story propositions, number of correct information
units, number of syntactically accurate and complete sentences) are planned on these data
and on those of nine additional participants with aphasia, five measures of verbal
productivity (# Utterances, # words, and # words per minute (WPM), mean length of
utterance (MLU), type-token ratio (TTR)), one measure of syntactic complexity (#
conjunctions), and three measure of verbal disruption (# mazes, # abandoned
sentences, % intelligible utterances) have been completed. Data were analyzed with
correlation coefficients and repeated measures ANOVA within measure across sampling
procedure.
RESULTS
Results show substantial inter-subject variability across language sampling
procedures (Figure 1) for some dependent measures. The SRP provided significantly
higher values for Number of Utterance, Words, Abandoned Utterances (except Cinderella
Story), and Conjunctions than all other sampling procedures. No significant differences
were observed between SRP and the other sampling procedures for MLU (except
WAB/BDAE pictures), WPM (except B&N pictures), or percent Mazes (Figure 2).
The SRP correlated highly and positively across all sampling procedures for the
measures that were not significantly different (MLU, WPM, Mazes). While The SRP
correlated positively and significantly with various language measures across most
sampling procedures, it correlated consistently highly with the Cinderella Story across
measures (Table 1).
DISCUSSION
The SRP provided a larger sample of behaviors than the other sampling
procedures for several variables and correlated highly and significantly with most
sampling procedures for both measures of verbal productivity and disruption. The SRP
correlated highly with the Cinderella Story procedure across most dependent measures.
CONCLUSIONS
Preliminary data suggest that SRP is a valid measure with which to sample
connected spoken language in persons with aphasia.
REFERENCES
Berndt, R. S., Wayland, S., Rochon, E. Saffran, E. & Schwartz, M. (2000). Quantitative
Production Analysis (QPA). Philadelphia: Psychology Press.
Doyle, P.J., McNeil, M.R., Park, G., Goda, A., Rubenstein, E., Spencer, K.A., Carroll, B.,
Lustig, A., and Szwarc, L. (2000). Linguistic validation of four parallel forms of a
story retelling procedure. Aphasiology, 14, 5/6, 537-549.
Nicholas, L. E. & Brookshire, R. H. (1993). A system for quantifying the
informativeness and efficiency of the connected speech of adults with aphasia.
Journal of Speech and Hearing Research, 36, 338-350.
Figure 1. Inter-subject variability across sampling procedures
Number of Utterances by Subject
80
70
60
50
40
30
20
10
0
SRP
CIND
WAB/BDAE PICT
SEQ PICT
B&N PICT
PROCED
PERSON
1
2
3
4
5
6
7
8
9
10
11
Number of Words by Subject
800
700
600
500
400
300
200
100
0
SRP
CIND
WAB/BDAE PICT
SEQ PICT
B&N PICT
PROCED
PERSONAL
1
2
3
4
5
6
7
8
9
10
11
Words per Minute (WPM) by Subject
200.00
180.00
160.00
140.00
120.00
100.00
80.00
60.00
40.00
20.00
0.00
SRP
CIND
WAB/BDAE PICT
SEQ PICT
B&N PICT
PROCED
PERSONAL
1
2
3
4
5
6
7
8
9
10
11
Mean Length of Utterance by Subject
20.00
SRP
15.00
CIND
WAB/BDAE PICT
10.00
SEQ PICT
B&N PICT
5.00
PROCED
0.00
PERSONAL
1
2
3
4
5
6
7
8
9
10
11
Type Token Ratio by Subject
1.00
SRP
0.80
CIND
0.60
WAB/BDAE PICT
0.40
SEQ PICT
B&N PICT
0.20
PROCED
0.00
PERSONAL
1
2
3
4
5
6
7
8
9
10
11
Number of Conjunctions by Subject
70
60
50
40
30
20
10
0
SRP
CIND
WAB/BDAE PICT
SEQ PICT
B&N PICT
PROCED
PERSONAL
1
2
3
4
5
6
7
8
9
10
11
% of Mazes by Subject
50.00
45.00
40.00
35.00
30.00
25.00
20.00
15.00
10.00
5.00
0.00
SRP
CIND
WAB/BDAE PICT
SEQ PICT
B&N PICT
PORCED
PERSONAL
1
2
3
4
5
6
7
8
9
10
11
Number of Abandoned Utterances by Subject
20
SRP
15
CIND
WAB/BDAE PICT
10
SEQ PICT
B&N PICT
5
PROCED
0
PERSONAL
1
2
3
4
5
6
7
8
9
10
11
Intelligibility by Subject
120.00
SRP
100.00
CIND
80.00
WAB/BDAE PICT
60.00
SEQ PICT
B&N PICT
40.00
PROCED
20.00
PERSONAL
0.00
1
2
3
4
5
6
7
8
9
10
11
Figure 2. Variability among sampling procedures
Number of Utterances by Sampling Procedure
60.00
50.00
40.00
30.00
20.00
10.00
0.00
SRP
CIND
WAB/BDAE
PICT
SEQ PICT
B& N PICT
PROCED
PERSONAL
Sampling Pr ocedur e
Number of Words by Sampling Procedure
500.00
450.00
400.00
350.00
300.00
250.00
200.00
150.00
100.00
50.00
0.00
SRP
CIND
WAB/BDAE
PICT
SEQ PICT
Sampling Pr ocedur e
B& N PICT
PROCED
PERSONAL
WPM by Sampling Procedure
160
140
Words Per Minute
120
100
80
60
40
20
0
SRP
CIND
WAB/BDAE PICT
SEQ PICT
B&N PICT
PROCED
PERSONAL
PROCED
PERSONAL
Sampling Procedure
MLU by Sampling Procedure
12.00
Mean Length of Utterance
10.00
8.00
6.00
4.00
2.00
0.00
SRP
CIND
WAB/BDAE PICT
SEQ PICT
Sampling Procedures
B&N PICT
Type Token Ratio by Sampling Procedures
0.90
0.80
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
SRP
CIND
WAB/BDAE PICT
SEQ PICT
B&N PICT
PROCED
PERSONAL
PROCED
PERSONAL
Sampling procedure
Number of Conjunctions by Sampling Procedure
45.00
40.00
35.00
30.00
25.00
20.00
15.00
10.00
5.00
0.00
SRP
CIND
WAB/BDAE
PICT
SEQ PICT
Sampling Pr ocedur e
B& N PICT
Mazes by Sampling Procedure
16.00
14.00
Total Number off Mazes
12.00
10.00
8.00
6.00
4.00
2.00
0.00
SRP
CIND
WAB/BDAE PICT
SEQ PICT
B&N PICT
PROCED
PERSONAL
Sampling Procedures
Abandoned Utterances by Sampling Procedure
8.00
7.00
6.00
5.00
4.00
3.00
2.00
1.00
0.00
SRP
CIND
WAB/BDAE
PICT
SEQ PICT
Sampling Pr ocedur e
B& N PICT
PROCED
PERSONAL
Sentence Intelligibility by Sampling Procedure
101.00
100.00
99.00
Percent Intelligibility
98.00
97.00
96.00
95.00
94.00
93.00
92.00
91.00
SRP
CIND
WAB/BDAE PICT
SEQ PICT
B&N PICT
PROCED
PERSONAL
Sampling Procedure
Table 1.
Pearson correlation among SRP and other sampling procedures
CIND
# of Utterance
# of words
WPM
MLU
TTR
# of Conjunctions
% of Maze
# of Abandoned
Utterance
% of Utterance
Intelligibility
SEQ PICT
0.29
0.737(*)
0.945(**)
0.743(*)
0.862(**)
0.60
0.965(**)
0.943(**)
WAB/BDAE
PICT
0.16
0.28
0.837(**)
0.878(**)
0.14
0.58
0.930(**)
0.28
PROCED
PERSONAL
0.06
0.32
0.859(**)
0.806(**)
0.20
0.40
0.950(**)
0.24
B&N
PICT
0.31
0.817(**)
0.842(**)
0.734(*)
0.815(**)
0.744(**)
0.966(**)
0.26
0.34
0.733(*)
0.913(**)
0.788(**)
0.55
0.51
0.918(**)
0.57
0.42
0.729(*)
0.926(**)
0.886(**)
0.40
0.52
0.58
0.705(*)
0.963(**)
0.963(**)
0.45
0.735(**)
0.48
-0.20
**. Correlation is significant at the 0.01 level (2-tailed).
* . Correlation is significant at the 0.05 level (2-tailed).