Modality Based Working Memory:
James Sulzen
School of Education
Stanford University
April 1, 2001
Abstract
This study tested a hypothesis that working memory is primarily modally organized. A free recall task was
performed by presenting randomized stimuli sequences in seven presentation modalities (visual (V), auditory (A),
haptic (H), kinesthetic (K), linguistic-auditory (LA), linguistic-visual (LV), and spatial-auditory (SA)). The same
number of stimuli was presented in each modality on any given trial run. Results showed recall was linearly
dependent upon the number of items in each modality up to a limit of about three items presented for a modality
and then leveled out thereafter. Recency and primacy effects indicated that at least several of the modality recall
sequences operated with differing underlying processes indicating further support for the independent modalities
memory hypothesis.
Modality Based Working Memory
Table of Contents
INTRODUCTION .............................................................................................................................................3
MODALITY-ORGANIZED COGNITION ....................................................................................................................4
PREDICTIONS ..................................................................................................................................................6
METHOD.........................................................................................................................................................7
RESULTS.........................................................................................................................................................8
DISCUSSION .................................................................................................................................................13
CONCLUSION ...............................................................................................................................................15
REFERENCES ...............................................................................................................................................16
APENDICES ..................................................................................................................................................17
APPENDIX A – LIST OF ALL STIMULI ..........................................................................................................17
STIMULUS CODES ..........................................................................................................................................19
APPENDIX B –ORGANIZATION OF TRIAL RUNS .........................................................................................22
APPENDIX C – PROCEDURES .......................................................................................................................25
APPENDIX D – ORGANIZATION AND ADMINISTRATION OF THE STIMULI ...............................................27
APPENDIX E – TABULATED RECALL DATA ................................................................................................29
APPENDIX F – RAW RECALL DATA .............................................................................................................32
APPENDIX G – NORMALIZATION FORMULA FOR CALCULATING SERIAL RECALL PROBABILITIES .....40
APPENDIX H – TABULATED RECENCY AND PRIMACY DATA ....................................................................41
APPENDIX I – RAW RECENCY AND PRIMACY DATA ..................................................................................42
APPENDIX J – CLUSTERING OF SUBJECT RESPONSES ...............................................................................47
-2-
Modality Based Working Memory
Introduction
"The study of models of memory often seems like a backwater in the overall study of memory.
Models do not have a prominent place in experimental studies of memory and they are not used
or examined by most researchers in the field... Recent development of models of long-term
memory has proceeded relatively independently of other areas of memory research." (Ratcliff &
McKoon, 2000, p. 571)
Studies of human short term and working memory have a very rich and long history (Ebbinghaus, 1885; James,
1890; Miller, 1956; and for surveys: Crowder, 1993; Bower, 2000; Baddeley, 2000 ). A number of models of
human memory and working memory have been proposed and tested over time, especially those involving verbal
or visual elements. There have also been a number of studies demonstrating various modal forms of short term
memory (STM) such as for haptic and olfactory capacities (Schurman, 1973; White, 1997). Baddeley and Hitch’s
(1974) classic modal model of working memory combining a spatio-visual, phonological, and executive control
system was an initial attempt to articulate perceived modal-related sub-components of working memory. Since
then, it seems reasonable to suppose that working memory is in fact fractionated among a number of modular
systems as evidence accumulates for the existence of more and more different components (Weiskrantz, 1987;
Baddeley, 2000). Recently, fMRI evidence has started to accumulate for a neurological basis for the phonological
loop (Paulesu, Frith, & Frackowiak, 1993; Awh et al., 1996) and even for a modal basis of representing categories
of objects such as living things (Schill-Thompson et al., 1999).
In addition to the mounting evidence that both working memory and perhaps long term memory (LTM) are
organized along modal lines, there is strong evidence to indicate that the modal systems highly interact with each
other. In the Schill-Thompson study (1999), it appears that visual centers are always activated whenever a subject
is asked to think about any aspect of a living thing (even such as parts of or the food of living things – i.e., “are
snails edible”). This is taken to indicate that the category of living things seems to have a primary visual element
which seems principally responsible for triggering other modalities, and brain damage to a modal visual area might
therefore well impair retrieval of the associated memories in the other modalities. Cross-modal priming is a fairly
clear example of interaction. McKone (McKone & Dennis, 2000) found that auditory or visual stimuli acted to prime
stimuli in the other modality. Perhaps of more interest in terms of the current writing, they found that same
modality priming has a greater effect than cross modality priming, and that visual versus auditory priming of nonwords is different (auditory performs better). McKone interprets these results as indicating a perceptual basis locus
for priming with some form of weak re-encoding occurring to effect the cross-modal priming.
There is also evidence for non-sensory based, but modal storage. Penney (Penney, 1989) reviewed the literature
on auditory and visual modality effects and concluded that auditory and visually presented words were re-encoded
in a phonological store accessible from either, and that the auditory and visual channels represent two separate
processing streams. Her argument is based upon five points:
1) Improved ability to perform two concurrent verbal tasks when different input modalities are employed
relative to the single-mode situation;
2) Improved memory when different items are presented to two sensory modalities rather than one;
3) Selective interference effects within, but less so across, modalities;
4) Subjects' preference for, and greater efficiency of, recall organized by modality than by time of
presentation; and
5) The presence of short-term memory deficits that appear to be specific to the auditory or visual modalities.
Additionally, Penney showed that bilingual speakers prefer to organize recall tasks by modality of presentation, as
opposed to organizing recall by language of presentation, time of presentation, or category of item.
Another bilingual study (Dehaene et al., 1999) showed that precise arithmetic calculations are carried out in one’s
native language (i.e., the language in which arithmetic was presumably learned), whereas approximate arithmetic
-3-
Modality Based Working Memory
calculations are carried out via visual and spatial means. This finding in conjunction with the concept of the
independent phonological store, leads to an implication of language, or perhaps rather a linguistic capability,
existing independently of any of the standard modalities.
On an informal, but perhaps intuitively satisfying basis, as far back as 1890 William James (James, 1890) provides
an elegant example of cross-modal encoding of knowledge. Holding open the lips prior to thinking of any word with
labials or dentals such as "bubble" or "toddle" distinctly affects most people's recall process. (“Is your image under
these conditions distinct? To most people the image is at first ‘thick’ as the sound would be if they tried to
pronounce it with lips parted.” p. 63). This would seem to be an example of interfering with a cross-modal retrieval
across at least the haptic (touch), kinesthetic (sensomotor), visual, and verbal systems.
Given the evidence for both some sort of modularized sub-specialization of working memory, some of which
certainly seems to organize along modal lines, it seems reasonable to suppose that each modal sensory system
may have its own working memory component. Goldstone and Barsalou (1998) have argued that there are many
reasons to believe that much of cognition is perceptually based and proceeds via perceptual representation
processes. They argue along the following lines:
1) That many if not in fact all of the properties associated with amodal symbol systems can be achieved with
perceptually-based systems (such as productivity);
2) Raw perceptual processing is often much more powerful for certain tasks than an equivalent amodal
system;
3) Perception naturally supports similarity;
4) Perception can be readily tuned to conceptual demands;
5) Perceptual simulation occurs in conceptual tasks and which have no explicit perceptual demands (for
example, Maxwell’s imagining microscopic spinning spheres in dielectrics when developing his
Electrodynamics equations (Nersessian, unpublished), or Einstein utilizing his visualizations of space-time
when developing relativity).
Countering these claims and conjectures, have been theories of episodic, semantic, and other memory
organization (Baddeley, 2000). There is also strong evidence that people can organize working memory around
categories – that is to say that structuring items by category in effect seems to create something of a “separate”
short term memory for each category leading to a two to three fold improvement in working memory capacity
(Watkins & Peynircioglu, 1983; Bower, et al., 1969). These category effects even show recency and primacy
effects. We will address these issues of categorization and non-modal organization in the discussion section.
Modality-Organized Cognition
The evidence for multiple, modality-related working memory components leads to a supposition that perhaps each
modality has its own working memory and some level of cognitive processing capability. If each modality has its
own working memory and processing capability, then why not its own long term memory and its own deeper
cognitive processing capability? Following these conjectures to some sort of logical conclusion leads to a possible
memory and cognitive functional organization as illustrated by Figure 1.
Figure 1 illustrates that a certain number of modal units interact with each other to create the experience of
cognition. Some of these are “first-level” modal processing loci, each directly connected to its own sensory system
via the sensory registers. There are also a number of “second-level modal loci” each with its own specialization. In
this model, every modality loci (hereafter referred to as modalities) is connected to and capable of stimulating or
receiving stimuli from any other modality. This interaction probably operates through or in conjunction with the type
of centralized switching network referred to as a “central executive” (Baddeley & Hitch, 1974). The second-level
stimuli have no direct connection to external sensory registers and so must receive their sensory inputs only by
first-level restimulation.
-4-
Modality Based Working Memory
The set of modalities represented here were selected because experimental evidence indicated a functional nexus
for each and because they seem to represent a minimal set that spans many cognitive phenomena. There may
also be “tertiary” or other modalities serving to organize social cognition, personality or other functions, but the
above model does not address such possibilities. The model provides an organizing framework for representing
relatively low-levels of cognition involving perception and knowledge representation.
Figure 1 – Modality-organized cognition
Sensory
Registers
G
H
K
O
First Level
Modalities
A
V
L
S
Second Level
Modalities
E
A - Auditory
G - Gustatory
H - Haptic
K - Kinesthetic
O - Olfactory
V - Visual
E - Emotional/affective
L - Linguistic
S - Spatial
The rest of this writing will use the single-letter abbreviations listed in Figure 1 to identify each of the modal
systems. When it is necessary or useful to distinguish which first level modality is interacting with a given second
level one, the two letters are combined, so “LV” means a visually presented linguistic item, while “SA” means an
auditory spatial stimulus.
Figure 1 should be interpreted in light of the following:
-
Representational Systems: Each modality should be thought of as a “representational system” which
represents processes, knowledge, perceptions, and sensory experience in its own particular way. V
represents knowledge in pictures and images, A in sounds, and so on. K is the kinesthetic sensomotor
system. L is a pure linguistic system that represents knowledge and does its processing in terms of
sequenced and syntactically ordered symbols. S is a system that represents spatial knowledge and
performs spatial processing. E controls our affective memories and processing. The other modalities
should be self-explanatory.
-
Completeness of each modality: In this model, each modality is a complete cognitive processing
system with its own working memory, long term memory, and processing capabilities. The type and
manner of internal organization is probably very specific to the given modality (i.e. S is probably very
differently organized than E or than V, for example). This helps explain some of the modality
differences observed in the literature such as the slight superiority of recalling auditory-presented
words as opposed to visually presented ones during free recall tasks.
-
Cross-stimulation and multi-modal representations: Each modal system is constantly stimulating each
other system with its outputs, including stimulating itself with its own outputs (i.e. feedback). This cross
stimulation probably provides a capacity for feedback loops and re-encoding of stimuli, as well as
higher level organizations of cognition.
The question arises as to how these separate systems combine or interact, and why is it not more obvious that
such separate systems exist? Following evidence from Schill-Thompson (1999), it seems probable that it may
often require several cross-stimulating modal systems to meaningfully represent concepts and various sorts of
-5-
Modality Based Working Memory
knowledge. Consider the category of 'living things', which, according to their data, appears to have a necessary
visual component,but which also has elements in other modalities to define its representation. If the visual portion
of the ‘living things’ representation were impaired via a lesion for example, then the other elements that make up
the 'living things' representation would still be intact, but not be capable of being stimulated. Therefore the person
loses knowledge of what a 'living thing' is, even though most of the knowledge is still available (and indeed may be
accessible via other cue paths.) The concept of ‘living things’ cannot be kicked into gear because the necessary
visual element is missing from the stimulus chain. In a similar vein, James’ (1890) example with ‘bubble’ and
‘toddle’ could therefore be understood as indicating that the meaning or knowledge of these words is encoded
across the L, H, K, and V modalities; and that interfering with one modality (K, when the lips are parted,) interferes
with the retrieval process and the associated V image gets changed.
As for it not being more obvious that these hypothesized internal systems have a distinct existence, the explanation
might be that the extent of interactivity makes the whole seem like a monolithic entity making it tremendously
difficult to discern the individual elements. Consider, as an analogy, aborigines trying to discern the internal
structure of an automobile by being able to examine only its external appearance and perhaps drive it only in very
limited and controlled circumstances. With neither the concepts nor useful tools for investigating internal
combustion engines, they would have little chance of deducing internal electrical, carburetion, fuel, cooling, exhaust
and other internal systems (although they might be able to deduce the existence of some systems such as steering
and brakes that have relatively easily observed external correlates.) Similarly, with the human cognitive system there is a tendency to regard memory as one large undifferentiated system with perhaps some salient subsystems
such as vision, auditory, or spatial processing.
Predictions
Given the above hypothesis regarding the modal basis for cognition, the following predictions seem likely:
1)
Single-modal presentations: A set of simple stimuli limited to a single modality is more likely to be
primarily encoded in that modality rather than being re-encoded and cross-stored. However, it is
necessary to remember that there are probably significant exceptions to this, for example the fact that
people seem to be particularly capable of recoding linguistic items among the V, L, and A modalities.
2)
Total working memory capacity: Assuming that each modality has an amount of independent working
memory, and that each one functions similarly to how we currently consider working memory to
function, then the total working memory capacity of a person should approximate the sum of the
capacities of the individual modal working memories. This prediction ignores duplicate encoding
affects and the apparent need to cross-modally encode some types of items, (i.e. the' living things'
category,) which would require using capacity from several modalities, and thereby reduce the
seeming total capacity of the system
3)
Testing working memory capacity: If the above predictions hold it should be possible to present a
tuned set of stimuli to fill each particular modal working memory. This should lead to an apparent
increased memory capacity compared to presenting a more randomly chosen set of stimuli. In fact,
there should be a linear increase in items recalled as the number of items presented in each modality is
increased. At some point there should be a leveling off of the number of items recalled despite a
continuing increase in the number of items presented.
4)
Seven plus or minus two: Much of the free recall literature has focused on presenting either images or
words as the basic stimuli, (with the words having either auditory or visual presentation). In terms of
the cognitive model presented here, this consists of stimuli in the V, LA, and LV modalities. As
mentioned above, it seems likely that people have a facile ability to cross-encode between the L, V,
and A modalities. If this is so, in terms of free recall literature, the V, LA, and LV modalities might be
thought of as one common store that should show the familiar “seven plus or minus two” total capacity
limitation characteristic of unordered free recall tasks.
5)
Primacy / recency effects: If each modality has its own working memory, then it seems likely each
should show some of the normal primacy and recency effects, (remembering initial and most recent
stimuli the best).
-6-
Modality Based Working Memory
These predictions are the basis for the experiment described in this study.
Method
Subjects
Subjects were 9 volunteers, all acquainted with the author, ranging in age from mid-twenties to early-sixties, (mean
age of 40 years); five female and four male. All subjects had either graduate degrees or were engaged in postgraduate study at major institutions.
Design
Subjects performed a free recall task in which the basic manipulation varied the number of items presented in each
of seven modalities, (A, V, LA, LV, H, K, and SA). These particular seven modalities were chosen because it was
relatively easy to create a suitable stimulus sequence for each one to test the conjectures.
Trial run organization. Presentation sequences of stimuli (“trial runs”) were set up so each trial run had the same
number of stimuli from each of the seven modalities. For example, a given trial run had two stimuli in each
modality, another had three in each modality, and so on. The number of items in each modality will be referred to
as the IM count (items per modality). Within a given trial run, stimuli were randomly mixed in presentation order.
(For example, for a block with two stimuli from each of the seven modalities, the order of presentation might be: A,
LV, H, K, A, SA, K, LA, H, V, SA, LV, V, LA. At the completion of each presentation sequence, subjects were
asked to recall as many items as they could. Each stimulus was presented only once to any subject. Stimuli were
carefully screened to avoid inadvertent redundancies (i.e., saying the word “pig” and showing a picture of a pig).
Fill sequence. As the working memory of V, A, and L modalities are experimentally well established (Penney,
1989), it is important to establish the independent existence of other modal systems. Therefore a fill sequence of at
least eight items was included at the end of the trial run for each run of stimuli consisting of items in the V, LA, and
LV modalities to minimize the likelihood that Ss used these three modalities for cross-storage from the H, A, K, and
SA modalities.
Materials
Table 1 gives the total number of stimuli presented in each trial run, the number of modality-specific stimuli in each
run, and the length and make-up of each fill sequence. A set of cards was prepared for the experimenter to use in
administering stimuli. Each card listed one stimulus and identified both the modality and the stimulus to be
presented to a subject. In the case of the SA modality, a card indicated a direction relative to the subject, and the
experimenter clicked a staple remover twice at the appropriate location to generate a directional sound. See
Appendix A for a complete list of stimuli and Appendix D for how stimuli were organized and administered in the
experiment.
-7-
Modality Based Working Memory
Table 1 – Length and make-up of each trial run
Series
Trial
Run
Total
number
of
stimuli
Number of items in each of the
various modalities
IM Count
Fill Sequence
(V / LA / LV)
Number
of
Subjects
T0
16
-
16
(3/6/7)
4
T1
16
-
16
(4/5/7)
3
Series
T2
22
2
8
(2/3/3)
3
A
T3
29
3
8
(2/3/3)
3
T6
29
3
8
(2/3/3)
3
T7
30
2
16
(4/6/6)
3
T8
29
3
8
(2/3/3)
6
T9
36
4
8
(2/3/3)
6
T10
44
5
8
(2/3/3)
5
Series
B
The stimuli assigned to each trial run are listed in Appendix B. In assembling the trial runs, procedures were
followed to minimize the chance that any stimulus or stimulus sequence within a trial run had an undue recall bias
(see Appendix C for the details of how trial runs were assembled). Note that, in Table 1, the length of each trial run
is given by IM*7 + (length of fill sequence).
Procedure
Subjects were each presented with several trial runs. Each trial run consisted of a sequence of stimuli, which,
without interruption, was immediately followed by each subject’s attempt to recall as many items out of the
presentation as possible in whatever manner they chose. The experimenter recorded the recalled items in the
order recalled. A cassette recorder was used as a back-up to check that each subject’s recalled list was accurately
recorded.
Subjects were run in two separate series (identified as series A and series B), where each series had a separate
set of trial runs and its own separate group of subjects. Subjects within each series received exactly the same
treatment (i.e. same sequence of trial runs organized in exactly the same way from subject to subject). The
number of subjects was too small to try balancing the order of trial runs. It simplified performing the experiment and
analysis to give all subjects in a series the same treatment. Two series were used because subjects were all
unpaid volunteers and the initial pilot testing indicated that it was too time taxing to run subjects through more than
three or four trial runs, (especially with the longer sequences). This necessitated using two distinct groups of
subjects with two separate sets of trial runs to span the necessary range of IM values. Some unfortunate
differences cropped up in performance between the two series, which will be addressed in the discussion section.
No trial runs were conducted at IM=1 because of limited subject availability. It seemed superfluous to use a trial
run at such a simple level because subjects would likely have a high success rate at such a simple level, and the
IM=1 values would just show strong linearity in the data. In the interests of expediency, this linearity is assumed
herein.
Results
Coding of Answers
During recall, subjects typically identified stimuli via a single word or a simple description (i.e. “pig”, “rain”, “touched
me on the shoulder, back of head, and arm”, etc.). Since stimuli in the V, LV, and LA modalities had been carefully
chosen to avoid any redundancies across modalities, it was a easy to match the great majority of subject responses
to stimulus items in these modalities. Coding the H, K, A, and SA modalities was a bit more challenging as
-8-
Modality Based Working Memory
presentations were done non-verbally, but recall was verbal, necessitating a cross-modal translation on the part of
the subject to produce the recall. This meant that subjects gave a variety of descriptions and it required some
judgment in a number of cases to match a recalled item to a particular stimulus. In the case of the H, K and SA
modalities, a number of stimuli were combined to simplify the coding and increase the ability to accurately
distinguish subject responses. (See Appendix A for the differences between the encoding versus stimulus lists.)
Double checking was achieved by coding each recall sequence twice (with at least a few weeks between codings);
there were only five items out of a total of about 370 total whose coding was changed on the second go-through.
Analysis
Chart 1 shows the number of items correctly recalled by each subject for every trial run. There is a distinct linear
trend (r=0.86, p<.001), but the data is rather dispersed, especially for IM>2 values. An analysis of variance showed
significance F(4,30)=4.3, p<.01. However, only the IM=0 group showed any significant difference (p<.01), while
the IM=2 showed near significant difference from the IM=5 group (p<.08).
Chart 1 –
Number of correctly recalled items (each data point represents one subject’s
successful recall count for one trial run)
# items successfully recalled
25
20
Total Recalled
(V+LV+LA+A+H+K+SA)
15
10
5
V + LV + LA
0
-1
0
1
2
3
4
5
6
IM Count (number of items presented per modality)
Also graphed on Chart 1 are the totals for the visual and linguistic items (i.e., V+LA+LV). These correspond to
traditional free recall tests where subjects are presented either pictures or word items for recall. These modalities,
when summed together, have no particular sensitivity to the IM count with a correlation of r=0.04; they show a fairly
constant sum of about 7.5 recalled items (SD=1.4) across all values of the IM count.
Given that V+LV+LA is a near constant, Chart 2 shows the that a large fraction of the variability of the Total
Recalled value can be attributed to the sum of the recalled items in the four A, H, K, and SA modalities. The two
data sets correlate at r=0.89 (p < .01). The sum of A+H+K+SA also has a very linear dependency upon the IM
Count, correlating with r=0.90 (p < .01).
-9-
Modality Based Working Memory
Chart 2 – Graph of Total Recalled and Total of A+H+K+SA
25
Total
Recalled
20
# items
y = 2.5x + 8.3
R2 = 0.75
15
A+H+K+SA
10
V + LV + LA
5
y = 2.3x + 1.4
R2 = 0.79
0
0
1
2
3
4
5
6
IM Count (# items per modality)
Chart 3 shows the number of correctly recalled items for all IM Counts, broken out by the Series A and Series B
data. (Series A and B are the two sets of trial runs and of their corresponding subjects.) It also shows the totals for
the A, H, K, and SA modalities labeled as line D in the chart. The trend lines for the A and B series data are rather
distinctly different from each other and appear to break right at IM=3. This discontinuity between the two series will
be covered more in the discussion section. An analysis of variance showed possible significance between the
Total Recalled IM=2 group and the Series B data, with F(2,17)=2.8, p<.09.
The line labeled “C” in Chart 3 shows the average of the percentage of total items in a each trial run which were
correctly recalled. This shows a peak value at IM=2 and then declining thereafter. This indicates that subjects
were recalling a smaller and smaller proportion of the total number of items presented to them. An analysis of
variance showed significance F(4,29)=3.5, p=0.05.
- 10 -
Modality Based Working Memory
Chart 3 – Correctly recalled items for series A and B, and per cent recalled
Series #A
Series #B
(error bars +/- 1 SD)
120%
% correct minus fill sequence
(right axis)
20.0
B
100%
Total
Recalled
80%
15.0
C
A
60%
10.0
40%
A+H+K+SA
5.0
20%
0.0
0%
0
1
2
3
4
IM Count (# of items per modality)
5
% Correct
# Corrrect Items
25.0
6
Recalled Items in Each Modality
Chart 4 – Averages of V, LV, and LA Recalled Items
# of items
Chart 4 shows the graph of the
9.0
means of the recalled items for V,
LV, LA, and A. Of particular note is
8.0
the slight negative correlation
V+LA+LV
between the LA+LV versus V plots
7.0
(one goes down where the other
goes up and vice versa), r= -0.33,
6.0
(p<.07). Given the near constant
value of the V+LA+LV plot, this
5.0
seems like indirect evidence for
LA+LV
recoding going on between the V
4.0
and L modalities. That is, an item
received in say LA gets recoded and
LA
V
3.0
stored in V, thereby lowering the
A
number of V items that can be
2.0
recalled, but apparently increasing
LV
the number of LA items that have
been remembered. There is no such
1.0
systematic variation between the LA
and LV curves, indicating no
0.0
seeming relationship; one might
0
1
2
3
4
5
6
hope for counter-correlation since, by
the memory model used here, they
IM Count
are using a shared resource.
However, the possible occurrence of
recoding into V and/or A may have obscured this relationship. Of note also is that the LA and A modalities are
slightly monotonic in opposing directions (r= -0.9 of the means), perhaps indicating that introduction of the A
- 11 -
Modality Based Working Memory
modality items starting at IM=2 starts to place a slight burden on the auditory systems and lowering its efficiency,
ever so slightly, of passing LA items through to the L modality. This is just conjecture however as there is no
significance to any of these measures (except perhaps the V vs. LV+LA measure above).
Chart 5 – Averages for recalled items in A, H, K, and SA
# of items
Chart 5 shows the plot of the recalled items from the
4.0
other four modalities under study. There is a general
SA
3.5
monotonic increase for all but the H modality, which itself
K
3.0
has a slight puzzling downward trend after IM=3. In
particular, the H, A, and K plots are distinctly similar and
2.5
A
linear at the IM=2 and IM=3 values (varying from 1.5 to
2.0
2.4 recalled items, respectively). Since at IM=2, a
H
1.5
maximum of two items can be recalled (and 3 items at
1.0
IM=3, and so on), no curve can perforce increase at a
slope of faster than one. The very linear relationship in
0.5
the range IM=2 to IM=3 seems to provide additional
0.0
evidence that a linear process is occuring in these
0
1
2
3
4
5
6
modalities. At IM=4 there seem to be distinct nonlinearities introduced into all but the K curve, indicative
IM Count
perhaps some sort of internal effects, such as
inteference or capacity limitations, starting to occur
within those modalities. Of further interest might be the fact that virtually none of these curves correlate with each
other on a within-subject basis (the highest correlation is between SA and K on a subject-by-subject basis, r=0.46,
(p<.05)). This means that a subject performing well or poorly in a given modality seems to have no bearing on how
the same subject does in other modalities for a given trial run.
Recency and Primacy Curves
Chart 6 – Probability of recall of V, LV, and LA as a
function of item’s original presentation position
Probability of Recall
Chart 6 shows the
probability of an item’s
0.9
V
being recalled as a
0.8
function of where the item
LA
0.7
occurred within the
V+LV+LA
presentation sequence
0.6
LA+LV
within its own modality.
0.5
The data across trial runs
LV
and across modalities had
0.4
to be normalized relative
0.3
to each other to
0.2
compensate for the fact
that the number of items
0.1
presented in any given
0
modality varied
considerably depending
upon the trial run and the
modality. For example, in
Serial Position of Recall (normalized for trial run length)
trial run T9, there were
seven LA and seven LV
items, six V items, and four for each of the other modalities. This meant that each modality had a varying number
of items presented which differed across trial runs. Calculating the probability that the ith item presented for
modality m, required normalizing the length of all modality sequences. See Appendix G for details of how the
normalization was performed and the primacy/recency curves were calculated.
0.95
0.85
0.75
0.65
0.55
0.45
0.35
0.25
0.15
0.05
As to be expected, Chart 6 shows fairly solid recency and primacy effects for virtually all combinations of the V, LV,
and LA modalities, one of the marks of short term memory. Note how the LA+LV plot shows that the LV and LA
curves tend to cancel out their individual swings, especially in the somewhat wild LA swings of the latter third of the
- 12 -
Modality Based Working Memory
curve. Such behavior seems indicative of the existence a single underlying shared resource that they are both
making use of.
Chart 7 – Probability of recall of A, H, K,
and SA for trial runs T9 and T10
Probability of Recall
Chart 7 shows the
probability of an item in
Total = combined K, H, A, & SA
the A, H, K, or SA
1.2
modalities being correctly
recalled, depending upon
1
the order it was presented
K
to a subject. This chart is
0.8
organized identically to
H
Chart 6, except that it only
0.6
Total
shows data for trial runs
A
where each modality had at
0.4
least four items (trial runs
SA
T9 and T10). The data was
0.2
restricted to these trial runs
because the other ones had
0
modality sequences of only
two or three items per
modality. Such sequences
are just too short to show
Serial Position of Recall (normalized to trial run length)
much in the way of recency
or primacy effect, especially
considering the small number of subjects. (As it is, because the recall sequences are so short, there is noticeable
quantization effects in the chart from the normalizing operation.)
95%
85%
75%
65%
55%
45%
35%
25%
15%
5%
In Chart 7, only the H curve really shows much of the normal primacy/recency curve. The other modalities seem to
have no readily discernable overall pattern. Notable is the near 100% recall of the initial sound (in either A or SA)
and similarly the near 100% recall of the last kinesthetic action performed across all subjects and all trial runs.
There certainly generally seems to be a primacy effect (except for K). The lack of recency may be due to the small
number of items presented in each modality and due to the relatively long time from the end of each sequence until
recall actually started. Each sequence was intermixed with each other and with the V, LV, and LA sequences. In
addition there was the fill sequence (in V, LV, and LA) which further delayed getting to the recall of these four
modalities. So much delay may have limited the recency effect. Additionally, the sudden dip of the SA curve at the
end may be due to a coding artifact from the last SA item in trial run T9; this stimulus seemed to have been
confounded with another stimulus in that trial run. This may have artificially lowered the end of the SA curve. It
would also explain the unexpectedly flat response at IM=4 for the SA curve in Chart 5.
Discussion
The modality cognition model (Figure 1) was used to make a number of predictions and which were tested in this
experiment.
Total Working Memory Capacity
As predicted, the total working memory capacity seemed to be increased by using modal specific stimuli. Chart 1
shows subjects recalled about 7 to 8 items in the V, LV, and LA modalities. Chart 2 shows subjects were able to
reliably recall some eight to 15 additional items by use of additional modalities. Similarly organized free recall
experiments typically report recall lengths of 5 to 10 items for arbitrarily length lists of unordered stimuli lists
presented in either V or A (Miller, 1956). In this experiment, the 15-23 items recalled by subjects is some 250% to
300% higher than those other reported rates, and similarly higher than the base rate of some 7.5 words for the
V+LV+LA levels recorded in this study. The much higher recall rate in this study certainly seems indicative of some
additional memory aid being employed.
- 13 -
Modality Based Working Memory
Other techniques, such as categorizing the stimuli, have been used to increase recall rates. If free recall lists are
organized into categories and presented with items blocked together by category, then the recall rate seems to be
improved by approximately 15% to 70%, depending upon the study (Dallett, 1964; Cofer, Bruce, & Reicher, 1966).
This is still well short of what was found here (which had random presentation of stimuli).
As predicted, and as illustrated by Charts 1 through 3, there appears to be a linear increase in recalled items as IM
is increased.
As for the predicted leveling off of working memory capacity as IM continues to increase, the discontinuity is
unfortunate in Chart 3 between the Series A and Series B data at the IM=3. Both the subjects and the organization
of the trial runs changed right at this juncture when switching from Series A to Series B. This makes it difficult to
discern whether the apparent leveling off is due to the increasing value of IM, the use of a different set of subjects,
or the differing set of cards used in the trial runs. One reasonable interpretation of difference in slopes of the two
series is indeed that Series B represents a leveling off of the linear increase shown by the Series A trial runs.
Unfortunately, the same data can also be reasonably interpreted as a noisy linear increase all the way form IM=2 to
IM=5 (see Chart 1). As such, it remains for future work to establish if the predicted leveling off of total working
memory capacity does indeed occur as expected.
Seven Plus or Minus Two
The sum of the V+LV+LA values across all trial runs show a very consistent average of seven to eight items being
recalled from these three presentation modalities. Indeed, only two out of 33 trial runs with subjects had any
V+LV+LA score higher than nine. The strong consistency reconfirms the seeming limits of these three modalities
operating together. The items recalled above and beyond these three modalities are strongly indicative that some
other memory mechanism is operating in addition to the usual visual/verbal one as tested by traditional free recall
tasks.
Primacy / Recency Effects
There is a clear primacy / recency effect for the V, LV, and LA modalities as expected and as shown in Chart 6.
Chart 7 is not nearly so clear with the A, H, K, and SA modalities. At best there either seems to be some evidence
(the primacy effect seems strong, H shows clear recency, and the sum of the curves shows primacy/recency) or
there are explanations for lack of a clearly discernable effect. It seems more data needs to be collected to resolve
this. However, it seems like there is reason to cautiously expect that this effect exists, especially considering the
small numbers of trials and of subjects in this study.
Other Evidence
Other evidence for the modality cognition model, mentioned earlier, can be found on Charts 4 and 7. On Chart 4,
the countervailing swings of the V and LV+LA curves indicate either a shared resource or significant crossencoding and cross-storage is occurring. Similarly, on Chart 7, the countervailing swings of the LV and LA curves
indicate a shared resource constraining their individual capacities. This shared resource has often been referred to
as verbal memory (Baddeley, 2000) or phonological memory (Penney, 1989). This shared construct is here
characterized as a linguistic representational system and which can readily take its input (with literate individuals)
from either V or A. Note that this linguistic model also nicely integrates with other linguistic inputs such as with
Braille or American Sign Language (KL and VL).
There is other evidence developed during the course of the study, but not reported in detail due to lack of space.
Independently of presentation order, subjects frequently clustered stimuli from the same modality into sequences
during recall. According to Penney (1989), when subjects are presented stimuli in varying modalities and which are
also organized by category, during recall they have a strong preference for clustering items via modality as
opposed to category. This certainly indicates that modality is a stronger associative bond than category seems to
be, and that perhaps modality association occurs because it is a deeper underlying mechanism than category. The
fMRI evidence to date seems to indicate this with the demonstration that the category of “living things” is encoded
across several modalities (Schill-Thompson, et al., 1999).
- 14 -
Modality Based Working Memory
It has been suggested that the modality enhancement effect found here is nothing other than a fancy form of
categorization (Greeno, 2001). Appropriate categorization can improve free recall by 50% to 70% (Dallett, 1964),
but this is only a fraction of the improvement that using multiple modalities seems to have offered here.
Bower, et al. (1969) showed a 150% to 350% improvement on recall when stimuli were carefully categorized and
hierarchically organized, as opposed to the same items being presented in random order. Bower presented all
items at once (via a printed card) in a visually, spatially, and semantically organized hierarchy and gave subjects
approximately four minutes to study the hierarchy (of up to 112 items divided into some 30-40 categories). Test
subjects had a visual display whose items were carefully associated as to semantic content and spatial grouping so
as to make conceptual sense, whereas controls had items randomly organized into the same spatial groupings.
Additionally, each subject had four opportunities to see the display and write their recall. The present study of
course employed an entirely different presentation technique. However, it is notable that random presentation
caused a tremendous detriment in Bower, but random organization in the present study still lead to superior recall
performance. One can only conjecture what improvement it would have been to the present study’s subjects to
have had all stimuli grouped by modality during presentation.
Another way of looking at Bower’s results is through the lens of the current modality model. Bower’s subjects were
presented stimuli in V, LV, and SV, and had ample time and cause during the several minute study period to recode
to L and A, and possibly into other modalities. Additionally, the items were grouped into small categories of two to
three items per category. As such, Bower’s results might possibly be reasonably interpreted as comparable to the
current study’s – Bower’s subjects were simultaneous using multiple modalities, just as the current study’s subjects
did, and achieved similar results, and in additon had the benefit of semantic categorization. It is also clear, both in
the case of Bower, the present, and other category-related studies (Dallet, 1964; Pollio, Richards, and Lucas,
1969), that organizing stimuli and hierarchically grouping them is of enormous benefit in recall.
Other studies have demonstrated greatly enhanced recall capabilities and/or recency effects via categorization or
similar organization, but these too can be interpreted in terms of utilizing modalities beyond the V, LV, and LA of
traditional free recall tasks. Watkins & Peynircioglu (1983) used six categories (riddles, sounds, objects, favorites,
quiz questions, and drawings) and which were run in two groupings of riddle-sound-objects and favorites-quizdrawing. The first grouping can easily be interpreted as presentations in LV-A-V and the second group as E-L-K/V.
They showed subjects were able to recall about five items from each of three categories; this is a performance
comparable to the present study’s. As a counter-argument, studies which try to use purely taxonomic categories
(except for Dallet (1964)), tend to show little benefit from categories (Cofer, Bruce, Reicher, 1966).
The net result is that the modality utilization of the current study could be interpreted as a form of categorization.
However, categorization only seems to succeed where multiple modalities are employed. This makes it seem more
likely that successful categorization used to enhance recall should be interpreted as a special case of use of
multiple modalities.
Conclusion
A number of working memory phenomena were consistently reproduced in this experiment (the various well-known
effects such as fixed capacity, primacy, and recency) with other phenomena occurring on top of these (greatly
increased working memory capacity for one). Various sources of evidence seem supportive of the modality
cognition model proposed here. While these modality effects can be explained as a form of categorization, it
seems as likely, or possibly even more likely, that categorization effects can be explained as a form of modal crossencoding. In particular support of this view point is the recent fMRI evidence of the modal organization of working
memory and likely cross-modal encoding of categories.
In conclusion, this study has presented a model for how certain aspects of relatively low-level human cognition
occurs via a number of distinct modal loci processing centers ordered into at least two layers. Evidence was
developed to show support for a linguistic and spatial modalities as well as for the sensory based ones. In fact, one
reasonable interpretation of the present study’s data is that each of the modalities seem to have a working memory
capacity of about three items in a free recall task. If this interpretation is valid, then Miller’s famous seven plus or
minus two dictum might really be something more akin to say seven = 3+2+2 (V+LV+LA).
- 15 -
Modality Based Working Memory
References
Baddeley, Alan D. & Hitch, Graham (1993). The recency effect: Implict learning with explicit retrieval?. Memory
and Cognition, 21(2), 146-155.
Baddeley, Alan (2000). Short-Term and Working Memory. In E. Tulving & F. I. M Craik (Eds.) (eds.) Oxford
Handbook of Working Memory. Oxford : NY.
Bower, Gordon H., Clark, Michar C., Lesgold, ALan M., and Winzenz, David (1969). Hierarchical retrieval schemes
in recal of categorized word lists. Journal of Verbal Learning & Verbal Behavior, 8(3), 323-343.
Bower, Gordon H. (2000). A Brief History of Memory Research. In E. Tulving and F. I. M. Craik (eds.) (eds.) The
Oxford Handbook of Memory. Oxford : NY.
Cofer, C. N., Bruce, D. R., and Reicher, G. M. (1966). Clustering in free recall as a function olf certain
methodological variables. Journal of Experimental Psychology, 71, 858-866.
Crowder, Robert G. (1993). Short-term memory: Where do we stand?. Memory and Cognition, 21, 142-146.
Dallett, Kent M. (1964). Number of categories and category information in free recall. Journal of Experimental
Psychology, 68(1), 1-12.
Dehaene, S., Spelke, E., Pinel, P., Stanescu, R., Tsivkin, S. (1999). Sources of Mathematical Thinking: Behavioral
and Brain-Imaging Evidence. Science, 284, 970-974.
Duis, Sandra S., Dean, Raymond S., Derks, Peter (1994). The modality effect: A result of methodology?.
Elman, Jeffrey L. (1990). Finding structure in time. Cognitive Science, 14, 179-211.
Goldstone, Robert L., Barsalou, Lawrence W. (1998). Reuniting perception and conception. Cognition, 65, 231262.
Greeno, 2001. Personal conversation.
Groeger, John A., Field, David, Hammond, Sean M. (1999). Measuring memory span. Quebeck 98 Conference
on Short-Term Memory (Jun: Quebec City, PQ, Canada).
James, William (1890). The principles of Psychology. Dover Publications: NY.
McKone, Elinor & Dennis, Christopher (2000). Short-term implicit memory: Visual, auditory, and corss-modality
priming. Psychonomic Bulletin & Review, , 341-346.
Miller, George A. (1956). The magical number seven, plus or minust two: Some limits on our capacity for
processing information. Psychological Review, 63(2), 81-97.
Nairne, James S. (1990). A feature model of immediate memory. Memory and Cognition, 18(3), 251-269.
Nersessian, Nancy J. (unpublished). Abstraction via Generic Modeling in Concept Formation in Science.
Penney, Catherine (1989). Modality effects and the structure of short-term verbal memory. Memory and Cognition,
17(4), 398-422.
Pollio, Howard R., Richards,Steven, and Lucas, Richard (1969). Temporal properties of category recall. Journal of
Verbal Learning & Verbal Behavior, 8, 529-536.
Ratcliff, Roger & McKoon, Gail (2000). Memory Models. In Endel Tulving & Fergus I. M. Craik (eds.) (eds.) The
Oxford Handbook of Memory. Oxford : NY.
Schill-Thompson, S. L., Aquirre, G. K., D'Esposito, M. & Farah, M. J. (1999). A neural basis for category and
modality specificity of semantic knowledge. Neuropchologia, 37, 671-676.
Schurman, D. L., Bernstein, Ira H., Proctor, Robert W. (1973). Modality-specific short-term storage for pressure.
Bulletin of the Psychonomic Society, 1(1B), 70-75.
Schweickert, Richard (1993). A multinominal processing tree model for degradation and deintegration in immediate
recall. Memory and Cognition, 21, 168-175.
Shiiffrin, Richard M. (1993). Short-term memory: A brief commentary. Memory and Cognition, 21, 193-197.
Watkins, M. J. & Peynircioglu, Z. F. (1983). Three recency effects at the same time. Journal of Verbal Learning &
Verbal Behavior, 22, 375-384.
Weiskrantz, L. (1987). Neuroanatomy of memory and amnesia: A case for multiple memory systems.
White, Theresa L. & Treisman, Michel (1997). A comparison of the encode of content and order in olfactory
memory and in memory for visually presented verbal materials.
- 16 -
Appendices
Apendices
Appendix A – List of All Stimuli
This appendix lists the stimuli presented to subjects. Stimuli were divided among the seven modalities
Trials 1 & 2
Circle where you were contacted
400
402
401
403
406
133
413
404
406
102
134
103
405
406
106
101
105
104
409
411
412
front
rear
109
108
131
123
Circle what you were told to move.
130
107
122
121
Trials 3 & 4
621
622
625
626
125
126
623
124
118
624
129
128
127
111
627
112
113
116
114
628
front
132
115
119
Circle the direction of any click-clacks you heard.
117
564
front
551
563
557
553 552 554
561
558
rear
501
502
503
504
505
506
507
508
509
above
559
560
552
562
556
555
under study. Each stimuli had an associated numeric code which is listed in this appendix. The code is used
elsewhere to identify each stimulus in the analysis and in other appendices. Stimuli which have been struck-thru
were eliminated because of their possible redundancy or priming effect on other stimuli. Stimuli in the V, LV, and
Appendices
LA modalities were randomly divided between two equal sized lists. Any given trial run was constructed only from
one or the other of the two lists as a means to see if there was any differential effect in how items in these
modalities were recalled (i.e., to see if there was a non-random preference for V, LV, LA items across trials – no
such preference showed up, validating some level of homogeneity of stimuli within these modalities).
Words
Trials 1 & 2
701
703
705
707
709
201
713
715
203
717
719
721
723
725
205
207
apple
ball
blanket
bottle
bread
brunch
candle
cheese
cloud
coat
couch
desk
egg
fish
fork
goose
209
211
731
733
735
213
737
739
741
745
747
215
749
217
751
219
755
hat
ink
juice
leaf
lunch
moon
night
page
pants
picture
pliers
pole
pond
river
rock
rug
salad
757
221
759
761
223
225
753
227
743
229
729
727
231
233
711
235
237
Noises
scarf
seed
shark
shirt
shovel
stairs
stamp
storm
street
string
tea
tissue
towel
tree
water
whale
window
Trials 1 & 2
98
72
74
76
80
70
88
86
96
Trials 3 & 4
Trials 3 & 4
702
704
706
708
710
202
714
716
718
720
204
726
206
728
208
210
bag
bed
bookcase
box
breakfast
brush
carrot
clock
coffee
cup
fence
flower
frog
gate
grass
house
212
732
734
736
738
214
742
744
746
216
218
220
756
222
760
754
knife
lake
letter
milk
ocean
pan
paper
pencil
pillow
rain
roof
ruler
sandwich
shelf
soap
socks
758
224
752
750
226
748
228
740
730
230
724
232
722
234
236
238
broken glass
buzzer
car crash
cash register
cow moo
frog
phone ring
piano / music note
wolf
soda
spoon
stapler
stars
stick
store
stove
sun
table
tissue
toothbrush
train
tree
turtle
wheel
wine
- 18 -
99
71
75
77
79
81
83
87
85
89
93
95
bubbling
burp
car horn
cat
clapping
dog bark
drum roll
horse neigh
laughter
police siren
trumpet
wind
Appendices
Stimulus Codes
Visual 1
(V)
109
130
131
103
134
108
102
133
101
106
122
105
104
123
107
121
Visual 2
(V)
book
camera
car
chicken
cow
dice
duck
horse
pickup truck
pig
seat belt
sheep
tractor
truck
TV
wrench
115 bicycle
117 bird
124 boot
126 castle
116 chair
118 cigar
113 comb
129 eye glasses
127 hammer
111 hand
114 lamp
119 piano
112 scissors
125 teepee
132 turtle
128 whistle
Auditory 1
(A)
98
72
74
76
80
70
88
86
96
Auditory 2
(A)
broken glass
buzzer
car crash
cash register
cow moo
frog
phone ring
piano / music note
wolf
99 bubbling
71 burp
75 car horn
77 cat
79 clapping
81 dog bark
83 drum roll
87 horse neigh
85 laughter
89 police siren
93 trumpet
95 wind
LinguisticVisual 1 (LV)
LinguisticVisual 2 (LV)
201
203
205
207
209
211
213
215
217
202
204
206
208
210
212
214
216
218
brunch
cloud
fork
goose
hat
ink
moon
pole
river
brush
fence
frog
grass
house
knife
pan
rain
roof
LinguisticAuditory 1
(LA)
1 apple
3 ball
5 blanket
7 bottle
9 bread
13 candle
15 cheese
17 coat
19 couch
LinguisticAuditory 2
(LA)
2 bag
4 bed
6 bookcase
8 box
10 breakfast
14 carrot
16 clock
18 coffee
- 19 -
Appendices
219
221
223
225
227
229
231
233
235
237
rug
seed
shovel
stairs
storm
string
towel
tree
whale
window
Haptic (H)
encoding
220
222
224
226
228
230
232
234
236
238
ruler
shelf
spoon
stick
stove
tissue
train
turtle
wheel
wine
Haptic (H)
(stimuli list)
21
23
25
31
33
35
37
39
41
45
47
49
51
55
57
59
61
53
43
29
27
11
409 hand
405 upper back
411 thigh
412 knee
413 middle back
20
26
28
32
34
36
38
42
44
46
56
60
54
58
52
50
48
40
30
24
22
cup
flower
gate
lake
letter
milk
ocean
paper
pencil
pillow
sandwich
soap
socks
soda
stapler
stars
store
sun
table
toothbrush
tree
SA Spatial-Auditory (SA)
enc
(stimuli list)
odin
g
400 top of head
401 back of
head
402 side of head
403 back of neck
404 shoulder
405 upper back
406 406 upper arm
406 407 elbow
406 408 lower arm
desk
egg
fish
juice
leaf
lunch
night
page
pants
picture
pliers
pond
rock
salad
scarf
shark
shirt
stamp
street
tea
tissue
water
557
558
557
558
557
551 immed. above
head
552 immed. behind
head
553 near L. ear
554 near R. ear
555 45 to rear L.
556 45 to rear R.
557 to L.
558 to R.
559 forward upper L.
558 560 forward upper R.
561 behind back
561 562 behind near floor
563 forehead
551 564 far above head
Kinesthetic (K)
(stimuli list)
Kinesthetic
(K)
encoding
601 raise elbows
623
602 twist legs
627
head 621
603
604
605
606
607
608
609
622
623
624
628
627
625
621
shoulders 622
arms 623
hands 624
torso 625
stomach 626
legs 627
feet/ankles 628
610
611
612
613
614
rotate shoulders
raise arms
flex hands
flex ankles
move legs
twist torso
head forward &
back
rotate head
tighten stomach
tighten arms
tighten legs
put arms
backwards
621
626
623
627
623
The following stimuli were also originally included in the study to test for a spatial-visual (SV) modality. They had to
be eliminated when it became apparent during piloting that the design of the stimuli were faulty (some Ss were
using a mnemonic device of representing all presented SV stimuli into the pattern of a tic-tac-toe board rather than
recalling them as separate stimuli).
Spatial-Visual
- 20 -
Appendices
(SV)
501 upper L.
502 upper
middle
503 upper R.
504 middle L.
505 center
506 center R.
507 lower L.
508 lower
middle
509 lower R.
511 upper L.
512 upper
middle
513 upper R.
514 middle L.
515 center
516 center R.
517 lower L.
518 lower
middle
519 lower R.
- 21 -
Appendices
Appendix B –Organization of Trial Runs
Organization of Trial Runs
The exact ordering of stimuli in each trial run is given below. The codes listed with each trial run identify stimuli and
are defined in Appendix A.
The trial runs were administered in two separate series. Subjects in a given series received exactly the same
sequence of trial runs. The series were organized as follows:
Series Number
Number of Subjects
Trial Run Sequences
A
4
T0, T1, T2, T3, T6, T7
B
6
T8, T9, T10
Series A validated various aspect of the design as well as providing extensive data for testing against two and three
stimuli in each modality. Additionally it validated that the eight item fill sequence length was adequate by using
sixteen fill items for trial run T7 (which otherwise was set up identically in form to trial T2). T7 and T2 showed no
differences in the pattern of fill item responses. Series B tested the major premise of the experiment by testing
against three, four, and five stimuli items in each modality.
The stimuli sequences for trial runs for T0 through T7 are given below:
T0
T1
(List 1)
1
2
3
4
5
6
7
8
9
10
235
49
106
1
53
39
213
123
233
219
11
45
T2
(2seq)
(List
1)
(List 2)
1
2
3
4
5
6
7
8
9
10
44
228
2
36
118
18
230
127
236
113
11 234
12 217
12
48
13 107
13 111
14
9
14 224
15
11
15
38
16 201
16
56
1
2
3
4
5
6
7
8
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
402
3
96
237
108
623
554
628
101
561
T3
(3seq)
(List
1)
1
2
3
4
5
6
7
8
9
10
80
104
551
76
102
31
623
552
407
203
74
11 626
205
12 405
41
13
15
409
14
98
47
15 121
5
16 560
122
17 627
T6
(2seq)
(List
2)
1
2
3
4
5
6
7
8
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
T7
(3seq)
(List
2)
89
556
405
202
623
232
406
553
119
87
1
2
3
4
5
6
7
8
9
10
71
24
403
562
621
75
212
220
126
8
625
11 623
208
12 129
404
13 557
34
14 401
16
15
50
114
16 238
115
17 125
- 22 -
Appendices
1
8
1
9
2
0
2
1
2
2
double lines
under
items indicate
end
of test
sequence and
start of the fill
sequence
221
18
229
19 412
231
20 215
43
21 225
105
22
25
61
23 109
24 209
25 227
26 103
27 211
28
55
29
29
1
8
1
9
2
0
2
1
2
2
2
3
2
4
2
5
2
6
2
7
2
8
2
9
564
95
18 222
19
621
20 116
27
21 206
218
22
124
23 210
22
24 117
25
214
26 216
112
27 204
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
402
3
96
237
108
623
554
628
101
561
72
205
41
409
223
88
116
627
59
555
401
207
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
40
14
28
30
29 128
The stimuli sequences for trial runs T8 through T10 are given below:
T8
T9
T10
(3(4(5seq)
seq)
seq)
(List
(List
(List
1)
1)
2)
225
80
104
551
102
31
76
623
552
407
203
626
405
15
83
121
560
627
25
412
215
61
32
226
30
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
10
218
89
556
405
202
615
411
553
232
119
87
625
208
406
34
16
114
115
564
79
621
27
- 23 -
20
4
Appendices
23
24
25
26
27
28
29
109
209
227
103
211
55
29
23
24
25
26
27
28
29
30
31
32
33
34
35
36
81
7
624
563
112
406
47
5
122
221
229
231
43
105
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
71
24
403
562
627
75
212
220
126
8
614
129
563
400
210
117
40
216
204
20
128
4
- 24 -
Appendices
Appendix C – Procedures
This appendix describes the procedures followed to setup and perform this experiment. The appendix first
describes the procedure used to create the various trial runs and used to minimize any preference bias for
individual stimuli of stimuli sequences followed by a detailed description of the procedures used to perform the
experiment and gather data.
Procedures Used to Set Up Trial Runs
Since this was not a study about priming, and to minimize stimulus preference because of redundant presentations,
stimuli for each modality were carefully screened to avoid redundancies within and across modalities. For
example, if a picture of a pig existed in the V modality, the word “pig” could not also be used in either LV or LA.
Similarly, each of the several hundred stimuli were checked to minimize the likelihood of redundancy, priming, and
obvious association. It was not possible to eliminate all obvious associations (for example, “rain” and “cloud” in LA,
and “moon” in LA vs. “sun” in LV). However, various obvious associations were screened out such as a dog
barking in A and a picture of a dog in V. Additionally, stimuli were only presented once to each subject.
Method for Creation of Trial Runs
Stimuli were assembled into sequences called trial runs. Each trial run (numbered in the order that they were
created – T0, T1, T2, etc.) had the same number of stimuli from each modality and then had a fill sequence at the
end in the V, LA, and LV modalities. Each stimulus was presented only once to any given subject. Trial runs T0
and T1 consisted only of V, LA, and LV modalities and were used to demonstrate that this system could reproduce
the results of traditional free recall experiments (i.e., “seven plus or minus two”). The table below shows the
number of cards of each type in each trial run. Note that total sequence length equals seven times the middle
column plus the length of the fill sequence. (The following table is reproduced from the body of the text for
convenience.)
Table 1 – Organization of the Trial Runs
Number of cards in each of the
various modalities
Trial
Run
Total
Seq.
Length
V, LA, LV,
H, A, K, SA
T0
16
T1
Fill Sequence
(V / LA / LV)
Comments
-
16
(3/6/7)
Calibration sequence (see text)
16
-
16
(4/5/7)
Calibration sequence (see text)
T2
22
2
8
(2/3/3)
T3
29
3
8
(2/3/3)
T6
29
3
8
(2/3/3)
T7
30
2
16
(4/6/6)
T8
29
3
8
(2/3/3)
T9
36
4
8
(2/3/3)
T10
44
5
8
(2/3/3)
Each stimuli was initially written on its own separate card in the manner indicated above and grouped by modality
(i.e. all the haptic cards together, all the visual ones together, etc.). The cards in each modality were randomly
sorted (using 52-card pick-up style sorting – i.e., tossed into the air and allowed to fall to the floor and reassembled
into a single pile). Trial runs were created by drawing a suitable number of cards from each modality to create the
trial run. The order of the cards in each trial run was randomized (using the toss-up method). The following
conditions were checked for in each trial run and manual adjustments were made to eliminate them (pilot testing
indicated these caused very biased responses during the recall task):
- 25 -
Appendices
-
Cards in the H, A, K, or SA modalities that were adjacent or separated by only one card from each other;
-
Three cards in a row from the same modality;
-
Two or more adjacent cards with obvious associations (i.e. “sun” and “moon” or “rain” and “fall” even if in
differing modalities).
The typical adjustment was to swap one of the offending cards with another card a bit further down in the sequence
to eliminate the problem.
[NOTE: A better system for trial run creation would have been to create the modality piles, and then use psuedorandom selection from each pile, skipping a random number if one of the above conditions would occur. The tossup system did not sufficiently randomize cards and it became apparent that there were some unintentional
sequence biases which had to be corrected during pilot testing. The above rules arose because of these piloting
observations.]
Experimental Procedures
Subjects were presented with several trial runs in sequence. Each trial run consisted of a presentation of the trial’s
stimuli sequence, and without interruption, immediately followed by the subject’s attempt to recall as many items
out of the presentation as possible. The experimenter wrote down items in the order of recall as the subject
recalled items. A cassette recorder was also used as a back-up and to double check that each subject’s recalled
list was accurately recorded. Appendix C contains a detailed description of the exact procedures followed during a
session.
Session Introduction
At the start of a session, the experimenter explained the following:
“This is a study of short term memory to see how well people can recall things. I will present you with a
sequence of items and when we are done with the sequence I want you to try to recall as many of them
as you can. I will write them down and also am using the cassette recorder as a back up to make sure I
don’t miss anything. The tape will be erased after double checking your answers. Some of the items will
be words or pictures I’ll show you and some will be words which I will speak. Other items will be clicks I’ll
make in various directions around you [demonstrate use of staple remover to make a distinct clicking
noise] and you will need to remember the direction that you heard the clicks. I will also lightly touch you
in various places with a pencil [demonstrate the touching]. You will also hear various sounds which
aren’t words such as ‘screeching tires’ and which you should be able to readily recognize. Lastly, I will
also ask you to move various parts of your body by saying something like ‘Please roll your shoulders” or
some such. Please do concentrate and try to recall as many items as you can.”
Directions were then given as to how the subject should position themselves. The experimenter and subject sat
adjacent to each other at a table, with the experimenter always to the subject’s left (i.e., all stimuli were
administered from the left side). Subjects cupped both their hands around their eyes to eliminate peripheral vision
and so that they could only see the area of the table directly in front of them. This cupped hand technique created
a field of vision of about 60 degrees (i.e., 30 degrees to either side of the center line). The cupped hand technique
allowed a very consistent presentation of stimuli without the subject receiving undue visual clues (such as seeing
the experimenter reaching around behind them to touch them or create the SA clicks in the appropriate directions).
Performing a Trial Run
A measured consistent pace was attempted so that all items were presented at a comparable rate and comparable
length of time. Visual items were held in front of the subject in their field of view for approximately three seconds or
until the subject indicated that they were ready to go on. Similarly, other modality items were presented as given in
Table 1.
- 26 -
Appendices
The experimenter took great care to insure that no noise or other unintended sensory stimulus occurred that might
clue a subject as to anything that was about to happen. As verification of this, subjects consistently responded in
post-experimental debriefing that they never knew exactly when a stimuli was about to be administered (indeed
several subjects visibly startled the first time they were touched, an SA click was made, or a sound effect occurred).
It typically took just under two minutes to present a thirty item list, giving about four seconds for presentation of
each stimulus. Longer and shorter list presentation times varied in proportion to the list length. The recall sequence
also typically lasted just under two minutes before each subject gave up trying to recall further items. Allowing for
switching of card decks, stretching, and the like, each trial run lasted about five minutes. Each session typically
lasted 20 to 30 minutes, depending on how many trial runs a subject was put through and how long the debriefing
discussion ran.
All subjects in a given series were given the same sequence of trial runs, with each trial run always consisting of
the same stimuli sequence. Because subjects were unpaid volunteers participating as a favor to the author, no
familiarization runs were performed so as to minimize the required time commitment. Because of this, with about
half of the subjects it was necessary on the first trial run to prompt them to recall all modalities (they had left out one
or another modality in these cases). Performance did not seem to be particularly affected when subjects were
prompted (though no formal analysis was performed to verify this).
Post-session Debriefing
All subjects were asked after the completion of each session if they had used any special technique or had noticed
anything that helped or hindered them in trying to do the recall task. These were noted down along with any other
comments the subject provided. The fundamental purpose of the experiment was then explained and was usually
followed by some sort of free form discussion about memory and cognition. Subjects were then thanked for their
participation.
Appendix D – Organization and Administration of the Stimuli
This appendix describes how the stimuli were organized and administered to subjects.
Modality
Visual (V)
Auditory (A)
Linguistic-Visual (LV)
Linguistic-Auditory (LA)
Haptic (H)
Kinesthetic (K)
Stimuli Description and Method of Application
Stimuli were easily recognizable green colored line drawings of
common objects all in a similar drawing style. They were all garnered
from a single clip art package. The images were centered on the
card, and were of similar physical size (approximately 2 x 3 cm).
Stimuli were easily recognizable sound effects which varied in
duration from 1.5 to approximately 2 seconds. They were gathered
from a single CD of clip art sounds.
Stimuli were common words, mostly of one syllable, printed from a
laser printer on a card and which were shown to subjects as the
stimulus. Each word was centered on its card in a 36 point serif font
(approximately 1 cm high letters). 1
Stimuli were single words spoken by the experimenter. The words
were handwritten on a card in quote characters to signify that the
stimulus was to be spoken. 1
Stimuli were listed on a card indicating the area to be touched (i.e.,
“L. knee” or “L. shoulder”) and the subject was lightly rubbed with the
eraser end of a pencil for approximately one second. All touches
were either in the center or to the left side of the body so as not to
confuse subjects with left-right distinctions.
Stimuli were requests to move some part of the body (i.e. “flex your
1
The words from a single list, generated for this experiment, were randomly assigned to either the LV or
LA condition in order to create the LV and LA word lists.
- 27 -
Appendices
Spatial Auditory (SA)
ankles”). Stimuli were administered via the experimenter speaking a
request (“please flex your ankles”). Stimuli were selected so as to
avoid left-right distinctions so as not to confuse subjects with this
subtlety (subjects were always asked to move both legs, both arms,
etc.).
The direction of a stimulus was written on each card (i.e. “immediately
behind head”). Stimuli were administered by clicking a staple
remover twice in the appropriate direction. For this modality, left-right
directional stimuli were used.
Table 1 – How stimuli were presented
The V and A stimuli were tested for ease of recognition with two pilot subjects. A stimulus was replaced if either
pilot subject had any difficulty recognizing it. None of the other modality stimuli were piloted prior to the experiment
(see the codeing and analysis section for discussion of some confusions that may have been created as a result of
confusing stimuli among some subjects).
- 28 -
Appendices
Appendix E – Tabulated Recall Data
Following is the tabulated data from the experiment.
Tot Tot V+L
Tot
V+L #AL V
LV LA
A
H
K
SA A+H+K+S
Corr
+VL
A
Min
100 200
0
69 399 599 550
Max
140 250
69 100 430 650 599
2/3/2001
16
T0
T1
16
MS0
7
7
5
2
3
2
HG0
7
7
4
3
2
2
JK0
7
7
6
1
2
4
Rita0
MS1
8
8
8
8
5
5
3
3
2
2
3
3
HG1
9
9
7
2
3
4
JK1
7
7
5
7.6
7.6
5.3
2
16
2.3
1
15
2.1
4
22
3.1
0.8
0.8
1.0
0.8
0.7
0.9
Mea
n
Std
Dev
T2
T7
T3
T6
22
14 MS2
HG2
JK2
30 MS7
22 HG7
JK7
29
Two items per modality + 8 fill items (V+LV+LA)
12
5
2
3
1
1
2
15
8
4
4
2
2
2
14
7
3
4
1
2
2
12
7
5
2
1
4
0
15
8
6
2
4
2
1
13
8
4
4
1
3
1
19
10
14
8
Mea 13.5
7.2 4.0 3.2 1.7 2.3 1.3
n
Std
1.4
1.2 1.4 1.0 1.2 1.0 0.8
Dev
17
MS3
HG3
JK3
MS6
HG6
JK6
Three items per modality + 8 fill items (V+LV+LA)
17
8
3
5
1
2
2
17
7
4
3
1
3
3
16
9
6
3
2
4
2
16
8
4
4
2
2
2
15
6
4
2
1
3
3
14
5
3
2
1
2
3
19
8
16
15
Mea 15.8
7.2 4.0 3.2 1.3 2.7 2.5
n
- 29 -
V=visual
A=
Auditory
LA =
Linguistic/Auditor
y
LV =
Linguistic/Visual
H = Haptic
K=
Kinesthetic
SA =
Spatial/Auditory
1
2
1
2
2
1
9
1.5
2
2
2
1
2
1
10
1.7
2
1
2
2
2
2
11
1.8
7
7
7
5
7
5
6.3
0.5
0.5
0.4
1.0
3
3
2
2
3
3
16
2.7
3
2
2
2
1
1
11
1.8
1
2
1
2
2
2
10
1.7
9
10
7
8
9
9
8.7
Appendices
Std
Dev
T8
28
17
John8
Lisa8
Rita8
Karen8
MS8
1.2
1.5
1.1
1.2
0.5
0.8
0.5
Three items per modality + 8 fill items (V+LV+LA)
13
5
2
3
2
0
1
20
10
6
4
4
2
1
17
9
6
3
4
2
3
22
11
7
4
3
4
3
19
9
5
4
3
2
2
18
16
10
10
Mea 18.2
8.8 5.2 3.6 3.2 2.0 2.0
n
Std
3.4
2.3 1.9 0.5 0.8 1.4 1.0
Dev
0.5
0.8
0.5
1.0
2
3
2
3
2
12
2.4
3
3
2
3
3
14
2.8
2
3
1
2
3
11
2.2
8
10
8
11
10
9.4
0.5
0.4
0.8
1.3
T3+T7+T8
Mea
n
Std
Dev
T9
T10
36
44
20
John9
Lisa9
Rita9
Karen9
MS9
24
John10
Rita10
Karen1
0
MS10
16.9
7.9
4.5
3.4
2.2
2.4
2.3
2.5
2.3
1.9
2.6
2.0
1.6
0.9
1.2
1.1
0.8
0.5
0.8
0.7
2
1
1
3
3
10
2.0
2
4
3
3
2
14
2.8
1
3
2
2
2
10
2.0
9.4
1.0
0.8
0.7
1.5
Four items per modality + 8 fill items (V+LV+LA)
15
7
7
0
4
3
3
20
9
6
3
5
1
3
15
7
4
3
3
1
2
20
9
6
3
3
3
3
16
7
3
4
0
3
2
13
15
11
13
Mea 17.2
7.8 5.2 2.6 3.0 2.2 2.6
n
Std
2.6
1.1 1.6 1.5 1.9 1.1 0.5
Dev
Five items per modality + 8 fill items
(V+LV+LA)
15
6
4
2
1
20
8
4
4
3
23
8
4
4
2
Mea
n
Std
Dev
21
8
3
19.8
7.5
3.4
1.0
8
11
8
11
9
3
1
2
2
3
5
1
3
2
4
2
4
2
4
4
9
12
15
13
3.8
5
15
3.8
1
7
1.8
2
8
2.0
2
12
3.0
3
9
2.3
4
14
3.5
4
14
3.5
12.25
0.5
1.3
1.0
0.8
1.4
1.0
1.0
1.0
2.5
Steve's data are parked here because they do not belong in the data set but I don't want to completely
delete them
Steve8
10
4
2
2
2
0
1
1
2
2
Steve9
14
7
4
3
3
1
2
3
1
1
Steve10
12
5
2
3
1
1
1
0
2
4
- 30 -
Appendices
MUCH distraction in the room (pets, music, phone), older ~54
- 31 -
Appendices
Appendix F – Raw Recall Data
This appendix contains the actual data from the experiment. The listing is divided into four parts. The first part provides the
tabulation of parts two and three. The second part is the raw data for the V, LA, and LV modalities, while the third part is the
raw data for the A, H, SA, and K modalities. The last part is the list of errors subjects made on any given trial run. The table
has row numbers and column numbers which are common to each row and column to help identify which columns and rows
go together across the parts.
Part 1 - Tabulated Data
Tot Tot V+L
Min
0 10 20 69 39 55 59
0 0
9 0 9
69 14 25 10 43 59 65
0 0 0 0 9 0
Max
1
1 T0
2
3
4
5
6
7
8 T1
9
10
11
12
13
14 T2
15
16
17
18
19
20 T7
21
22
23
24
25
26 T3
27
28
29
30
31
32 T6
33
2
3
16
16
MS0
HG0
JK0
Rita0
16
16
16
16
16
7
7
7
8
16
16
16
16
7
7
7
8
2
2
4
3
2
3
1
3
3
2
2
2
5
4
6
5
16
16
16
8
9
7
16
16
16
8
9
7
3
4
4
3
2
2
2
3
1
5
7
5
22
22
22
12
15
14
14
14
14
5
8
7
1
2
2
3
4
4
1
2
1
2
2
2
1
2
1
2
1
2
2
2
2
2
4
3
30
30
30
12
15
13
22
22
22
7
8
8
4
2
3
2
2
4
1
4
1
0
1
1
2
2
1
2
2
2
1
2
1
5
6
4
29
29
29
17
17
16
17
17
17
8
7
9
2
3
4
5
3
3
1
1
2
2
3
2
3
3
2
1
2
1
3
2
2
3
4
6
16
MS1
HG1
JK1
22
14
MS2
HG2
JK2
30
22
MS7
HG7
JK7
29
17
MS3
HG3
JK3
29
4
5
6
7 8 10 11 9 12 13 14
15
Total Total Total V+L LA V LV A H SA K LA+LV
# of Cor- V+L
Stim. rect
17
- 32 -
Appendices
34
35
36
37
38 T8
39
40
41
42
43
44
45
46
47 T9
48
49
50
51
52
53
54
55
56 T10
57
58
59
60
61
62
MS6
HG6
JK6
28
29
29
29
16
15
14
17
17
17
8
6
5
2
3
2
4
2
2
2
1
1
2
3
3
2
3
3
2
2
2
2
1
1
4
4
3
28
28
28
28
28
28
13
10
20
17
22
19
17
17
17
17
17
17
5
4
10
9
11
9
0
0
2
2
4
2
3
2
4
3
4
4
2
2
4
4
3
3
1
1
1
3
3
2
2
1
3
2
3
2
2
2
3
1
2
3
3
2
3
2
3
3
2
2
6
6
7
5
36
36
36
36
36
36
15
14
20
15
20
16
20
20
20
20
20
20
7
7
9
7
9
7
3
1
1
1
3
3
0
3
3
3
3
4
4
3
5
3
3
0
3
2
3
2
3
2
2
3
1
1
3
3
1
1
3
2
2
2
2
1
4
3
3
2
7
4
6
4
6
3
44
44
44
44
44
15
12
20
23
21
24
24
24
24
24
6
5
8
8
8
3
1
1
2
2
2
3
4
4
5
1
1
3
2
1
2
1
3
5
2
1
0
3
2
3
2
4
4
4
4
4
2
2
4
4
4
2
4
4
3
17
John8
Steve8
Lisa8
Rita8
Karen8
MS8
36
20
John9
Steve9
Lisa9
Rita9
Karen9
MS9
44
24
John10
Steve10
Rita10
Karen10
MS10
- 33 -
Appendices
Part 2 – Raw Subject Data for V, LA, and LV Modalities
(recall order in which subject recalled the given stimuli)
Codes LA V
LV A
H
SA
K
Min
0 10 20 69 39 55
59
0 0
9 0
9
Ma
69 14 25 10 43 59
65
Tot
x
0 0 0 0 9
0
V+L
1
3 16 17 18 19 20 21 22 23 # 25 26 27 28 29 30 31 # 33 34 35 36 37 38 39 40 41
Linguistic-Auditory
Visual
Linguistic-Visual
(LA)
(V)
(LV)
1 T0
2
2
16 49
3
4
5
6
7
8 T1
9
MS0
HG0
JK0
Rita0
10
11
12
13
14 T2
15
MS1
HG1
JK1
16
17
18
19
20 T7
21
MS2
HG2
JK2
22
23
24
25
26 T3
27
MS7
HG7
JK7
28
29
30
31
32 T6
33
MS3
HG3
JK3
34
35
36
37
MS6
HG6
JK6
4 5 6 11 14 15
1 53 39 45 9 11
5
4
3
2
1
16 44
4
7
7
6
4
3 4 6 12 15 16
2 36 18 48 38 56
4
4
5
5
7
14
4
5
6
1
1
4
2
2
2 13 15 16 21
3 41 47 5 43
14
7
1
3
4
2 10 15 19 22 25 28 30
22 24 8 50 10 32 40 20 4
12
4
4
3
5
2
1
5
1
6 13 18 22 28 29
17 31 15 25 61 55 29
9
2
2
2
17
11
1
3
1
14 15 21 24 28 29
17 34 16 27 22 14 30
3
1
1
4
2
2
5
- 34 -
3 8 13
10 12 10
6 3 7
2
3
6 7 1
2
3 8 5
1 7 9 10 12 16
23 21 23 21 21 20
5 3 3 9 7 1
1
6
7
3
2
1
5
1
6
5 8 10 13
11 12 11 11
8 7 3 1
5
6 7
6 9
2 3
2 7 9 11 14
22 23 23 23 22
8 0 6 4 4
8 3
7
8 3
1
5 9 17 22
10 10 12 10
8 1 2 5
2 3
1
8 2 5 1
8 14 6 5
4 12 18 19 20
23 20 22 22 23
7 5 1 9 1
10
6
4
3
9 12 17 20 24 29
12 12 12 11 11 12
6 9 5 6 7 8
3
2
2 8
12
6
5 4
7 8 16 18 21 23 26 27
21 22 23 22 20 21 21 20
2 0 8 2 6 0 6 4
6
15
1 6 3
7
2 5 15 23 26
10 10 12 10 10
4 2 1 9 3
5 17 6 4 3
4
5
1
10 4
3
10 20 21 24 25 27
20 21 22 20 22 21
3 5 5 9 7 1
16
6
12
13
9 16 17 23 27
11 11 11 12 11
9 4 5 4 2
12 11 10
3
6
4
13 12
4 6 12 22 25 26
20 23 20 21 22 21
2 2 8 8 6 4
13
16
5
3
Appendices
38 T8
39
6 14 19 22 28 29
17 31 15 25 61 55 29
40
41
42
43
44
45
46
47 T9
48
John8
Steve8
Lisa8
Rita8
14
Karen8 17
MS8
49
50
51
52
53
54
55
56 T10
57
John9
Steve9
Lisa9
Rita9
Karen9
MS9
58
59
John10
Steve1
0
Rita10 16
Karen1
0
MS10
60
61
62
20
4
8
13
1
1
7
2
2
2 13 19 24 29 30 35
3 41 59 7 47 5 43
4
6
3 5 16 23 26
10 10 12 10 10
4 2 1 9 3
7
8
6
9
2
1 2 19
3
5 3
7
6 4
5 3
17
16 19 18
1 11 21 24 25 27
22 20 21 20 22 21
5 3 5 9 7 1
1 2
1
8
6
5 18 7
1 2
6 8
22
8
18
3 4
20
5 9 17 27 31 36
10 10 11 11 12 10
8 1 6 2 2 5
4 12 15 22 32 33 34
23 20 22 20 22 22 23
7 5 3 7 1 9 1
5
7
2 3
11 2 3
14 16 7
8
15
2 7
6
6
3 20
1
10
9
2
13
15
1
19
18 16
3
2
2
4
11 14
15 16 22 24 32 39 42 44
24 34 16 27 24 8 40 20 4
2
4
1
2
1
4
1
- 35 -
5
12
1
1
14
1
1
10 17 18 31 34 38 43
11 11 11 12 12 11 12
9 4 5 6 9 7 8
5
14
10
11 1
4
2
3
6
5
21
20
6 12
3
17 19 18
3
1 5 9 13 29 30 37 40 41
21 20 23 20 21 22 21 21 20
8 2 2 8 2 0 0 6 4
3
12
5
4
1
5
2
5
15
17
3
Appendices
Part 3 – Raw Subject Data for A, H, SA, and K Modalities
(cell values are recall order in which subject recalled the given stimuli)
Codes LA V
Min
0 10 20
0 0
69 14 25
0 0
Max
Column #'s
1
1 T0
2
3
4
5
6
7
8 T1
9
10
11
12
13
14 T2
15
LV
A
H
SA K
69 39 55 59
9 0 9
10 43 59 65
0 0 9 0
3 43 44 45 46 47 # 49 50 51 52 53 # 55 56 57 58 59
Auditory
Haptic
Spatial-Auditory
(A)
(H)
(SA)
61 62 63 64 65
Kinesthetic
(K)
0
MS0
HG0
JK0
Rita0
0
MS1
HG1
JK1
3 11
0 96 74
16
17
18
19
20 T7
21
MS2
HG2
JK2
22
23
24
25
26 T3
27
MS7
HG7
JK7
28
29
30
31
32 T6
33
MS3
HG3
JK3
34
35
36
37
38 T8
MS6
HG6
JK6
8 9
15 16
2 7
1 6
0 71 75
7
11
1 4 14
0 80 76 98
14 15
11 9 10
15
14
1 10 19
0 89 87 95
1 14
40 40
2 9
15
12 11
9
7 10
55 56
4 1
4 5
13
10 12
6 8
62 62
3 8
6 7
10 9
13 11
3
40
3
9
11
8
14
40
1
10
12
4
56
2
7
13
9
5 11
62 62
1 3
11
9 10
13
9
40
7
12
13
6
12
40
5
13
14
13
55
7
8
14
10
19
41
2
11
12
5
3 8 16
55 55 56
1 2 0
7
16 15
16
7 11 17
62 62 62
3 6 7
10 8 9
8 7
7 8
2 8 18
55 55 56
6 3 4
5 4
14
13
11 10
5 11 20
62 62 62
3 5 1
14 15
15
9
8
8
1
9
7 9
2 14
3 7 13
40 40 40
5 6 4
7 6
12 10 11
7 6 8
2
7 15
10 13 20
- 36 -
4
9 17
8 12 18
Appendices
39
0 80 76 83
40
41
42
43
44
45
46
47 T9
48
John8
13
Steve8 3
Lisa8
20
Rita8
4 11 12
Karen8 14 15 16
MS8
13 14
49
50
51
52
53
54
55
56 T10
57
John9 15 13 14
Steve9 7
8
Lisa9
3 5 4
Rita9
4
10
Karen9 17
14 15
MS9
12
13
58
59
John10 12
Steve1 3
0
Rita10
2
Karen1 7
0
MS10 20
60
61
62
40 40
7 5
12 11
7
12 11
9
11 13
11 12
3 11 16 23
0 96 72 88 81
2 11 20 23 28
89
87 79 71 75
0
13
18 19
8 11 9 10
21
41
2
55 55 56
1 2 0
9
10
6
7
9 17 10
17
9 10
6 7 5
62
3
3
4
14
16
20
9
1 14 21 28
40 40 40 40
2 9 1 6
12
11
5 4
6
9
9
7
8 19
8
9 10
7 10 20 26
55 56 55 56
4 1 5 3
10
12
10 11 12
8
3
9 10
4
5
6 8 18 25
62 62 62 62
3 8 7 4
8 9
13
20 19 18 17
12 11
13
13 12
11
6 7
4 7 14 25 36
40 41 40 40 40
5 1 6 3 0
15
3 8 19 26 35
55 55 56 56 56
6 3 4 2 3
10
11
9 7 6
8
6 12 21 27 33
61 62 62 62 61
5 5 1 7 4
7 6
9 8
4 5
11 12
10
19 20
7 6
18 17
14
16 15
16
10
12
- 37 -
8
7
8 9
15 16
9
6
62 62
6 7
4 5
5
15 13
15
19 21
10 8
14
13
14 23 22 13
10 13 11 12
Appendices
Part 4 – Subject Recall Errors and Other Notes
Codes LA V LV
Min
Max
0 10 20
0 0
69 14 25
0 0
A
H
SA K
69 39 55 59
9 0 9
10 43 59 65
0 0 9 0
Column #'s
1
3 66 67 68 69 70 71 72 73 74 75 76 77 78 79
1 T0
2
3
MS0
4
HG0
5
JK0
6
Rita0
7
8 T1
9
10
MS1
11
11
HG1
9
12
JK1
13
14 T2
15
16
MS2
22 11 10 48 4xx
Ax Ax Didn't
4 8 7
x x understand to
also remember
A,K,H,SA
17
HG2
S didn't know to key for all
modalities
18
JK2
19
20 T7
21
0
22
MS7
6xx 87 89 7x 16 48
23
HG7
Ax rattled recent ones, used repetition,
x? consolidated
24
JK7
11 11
4 2
25
26 T3
27
0
28
MS3
21 62 7x 48 5 11 3 5
8 8
1
29
HG3
55
1min 50sec presentation
8
30
JK3
kitchen used story
techniques
31
32 T6
33
0
34
MS6
56 4xx 7x 23
- -40
1
3 21
6
35
HG6
55 62
1 6
- 38 -
Appendices
36
37
38 T8
39
40
JK6
98
41
Steve8
42
Lisa8
43
Rita8
44
45
Karen8
MS8
46
47 T9
48
49
John9
50
51
Steve9
Lisa9
52
Rita9
53
Karen9
54
MS9
0
John8
44
0
55
56 T10
57
#REF!
58
John10
Steve1
59
0
60
Rita10
61
62
Motions kind of
stuck
ran MUCH environmental
ch distraction
40
1
55
7
S was VERY precise in recall
Ax 11 48
x
5
Karen1
0
MS10
55x 55x Tried
focusing on
written
words
40 55
5 7
55
7
12
S was
1
VERY
precise in
recall
55x 55x Ax Ax 80
x x
7
55
7
40
9
55x
- 39 -
Appendices
Appendix G – Normalization Formula for Calculating Serial Recall Probabilities
Calculating the recency and primacy recall probabilities required normalizing all the separate modality recall data to each
other. The number of stimuli items for each modality varied from trial run to trial run, and in the case of V, LV, and LA, also
varied among these modalities within a given trial run. For example, in trial run T9, there were seven LA and seven LV items,
six V items, and four for each of the other modalities. This meant that each modality had a varying number of items
presented which differed across trial runs. Calculating the probability that the ith item presented for modality m, required
normalizing the length of all the differing modality sequences.
The normalization formula employed was:
P(m,I) = (Tt) [ (Nts)(R(m,I,t,s) Cmt ) ] / (Tt)(Cmt Nt)
Where
The notation (Tt) means the sum across T things using index parameter t.
P(m,I) = probability, across all trial runs and across all subjects, that the item at interval location I was correctly
recalled for modality m. I varies from 0 to 1 and is a unit measure of item sequence length to which all
modality sequence lengths were mapped (i.e. I=0.5 means halfway through a sequence of stimuli).
P(m,I) is analogous to providing the probability that the Ith stimuli was correctly recalled in the
sequence of items presented for modality m.
R(m,I,t,s) = 1, if the stimuli for modality m at serial location I was correctly recalled for subject s on trial run t; else
0.
T =
Total number of trial runs.
Nt =
Number of subjects on trial run t.
Cmt =
Count of number of stimuli items in modality m for trial run t.
Appendices
Appendix H – Tabulated Recency and Primacy Data
The tabulated recency and primacy data is given below. The probability calculations are as given in another appendix. Table
values are probability that an item in the given serial position was recalled. Serial recall position (I) is given by the first row in
each table and is expressed as a percentage. For example, from the LV row in Table 1, a value of 0.34 for the 75% column
means there was an agregated probability of 0.34 that an LV item located 75% of the way through a stimuli list was recalled
by a subject. The percentage method was used so as to be able to equate the differing lengths of the various stimuli lists.
The data in the tables below are only for IM values of three, four and five (i.e., trial runs T3, T6, T8, T9, and T10) because
shorter IM lengths unduly bias the results due to quantization effects. For example, a stimuli list with only two items on it –
IM=2 – is dubious to use for assessing recency or primacy (even IM=3 seems doubtful, but it did seem to produce
measurable effects).
Table 1 – Probability that an item was recalled given its location in the serial recall sequence.
I 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95%
LV
LA
V
A
H
K
SA
Tota
l
V
+LV
+LA
LA
+LV
A+
H+K
+SA
A4-5
H45
K4-5
SA4
-5
Tota
l4-5
0.32
0.32
0.71
0.96
0.81
0.52
0.78
0.63
0.64
0.32
0.71
0.96
0.81
0.52
0.78
0.68
0.48
0.29
0.71
0.96
0.81
0.52
0.78
0.65
0.41
0.23
0.67
0.96
0.81
0.52
0.78
0.63
0.26
0.21
0.58
0.75
0.74
0.73
0.71
0.57
0.26
0.21
0.56
0.59
0.52
0.78
0.66
0.51
0.14
0.11
0.56
0.59
0.52
0.78
0.66
0.48
0.14
0.14
0.45
0.55
0.44
0.86
0.66
0.46
0.12
0.17
0.48
0.68
0.37
0.73
0.52
0.44
0.16
0.2
0.52
0.68
0.37
0.73
0.52
0.45
0.16
0.14
0.47
0.74
0.53
0.67
0.52
0.46
0.12
0.14
0.56
0.74
0.53
0.67
0.52
0.47
0.22
0.41
0.47
0.67
0.74
0.74
0.73
0.57
0.17
0.68
0.47
0.55
0.7
0.74
0.73
0.58
0.34
0.77
0.44
0.55
0.7
0.74
0.73
0.61
0.38
0.63
0.43
0.55
0.7
0.79
0.56
0.58
0.45 0.56 0.49 0.44 0.35 0.34 0.27 0.24 0.26 0.29 0.26 0.27 0.37 0.44 0.52 0.48
0.53
0.57
0.71
0.55
0.7
0.86
0.42
0.62
0.53
0.58
0.86
0.55
0.7
0.86
0.42
0.64
0.48
0.69
0.86
0.55
0.7
0.86
0.42
0.65
100
%
0.45
0.72
0.86
0.55
0.7
0.86
0.42
0.65
0.6 0.65 0.68 0.68
0.32 0.48 0.38 0.32 0.23 0.23 0.12 0.14 0.14 0.18 0.15 0.13 0.32 0.42 0.56 0.51 0.55 0.55 0.59 0.59
0.77 0.77 0.77 0.77 0.73 0.64 0.64 0.63 0.58 0.58 0.62 0.62 0.72 0.68 0.68 0.65 0.63 0.63 0.63 0.63
1
1
1
1 0.63 0.33 0.33 0.33 0.58 0.58 0.68 0.68 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55
0.65 0.65 0.65 0.65 0.53 0.13 0.13 0.13
0
0 0.3 0.3 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68
0.43 0.43 0.43 0.43 0.8 0.9 0.9 0.9 0.65 0.65 0.55 0.55 0.68 0.68 0.68 0.78 0.9 0.9 0.9 0.9
0.9 0.9 0.9 0.9 0.78 0.68 0.68 0.68 0.43 0.43 0.43 0.43 0.8 0.8 0.8 0.5 0.25 0.25 0.25 0.25
0.74 0.74 0.74 0.74 0.68 0.51 0.51 0.51 0.41 0.41 0.49 0.49 0.68 0.68 0.68 0.63 0.59 0.59 0.59 0.59
- 41 -
Appendices
Appendix I – Raw Recency and Primacy Data
Table 2 - Following is the raw primacy/recency data.
LV
5
1
1
1
1
1
1
1
1
1
LV
LV
LA
Interval I is given by the first row below (0 -> 100).
10 15 20 25 30 35 40 45 50 55 60
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
65
70
1
75
80
85
90
95
96
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
#
items # subj
6 4 T0
6
6
5 3 T1
5
5 3 T2
5
8 3 T7
8
6 3 T3
6
6 3 T6
6
6
6 5 T8
6
6
6
6
7 5 T9
7
7
9 4 T10
9
9
33
0.6 0.6 0.5 0.4 0.2 0.2 0.1 0.1 0.1 0.1 0.1 0.1 0.2 0.1 0.3 0.3 0.5 0.5 0.4 0.45
36 36 15 55 73 73 52 52 21 52 52 21 42 82 64 94 15 15 85
5
0.7 0.7 0.7 0.4
0
0
0 0.1 0.1 0.1 0.1 0.1 0.4 0.2 0.4 0.4 0.5 0.7 0.7 0.71
14 14 14 29
43 43 43 43 43 29 86 29 29 71 14 14
4
0.6 0.6 0.4 0.4 0.2 0.2 0.1 0.1 0.1 0.1 0.1 0.1 0.2 0.1 0.3 0.3 0.5 0.5 0.4 0.44
42 42 79 14
6
6
4
4 16 58 58 16 23 67 44 81 26 26 84
7
0.6 0.6 0.4 0.4 0.2 0.2 0.1 0.1 0.1 0.1 0.1 0.1 0.2 0.1 0.3 0.3 0.5 0.5 0.4 0.43
27 27 68 05 55 55 36 36 14 55 55 14 18 64 36 73 14 14 73
6
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 96
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
- 42 -
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
7
7
7
7
7
7
7
7
5
5
8
4 T0
3 T1
3 T2
3 T7
Appendices
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
8
8
8
6
6
6
6
6
6
6
6
6
7
7
7
7
8
8
8
3 T3
3 T6
5 T8
5 T9
4 T10
33
LA
LA
V
0.3 0.3 0.3 0.2 0.2 0.2 0.1 0.1 0.1 0.2 0.1 0.1 0.3 0.6 0.7 0.6 0.5 0.5 0.6 0.69
33 33 03 42 12 12 21 52 82 12 52 52 94 67 58 36 76 76 67
7
0.4 0.4 0.4 0.4 0.4 0.4 0.1 0.1 0.1 0.1 0.1 0.1 0.8 0.8 0.8 0.5 0.5 0.7 0.5 0.57
29 29 29 29 29 29 43 43 43 43 43 43 57 57 57 71 71 14 71
1
0.3 0.3 0.2 0.2 0.2 0.2 0.1 0.1 0.1 0.2 0.1 0.1 0.4 0.6 0.7 0.6 0.5 0.5 0.6 0.72
21 21 85 31 08 08 09 36 72 04
4
4 07 79 69 33 75 79 88
4
5
1
1
1
1
1
1
1
1
1
1
1
10
1
1
1
1
1
1
1
1
1
1
1
15
1
1
1
1
1
1
1
1
1
1
1
20
1
1
1
1
1
1
1
1
1
1
1
25
1
1
1
1
1
1
1
1
1
1
1
30
1
1
1
1
1
1
35
1
1
1
1
1
1
40
1
1
45
1
1
50
1
1
55
1
1
60
1
1
65
1
1
70
1
1
75
1
1
1
80
1
1
1
85
1
1
1
90
1
1
1
95
1
1
1
96
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
- 43 -
3
3
3
3
4
4
4
4
4
4
6
6
6
5
5
5
5
5
5
5
5
5
5
5
6
6
6
6
4 T0
3 T1
3 T2
3 T7
3 T3
3 T6
5 T8
5 T9
Appendices
1
1
V
V
A
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
7 4 T10
7
7
7
33
0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.4 0.5 0.5 0.4 0.5 0.4 0.4 0.4 0.4 0.7 0.8 0.8 0.84
58 58 58 27 36 06 06 85 15 45 85 45 55 55 55 55 27 48 48
8
1
1
1
1
1 0.8 0.8 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.5 0.5 0.5 0.5 0.5 0.57
57 57 71 71 71 29 29 29 29 71 71 71 71 71
1
0.7 0.7 0.7 0.6 0.5 0.5 0.5 0.4 0.4 0.5 0.4 0.5 0.4 0.4 0.4 0.4 0.7 0.8 0.8 0.85
13 13 13 71 81 57 57 49 79 21 73 57 67 67 43 25 07 56 56
6
5
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
10
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
15
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
20
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
25
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
30
1
1
1
1
1
1
1
1
1
1
1
1
35
1
1
1
1
1
1
1
1
1
1
1
1
40
1
1
45
1
1
50
1
1
55
1
1
60
1
1
65
1
1
70
1
1
75
1
1
80
1
1
85
1
1
90
1
1
95
1
1
96
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
3
3
3
3
3
3
3
3
3
3
4
4
4
4
4
5
5
5
5
3 T3
3 T6
5 T8
5
4
20
A
A
A4-5
H
0.9 0.9 0.9 0.9 0.8 0.6 0.6 0.6 0.7 0.7 0.7 0.7 0.7 0.5 0.5 0.5 0.5 0.5 0.5 0.55
5
5
5
5
5
5
5
5
5
5
5
5
5
5
0.9 0.9 0.9 0.9 0.7 0.5 0.5 0.5 0.6 0.6 0.7 0.7 0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.54
59 59 59 59 53 89 89 48 85 85
4
4 71 48 48 48 48 48 48
8
1
1
1
1 0.6 0.3 0.3 0.3 0.6 0.6 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6
5
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
10
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
15
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
20
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
25
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
30
1
1
1
1
1
1
1
1
1
1
1
1
35
1
1
1
1
1
1
1
1
1
1
1
1
40
1
1
45
1
1
50
1
1
55
1
1
60
1
1
65
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
- 44 -
70
1
1
1
1
1
75
1
1
1
1
1
80
1
1
1
1
1
85
1
1
1
1
1
90
1
1
1
1
1
95
1
1
1
1
1
96
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
3
3
3
3
3
3
3
3
3
3
3
4
4
4
4
5
3 T3
3 T6
5 T8
5 T9
4 T10
Appendices
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
5
5
20
H
H
H4-5
K
0.8 0.8 0.8 0.8 0.8 0.6 0.6 0.5 0.4 0.4 0.6 0.6 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7
5
5
5
5
5
5
5
0.8 0.8 0.8 0.8 0.7 0.5 0.5 0.4 0.3 0.3 0.5 0.5 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.69
08 08 08 08
4 21 21 38
7
7 34 34
4 99 99 99 99 99 99
9
0.7 0.7 0.7 0.7 0.5 0.1 0.1 0.1
0
0 0.3 0.3 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7
5
1
10
1
15
1
20
1
25
1
30
1
35
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
40
1
1
1
1
1
1
1
1
1
45
1
1
1
1
1
1
1
1
1
50
1
1
1
1
1
1
1
1
1
55
1
1
1
1
1
1
1
1
1
60
1
1
1
1
1
1
1
1
1
65
1
1
1
1
1
1
1
1
1
70
1
1
1
1
75
1
1
1
1
80
1
1
1
1
85
1
1
1
1
90
1
1
1
1
95
1
1
1
1
96
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
3
3
3
3
3
3
3
3
3
3
4
4
4
4
5
5
5
5
3 T3
3 T6
5 T8
5 T9
4 T10
20
K
K4-5
SA
0.5 0.5 0.5 0.5 0.7 0.7 0.7 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.8 0.8 0.8 0.8 0.85
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
0.5 0.5 0.5 0.5 0.7 0.7 0.7 0.8 0.7 0.7 0.6 0.6 0.7 0.7 0.7 0.7 0.8 0.8 0.8 0.86
21 21 21 21 26 81 81 63 26 26 71 71
4
4
4 95 63 63 63
3
0.4 0.4 0.4 0.4 0.8 0.9 0.9 0.9 0.7 0.7 0.6 0.6 0.7 0.7 0.7 0.8 0.9 0.9 0.9 0.9
5
1
10
1
15
1
20
1
25
1
30
1
35
1
40
1
1
45
1
1
50
1
1
55
1
1
60
1
1
65
1
1
70
1
75
1
80
1
85
1
90
1
95
1
96
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
3
3
3
3
3
3
3
3
3
3
4
4
4
4
5
5
5
5
3 T3
3 T6
5 T8
5 T9
4 T10
20
- 45 -
Appendices
SA
SA
SA45
0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.7 0.7 0.7 0.5 0.4 0.4 0.4 0.45
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.7 0.7 0.7 0.5 0.4 0.4 0.4 0.42
81 81 81 81 12 58 58 58 21 21 21 21 26 26 26 62 25 25 25
5
0.9 0.9 0.9 0.9 0.8 0.7 0.7 0.7 0.4 0.4 0.4 0.4 0.8 0.8 0.8 0.5 0.3 0.3 0.3 0.3
- 46 -
Appendices
Appendix J – Clustering of Subject Responses
This appendix lists the data for how subjects clustered their recall by modality (clustering means recalling items in sequence
from the same modality even though those items were not presented in sequence). Clustering is evidence that a subject
strongly associated items in a cluster and can be taken as evidence of a form of categorization. An item was considered
clustered if it was recalled in sequence with another item in the same modality. This data was not reported in the body of the
paper.
Table 1 – Total number of clustered responses by modality
#M = # of AHKSA modality items per Trial Run
N = # of subjects
IC = # of items for given modality within given trial run
R = # correctly recalled items within the given trial run
CL = # of responses which were clustred in sequence during recall
IM
T0
T1
T2
T7
T3
T6
T8
T9
T10
N
0
0
2
2
3
3
3
4
5
4
3
3
3
3
3
5
5
4
V
LV
LA
A
H
K
SA
IC R CL IC R CL IC
R CL R CL R CL R CL R CL
3
9
4
6
9
4 7
11
7
4
7
5
5
6
2 7
11
7
4 11
7
5
4
0 5
5
0
6
4
4
2
6
4
5
2
6
8
5
8
6
0 8
9
7
2
0
5
4
4
2
6
6
5 11
8
6
4
0 6
9
6
7
6
8
8
7
7
4
2
5
8
5
6
4
0 6
7
7
8
7
8
8
4
2
6
6
5 18 13
6 16
9 6
10
6 10
7 12 11 14 14 11
9
6 13
6
7 15
8 7
11
2 13 10 10
7 14 14 10
7
7 15
7
9
7
0 8
8
4 12 11
9
8 14 13 14 14
Table 2 – Percentage of responses which were clustered.
V
LV
LA
A
H
CLr Rr
CLr Rr
CLr Rr
Clr
Rr
Clr
Rr
44% 75% 44% 38% 64% 39%
71% 58% 33% 40% 64% 52%
64% 92%
0% 27%
0% 33% 67% 100% 50% 67%
63% 44%
0% 25% 78% 38%
0% 33% 80% 83%
73% 73%
0% 22% 67% 50% 86% 78% 100% 89%
63% 53%
0% 22% 100% 39% 88% 89% 100% 89%
72% 72% 56% 53% 60% 33% 70% 67% 92% 80%
46% 43% 53% 43% 18% 31% 77% 65% 70% 50%
47% 54%
0% 19% 50% 25% 92% 60% 89% 45%
Aver 59% 62% 25% 34% 54% 37% 55% 54% 65% 55%
age:
- 47 -
K
Clr
Rr
SA
Clr
Rr
67% 100% 40% 83%
50% 67% 100% 100%
100% 78% 50% 44%
50% 44% 100% 67%
100% 93% 82% 73%
100% 70% 70% 50%
93% 70% 100% 70%
66% 59% 61% 54%