Cognitive processing and knowledge representation
in decision making under uncertainty1
John Fox2 and Richard Cooper3
Abstract
This article is a contribution to the current debate on the role of cognitive theory in our
understanding of human judgement and decision making under uncertainty. We argue, with
Busemeyer et al and others, that the theoretical and methodological traditions of the JDM
community and mainstream cognitive science are divergent to an undesirable extent, and that
the exploitation of established concepts of information processing theories and knowledge
representation would considerably strengthen the field. The paper revisits and extends an
earlier study Making decisions under the influence of memory (Fox, 1980) in order to explore
how these proposals might be applied in practice. A central technique developed by cognitive
scientists is that of computational modelling; the paper makes extensive use of a new
modelling tool, COGENT, to show how cognitive theory can significantly illuminate the mental
processes involved in complex, real-world decisions.
To appear in Scholz and Zimmer (eds) Qualitative theories of decision making, in preparation.
1
We would like to thank David Budescu, University of Illinois; Alastair McClelland, University
College London; David Hardman of City University, London, and John Morton of the MRC
Cognitive Development Unit, London, for helpful comments on an earlier draft of the chapter.
2
Imperial Cancer Research Fund, Lincoln’s Inn Fields, London
3
Department of Psychology, Birkbeck College, Malet Street, London
Cognitive processing and knowledge representation
in decision making under uncertainty
John Fox and Richard Cooper
Introduction
The study of human decision making is an important topic in modern psychology. After all, our
effectiveness as individuals, and as societies, is profoundly influenced by individual’s abilities
to make the right choices when we are confronted by significant threats or opportunities.
Considering the importance of decision making it is surprising that the field does not seem to
have achieved the central position in cognitive research that other traditional psychological
topics have, such as the study of memory, perception or language. Why is this? One possible
reason is that the theoretical frameworks, experimental methods and even the sociology of
the decision making research community differ significantly from most other areas of
psychology and cognitive science.
A notable feature of the field of judgement and decision making (JDM) is that it is highly multidisciplinary. Researchers in economics, management science, sociology and statistics are all
contributors to JDM, as well as psychologists. From the resulting intellectual soup has
emerged a preoccupation which is not common elsewhere in psychology. This is a concern
with normativeness, or “the study of guidelines for right action” (Fishburn, 1988). Economists
who are interested in consumer choice, for example, are often concerned with “rational”
decision making. Statistical decision theorists formalise such ideas with normative concepts,
such as expected utility, a measure based on mathematical probability and quantitative value
scales. Social scientists and management scientists also frequently take normative
economical and statistical frameworks as given, though their explanatory accounts tend to be
more informal and intuitive.
Psychologists who are interested in the processes of decision making typically see their role
as illuminating and/or critiquing the application of normative thinking to the description of
human judgement.Theories of judgement, for example, often adopt or adapt classical decision
theory in order to model human decision functions (e.g the concepts of subjective probability
and subjective expected utility). Studies of indidividual and organisational decision making
often use the language of normative theory to describe decision making behaviour and its
prescriptions provide the standards against which human performance is judged.
Cognitive science, like the specific field of judgement and decision making, is concerned with
the study of mental processes, but it has developed very different theoretical and explanatory
frameworks. Cognitive scientists do not typically take the view that there is a normative theory
of how we should remember, see or hear, for example, but rather that these are processes
whose operation (good, bad or indifferent) is simply to be understood in terms of the
mechanisms which implement them.
While cognitive scientists often have very different interests and theoretical positions, they
generally share an interest in “how things work”. Cognitive psychologists tyically try to
understand the mental functions which underpin behaviour, and cognitive neuroscientists try
to understand these functions in terms of their implementation by specialised structures in the
brain. In contrast, as Jane Beattie of Sussex University has observed4, judgement
researchers are “behaviourists”, being primarily concerned with what people do rather than
how they do it. With few exceptions JDM researchers do not concern themselves, say, with
the role of memory in decision making, or investigate the effects of brain damage or
4
Personal communication (1994)
psychiatric dysfunction on choice behaviour, or explore uncertainty management in primate
problem solving.
This is not, in itself, a criticism of JDM research or its methods, but to outsiders like the
authors it is a concern that there is such a discontinuity between this important subfield of
psychology and other areas of cognitive science. In preparing this paper we have found that a
number of writers who are prominent in the field have also expressed concern about the
relatively poor links between theoretical research in JDM and mainstream cognitive
psychology (e.g. Wallsten, 1980) yet “The few cases of intersection ... are overwhelmed by
numerous other examples where research in one field proceeds without taking notice of
findings from the other fields” (Busemeyer, 1995, p xii).
Lola Lopes of the University of Iowa has observed that there are, roughly, three general
theoretical approaches to understanding human decision making; algebraic, procedural and
experiential approaches. Algebraic (also called “structural”) theories model judgment in terms
of mathematical choice functions such as expected utility functions. The aim of these theories
is to predict what people (students, consumers, managers, policy makers, doctors, patients,
or even courts of law) will decide under what conditions. Lopes estimates that perhaps 95% of
judgement research is carried out within this tradition which, unlike cognitive science, is
relatively unconcerned with mental processes or how individual knowledge and experience
may affect decisions and decision making strategies.
A number of other writers have questioned the centrality of the normative approach to
decision making under uncertainty from various points of view (e.g. Shanteau, 1987; Beach,
1990; Fox, 1990, 1994; Fox et al, 1990; Gigerenzer, 1994). However, these doubts continue
to fail to take root. Perhaps this is because they are questioning an orthodoxy which is deeply
rooted in modern political thought as well as in the dominant scientific tradition.
The neglect of non-algebraic frameworks seems highly undesirable to us 5. Potentially
important explanatory concepts from cognitive science which might have theoretical or
practical value may be lost, and the possibility of synergy with research in other traditions
missed. We guess that one of the reasons for this neglect is that many JDM researchers
assume that the cognitive processes involved in everyday decision making are so complex as
to make “mentalistic” theories intractable. We would question this. In his book Unified
Theories of Cognition Allen Newell describes a general theory of cognitive processing which
has at its heart a simple but powerful decision making process. While Newell’s theoretical
programme is ambitious and has serious difficulties (e.g. Cooper and Shallice, 1995) the work
suggests that decision processes, and many examples of complex behaviour which depend
upon decison processes, can be successfully modelled using information processing and
other cognitive concepts.
This gives us a particular reason to share what appears to be a growing concern that
judgement research is disconnected from mainstream cognitive research. In this article we
have attempted to demonstrate a number of ways in which bridges might be built between the
JDM and cognitive science communities, using information processing concepts and
computer simulation techniques. Our general objectives are to set out a “mentalistic” account
of decision making in complex decision making tasks. More specifically we aim to
demonstrate that:
1. decison making can be productively viewed as a process of applying symbolic knowledge
to information about a specific situation in order to resolve uncertainties in problem
solving
5
Politically as well as scientifically!
2. computational modelling techniques can be used to precisely express psychological
theories and to formulate predictions from them
3. the modelling techniques can be used to explore and evaluate competing theoretical
claims
We hope to show that there is considerable scope for applying recent developments in
cognitive science to in order to understand human decision making. Adoption of these
methods may improve the descriptive power of psychological decision theory, and establish
stronger links with other areas of cognitive research.
The starting point for our presentation is a study of medical decision making and a model of
the functioning of memory mechanisms in the decision making process (Fox, 1980). Memory
is crucial to most decision making yet has almost no place in modern judgement theory. As
Weber et al (1995) comment, “most models of decision or judgement performance do not
explicitly [incorporate] considerations of memory processes and [or] representations of
information in memory”.
Section 2 outlines the main features of the 1980 study and the data obtained from it, together
with a comparison of the behaviour of a group of medical students carrying out a diagnosis
task and the behaviour of a computer model of the role of memory processes in their decision
making. In section 3 we present a reimplementation of this model using COGENT, a tool
recently developed for modelling cognitive processes and investigating cognitive theory.
COGENT allows us to explore a number of additional theoretical questions about knowledge
representation and cognitive processing which were not possible in the earlier paper..
2. Making decisions under the influence of memory
Fox (1980) reported a study in which medical students learned to make diagnostic decisions
in a realistic but carefully designed task environment (Figure 1). The students were required
to diagnose “patients” simulated by a computer, in which the statistical relationships between
diseases and symptoms were precisely known. During the experiment a number of aspects of
the behaviour of the students were comprehensively recorded.
At a number of points in the experimental procedure the subjects also took part in a memory
retrieval task. This tested their growing knowledge of the relationships between the diseases
and symptoms, acquired as a result of their increasing experience on the task.
The data acquired during this study were used to inform the development of a model of the
medical students’ decision making process. This model was presented in terms of an
interaction between cognitive processing mechanisms and the representation of medical
knowledge in memory.
The 1980 paper also reported on a number of computer simulations of the students’
performance. The aim of the simulations was to investigate whether (a) it was possible to
predict features of the students’ decision making and (b) how the theoretical assumptions of
the model could impact on the accuracy of the predictions.
Figure 1: The task environment described by Fox (1980). On the left is a computer display which was used to
simulate patients, and the inset panel was the control panel used to select questions and enter diagnoses.
On the right is a computer terminal which was used for the memory experiment (see text).
Laboratory task and data collection
The study was organised as a number of blocks of trials, on each of which a patient was
simulated by a computer, and the subject’s task was to decide the diagnosis. On each trial the
computer presented one of five symptoms: dysphagia (difficulty in swallowing); vomiting,
headache, earache and pyrexia (raised temperature). The students knew that the “patient”
might be suffering from any one of five diseases: tonsillitis; laryngitis; meningitis; hepatitis or
(infection of the parotid glands). After being told the presenting symptom the subject could
ask about the presence or absence of any of the other symptoms, in any order, and could
offer a diagnosis at any point. The selection of the presenting symptom, and the answers to
any questions which were asked, were determined by the computer by reference to a set of
conditional probabilities (reproduced in table 1).
The experiment consisted of 4 blocks of 25 trials in which the subject was required to
diagnose what was wrong with each patient. Each block of diagnosis trials consisted of 5
instances of each of the 5 diseases (in randomised order). Various features of the students’
decision making were recorded by the experimental equipment. These were (1) the number of
questions asked for each decision, (2) the order in which questions were asked, and (3) the
diagnosis that the student arrived at on each trial.
Dysphagia
Tonsillitis
Laryngitis
Meningitis
Hepatitis
Parotitis
1.0
0
1.0
1.0
1.0
Vomiting
1.0
0
0.50
0
0.25
Headache
0
0.50
0.75
0.50
0.75
Earache
0
1.0
0
0
0.25
Pyrexia
0
0
0.25
1.0
0.25
Table 1: The conditional probabilities relating diseases and symptoms in the patient simulation.
Figure 2 shows the overall pattern of questioning shown by the medical students during the
final block of trials, for each presenting symptom. The first node in each tree is the presenting
symptom, and the branches show the main “routes” the subjects took in their question
selection. The numbers in the figure are the overall frequencies of the most common routes.
While there is considerable variety in the patterns there are also clear preferences in the
question selections.
Figure 2: The question selection patterns for each presenting symptom, recorded for 18 subjects in the final
block of decision making trials. Each circle represents the presenting symptom or a question (Dysphagia,
Vomiting, Pyrexia, Headache, Earache). A single line into a question means the previous symptom was
always present or always absent while a double line means that the question was asked whether or not the
previous question received a consistent answer from the computer. Small filled circles indicate the point at
which a diagnosis was given and the numbers underneath indicate the frequency with which the particular
path through the tree was taken. (For clarity all paths that were used only once during the experiment have
been excluded.)
Turning to the subjects decisions, all the students learned to do the task and to achieve a high
level of performance over the four blocks; Table 2 summarises the data obtained during the
fourth block of trials.
Each set of diagnosis trials was interleaved with a memory task which was intended to test
the students’ knowledge of the associations between the diseases and symptoms. The
memory task consisted of 25 trials in which a proposition of the form
<Symptom> is associated with <Disease>
was presented to the subject. In each case the subject was asked to confirm or deny the
statement presented, and their responses and response times were recorded. Figure 3
reproduces the group average response times for the 25 propositions. Although there is
considerable scatter in the data it can be seen that there is a systematic relationship between
the conditional probability of a symptom occurring with a disease and the response time; the
closer the probability is to 0 or 1 the faster was the response to confirm or deny the
association.
Dysphagia
Dysphagia
Vomiting
Headache
Earache
Pyrexia
No
question
2
3
38
10
0
48
8
33
9
21
31
0
X
Vomiting
7
Headache
31
Earache
38
Pyrexia
24
X
22
X
X
3
23
8
43
25
0
X
19
0
X
Diagnostic accuracy by comparison with the true presenting disease:
Average number of questions before giving a diagnosis:
81 %
2.12
Table 2: Summary of the main results of the medical students’ behaviour on the last block of trials. The table
shows the “one-ply” data, the frequency with which each question was asked given each presenting
symptom (the X shows the most frequent question).
Figure 3: the overall response times for subjects confirming or disconfirming the relationship between
symptoms and diseases, as a function of their actual frequency of co-occurrence (probability).
Simulation experiments and data collection
The principle aims of the simulations were to compare the predictive or explanatory power of
a family of algebraic models of the students’ performance with alternative “cognitive” models.
The basic algebraic model consisted of a number of quantitative functions:
(1) for revising probabilities in the light of symptom data (Bayes’ rule);
(2) for evaluating the expected information yield of alternative questions (based on a
probabilistic entropy maximisation function);
(3) a decision function which selected a diagnosis based on the posterior probability of the
diseases given the available symptom information (the simulation would decide that a disease
Di was the diagnosis if the probability of Di given a set of symptoms Ej exceeded some
threshold parameter Theta).
Table 3 summarises the results obtained by running this simulation. These are to be
compared with the summary data shown in Table 2.
The predictive power of the model was judged reasonable though not convincing. A number
of variants of the model were developed (e.g. by varying the hypothetical relationship
between the students’ subjective probabilities corresponding to the objective probabilities
shown in table 1) but none of the manipulations resulted in a significant improvment in the
predictions of the model.
Dysphagia
Vomiting
Headache
Earache
Pyrexia
No
question
Dysphagia
X
Vomiting
X
Headache
X
Earache
X
Pyrexia
X
Average number of questions before giving a diagnosis:
1.57
Table 3: Summary of the main results of the Bayesian decision model. The table shows the “one-ply” data,
the frequency with which each question was asked given each presenting symptom (the X shows the most
frequent question).
The second simulation consisted of an information processing architecture with two main
components; a reasoning mechanism and a working memory. The former was used to
implement a decision process as a set of if...then... production rules embodying knowledge
about the medical task; the rules reacted and progressively added to the contents of working
memory. As data entered working memory rules could “fire” if particular data patterns became
true, and conclusions were then added to working memory as new data. Each addition might
result in further rules firing, so that the reasoning and decision making process was modelled
as a cyclical process. The rules making up the decision process included rules for generating
diagnostic hypotheses (in response to symptom data); anticipations (of whether symptoms
would be present or absent); and queries (about the presence of particular symptoms).
Figure 4 reproduces the rule set published in the original paper.
The central point to note is that in this model all the rules are qualitative There is no
quantitative representation of uncertainty, such as the probabilities of diseases or the
conditional probabilities of diseases given symptoms. Uncertainty is represented implicitly, in
the availability of knowledge in memory. The basis for assigning relative availability to
information retrieved into working memory from the knowledge base was the average
response times of the subjects in the memory task (Figure 3).
Put simply, the model assumed that the knowledge which “comes most easily to mind” (that
is, becomes available in working memory) will have a greater influence on the processes of
reasoning and decision making than knowledge which becomes available more slowly. The
availability of information was encoded in the order of the conclusions of the rules (Figure 4).
The effect of this ordering is to determine the order in which other rules will fire. This will
control the order in which hypotheses and anticipations are added to working memory, and
hence how the rules which produce queries (and hence questions) are instantiated.
In short, if memory is systematically biased to retrieve some information more quickly than
others then this information will tend to dominate reasoning and decision making.
Figure 4: The production rules of the decision procedure in simulation 2 reproduced from Fox (1980)
The results of running two rule-based simulations are shown in table 4, to be compared with
the subject data in table 2 and the results of the algebraic model in table 3. It will be seen that
the behaviour of these two simulations resembles that of the subjects more closely than the
behaviour of the quantitative models.
The conclusions from this simulation study were that a qualitative representation of task
knowledge could implement a decision procedure whose decision making performance was
more similar to that of the human subjects than a variety of quantitative models, and that the
execution of this decision procedure, in interaction with a memory retrieval process, yielded a
better account of the sequential features of the subjects’ question selection behaviour than
functions designed to maximise information gain. To summarise, the qualitative decision
model was able to predict interesting features of behaviour on a relatively complex task, and
did this within a theoretical framework which was broadly in line with fairly established ideas
about the organisation of human cognitive processes.
Dysphagia
Vomiting
Headache
Earache
Pyrexia
No
question
Dysphagia
D
V
Vomiting
DV
Headache
D
V
Earache
D
V
Pyrexia
DV
Average number of questions before giving a diagnosis:
1.73
Table 4: Summary of the main results of the qualitative models. The table shows the “one-ply” data, the
frequency with which each question was asked given each presenting symptom (the D shows the preferred
question when the DISCRIM rule was used, and the V shows the preferred question when the VERIFY rule
was used to select the question).
In the remainder of this paper we present some new results from additional simulations. The
motivation for the additional modelling efforts is to explore these conclusions in more detail,
and to illustrate how recent developments in computational modelling may be able to provide
further insights into the cognitive processes involved in decision making.
3. How critical are processing and representational
assumptions to the decision making model?
In assessing the value of a computational model such as that above, two questions are likely
to come to mind. First, how much of a model’s explanatory power comes from theoretically
significant assumptions about information processing and/or knowledge representation, as
distinct from purely technical features of the program? Experiments 1 and 3 explore this
question.(This is a general methodological problem in simulation work which is extensively
discussed by Cooper, Fox, Farringdon and Shallice, 1996.) Second, we may ask the
complementary question: if we change the model in such a way that important theoretical
assumptions are violated, will the behaviour expressed by the model also change so that it no
longer provides as good an account of subjects’ behaviour? This is the focus of Experiment 2.
The “simulation experiments” described here have been carried out using a new tool for
cognitive research, the COGENT cognitive modelling package. COGENT is designed to make
it relatively easy for the psychologist to explore theories of cognitive processing by
implementing a computer simulation of the theory, and systematically experimenting with
alternative theoretical assumption in order to formulate and test empirical predictions. This
can be difficult to establish by hand if we are working with complex models and/or complex
tasks.
COGENT provides a kind of construction kit, a set of standard software modules that can be
linked together and parameterised in various ways in order to construct any of an indefinitely
large set of specific models. Once constructed the model can be run like any computer
simulation in order to observe its behaviour and compare it with human behaviour.
The main modules currently provided by COGENT are: storage buffers, symbol processing
modules, connectionist (subsymbolic) networks and data transmission functions (links
between modules).
A COGENT buffer is simply a place to store items of static information (such as chunks of
knowledge about diseases and symptoms) or dynamic information (such as inferences about
a patient).
A COGENT process is some sort of information processing mechanism, which is used to
manipulate (add, delete or modify) the contents of buffers to which it is linked.
We shall describe several variants of the decision model reported by Fox (1980) which have
been built using the COGENT system. Technically COGENT is very different from the
simulation software developed for the 1980 studies, but the empirical evaluations are carried
out using the same published data.
Experiment 1: Changing the implementation details
Figure 5 shows a COGENT implementation of the 1980 decision model. The model is
organised into two levels, illustrated by the contents of the two windows in the figure. The first
level (in the left window) is a simulation of the complete experimental situation shown in figure
1, i.e. a simulation of the student (subject model), and a separate but linked simulation of the
task environment (the experimenter model). The experimenter model simulates the patients,
and records and analyses the behaviour of the subject model. It communicates with the
subject model as needed by sending messages down the link (“presenting” symptoms and
answering questions). The subject model is shown in more detail in the right hand window. It
consists of a working memory implemented as a buffer (shown as a rectangle with rounded
ends) and a decision procedure implemented with a symbolic computation process (pointed
ends).
The second process in the right-hand window is not a significant part of our model, but only
a mechanism whereby the decision process model interacts with the experimenter model. It
does this by sending information to the experimenter model (as when it asks questions or
gives its diagnosis) and receives messages from it (the presenting symptom or the answers to
its questions).
The COGENT approach to building cognitive models out of standard components frees the
psychologist from a great deal of (potentially laborious) programming. However, these
standard components may have properties which are not intended to be part of the model
builder’s theory (Cooper et al, 1996). The structure of working memory in the 1980 simulation
is a case in point. In the published simulation working memory was modelled as a hierarchy of
chunks, each of which could be accessed through a header feature and then searched
serially. This hierarchical data-structure could be implemented, but the standard COGENT
buffer is primarily designed to store simple sets of elements (which can be accessed
randomly or scanned with recency or primacy priority) reflecting common assumptions about
working memory organisation. In the first simulation experiment we decided to reimplement
the 1980 model using a standard COGENT buffer for the working memory. (Otherwise the
decision process consists of a set of rules which is precisely equivalent to those in Figure 3,
apart from minor syntactic differences.) Since the decision model makes no assumptions
about the structure of working memory other than the order in which it is loaded with
information (hypotheses, anticipations, queries etc), and the way in which this information is
accessed, we should expect these implementation details to have no behavioural
consequences.
Figure 5: Reimplementation of the 1980 model.
COGENT provides a facility to view any buffer continuoualy during the execution of a model.
Figure 6 illustrates the set up of working memory, together with the contents of the buffer at
the end of a typical diagnosis trial.
At the top of Figure 6 is an area where various functional parameters of the buffer can be set.
These provide a number of ways to specialise the standard buffer, to give it the properties
required for a specific model. It can be seen, for example, that in order to use the buffer as a
working memory for this model it has been set up for a particular mode of access (LIFO, lastin-first-out) and that its contents do not decay over time (so no information is lost during a
trial). Obviously these parameters could be changed if a particular theory required different
assumptions.
Below the setup area is a window in which any initialisation data that are required on each
trial can be specified (there are none in this case). In the bottom window we see the contents
of the working memory at the end of a simulated diagnosis trial.
The operation of the model is as follows.
When a cogent model is executed all processes in the model are executed in parallel; reading
and writing to any buffers to which they are connected, sending messages to other processes
and so on. Although computation is parallel and distributed, however, data are added to
working memory sequentially, causing rules to fire and send messages and/or update
working memory on specific cycles. The numbers on the left of each working memory element
show the cycle on which an element was added; it can be seen that information builds up
cumulatively during the run.
At the bottom of the buffer we see that the patient’s presenting symptom was (“told(vomiting,
present”) on cycle 3, then two hypotheses, hepatitis and meningitis, are added on cycle 4.
This is followed on cycle 5 by adding expectations; the symptoms associated with these two
diseases. On cycle 6 the model generates a query about whether the symptom headache is
present. (The model has been set up to be run in “verify” mode, in order to try to confirm the
first hypothesis it finds in memory, which in this case is meningitis.) The answer comes back
from the task simulator on cycle 9 (it takes a couple of cycles to output the question and get
the answer through the input process). Given this new symptom information the model is
able to make a diagnosis on cycle 10. (Note in passing that the model has also generated
another query, about the presence of earache, on the same cycle. This is because of the
parallel nature of the system, which allows inferences to occur asynchronously. However this
query is not translated into an overt question since a diagnosis has already been arrived at.)
Figure 6: State of working memory at the end of a typical diagnosis trial
Table 6 shows the results of testing this new implementation of the 1980 model. If it is
compared with the results reported in the original simulation (Table 4) it will be seen that the
question selection pattern is identical. The quantitative results are also closely similar.
(Actually the average number of questions predicted by the COGENT model is 2.1, which is
closer to the equivalent figure (2.12) seen in the subject data in table 2 than the original
model. However this is due to random variation in the simulated patient sequence and has no
significance.)
In short the differences in the implementation details appear to have had no detrimental effect
on the fit between the model and the subject data.
Dysphagia
Vomiting
Headache
Earache
Pyrexia
No
question
Dysphagia
D
V
Vomiting
DV
Headache
D
V
Earache
D
V
Pyrexia
DV
Diagnostic accuracy by comparison with the true presenting disease:
D 72%; V 94%; overall 83%
Average number of questions before giving a diagnosis:
D 1.5; V 2.7;
av = 2.1
Table 6: Results of the COGENT reimplementation of the 1980 model. The results are identical to
those reported in the earlier implementation, which had a very close firt with the observed subject data
Experiment 2: The effects of working memory access assumptions
In contrast to technical changes which have no theoretical motivation, any change to the
model which might affect the significant parameters of information processing would be
expected to change behaviour. Trivially, if the rules in the decision procedure embodied no
knowledge about tonsillitis, then the decision process could not form hypotheses about
tonsillitis, select questions to verify tonsillitis, nor diagnose the condition. This expectation is
easily confirmed by simulation. Of greater interest is the central theoretical claim of the 1980
paper, that the operational use of knowledge is significantly influenced by its availability in
memory. If this is changed then, according to the theory, it should produce a significant
change in the pattern of behaviour, notably to the order in which questions would be asked.
In experiment 2 we investigated two changes to the effects of availability by changing the
access mechanism of working memory.
Recall that the 1980 model claims that working memory is loaded with information about
hypotheses and expectations in an order which is determined by the reliability of the
association between each symptom and disease. If we change the working memory access,
so that this relationship is no longer mirrored in the availability of information in working
memory, then we would expect the model to behave differently (though how differently is not
immediately obvious in a model like the present one which has scope for considerable
operational variability). The first experimental manipulation therefore consisted of changing
the working memory access regime so that the items which were loaded first into working
memory would now be the last to be available to the question selection process.
The results of this variation are shown in Table 7. The fit with subject data has now dropped
dramatically, from a perfect fit of 5 agreements to only 1 agreement for the and discrimination
strategy. The fit for the verification strategy is equally poor.
Dysphagia
Vomiting
Headache
Earache
Pyrexia
No
question
Dysphagia
DV
Vomiting
D
V
Headache
DV
Earache
DV
Pyrexia
DV
Diagnostic accuracy by comparison with the true presenting disease:
D 80%; V 96%; overall 88%
Average number of questions before giving a diagnosis:
D 2.0; V 3.0;
av = 2.5
Table 7: Results of rerunning the decision model with modified working memory access. Access
is based on linear scanning, as in experiment 1, but the direction of scanning is reversed.
Dysphagia
Vomiting
Headache
5
Dysphagia
Earache
Pyrexia
No
question
7
V D
7
Vomiting
2
4
D
Headache
3
4
Earache
6
3
Pyrexia
2
1
V
4
V
1
4
D
7
4
5
V
3
D
1
4
V
4
D
2
5
2
V
2
3
1
1
D
Diagnostic accuracy by comparison with the true presenting disease:
Average number of questions before giving a diagnosis:
D 2.3;
4
V
D 80 %; V 92 %; overall 83%
V 3.1; av = 2.7
Table 8: Effect of modifying the access of working memory from linear scanning to random selection.
This destroys the availability effect and introduces considerable behavioural variety (as shown by the
inset numbers which are the frequencies with which questions were selected by the
discrimination and verification strategies (left and right, respectively).
Little effect on diagnostic performance is expected, however. Since diagnosis is carried out by
rules which recognise particular patterns of symptoms the diagnosis decision is primarily
determined by the symptoms the patient has rather than the order in which the information is
presented. Table 7 confirms this: the diagnostic performance is unaffected by the changed
memory access.
A second manipulation of the working memory model aimed at disrupting the relationship
between symptom reliability and availability has also been carried out. This time working
memory has been set up to be randomly accessed. Every time a rule in the decision
procedure attempts to match one of its conditions with information in working memory a
random procedure is used to select the item to be matched, rather than carrying out a linear
search as in the previous experiments; COGENT will keep on randomly selecting items from
the buffer (without replacement) until it finds a match or exhausts the buffer without finding a
match. The consequence of this is to nullify any effect of loading working memory in any
particular sequence.
Table 8 shows the effect. Examination of the data reveals two obvious features:
(1) the fit between the first question preferred by the model and that preferred by the subjects
has again been reduced by comparison with the data in Figure 2.
(2) the clear peaks in the orginal deterministic model have now been “smeared” out, with the
result that there are a number of peaks. Given a particular presenting symptom, headache
say, the model sometimes asks about vomiting first, sometimes earache first. Overall the
most preferred questions no longer show a resemblance to the most preferred questions of
the subjects.
Although the memory access likelihood is equally distributed over the working memory by the
COGENT access function, the questions are clearly not being selected randomly (some
questions are never asked under some conditions). The reason for this is that availability of
information in memory is not, in fact, the only factor which determines how decision making
will proceed; there are also logical constraints built into the decision rules, and these will
exclude certain questions for reasons which are due to the structure of the task as distinct
from the organisation of the architecture. For example, since parotitis is never associated with
vomiting the decision procedure will obviously never attempt to verify a diagnosis of parotitis
by asking whether vomiting is present.
Experiment 3: Equivalence of different knowledge representations
In Experiment 1 we showed that changes to the representation of information in working
memory had no effect on behaviour, so long as parameters which affect the interaction
between the logical decision process and the accessibility of information are not changed.
Experiment 2 showed the reverse situation; the representation can be left unchanged but if
the processing parameters are significantly altered the decision making performance of the
system is not significantly affected, though the order in which questions are asked is much
changed. In neither experiment was the encoding of medical knowledge in the decision
procedure significantly changed from that used in the original 1980 model (Figure 3).
This raises the question of whether behaviour is sensitive to the representation of medical
knowledge or not. In principle our answer to this question is the same as when we considered
whether the encoding of information in working memory would affect behaviour; so long as we
do not change the medical content of the knowledge base, nor modify critical execution
parameters such as memory retrieval parameters, we would expect no change in the
behaviour of the system.
Figure 7: Abstraction of generalised diagnostic procedure from “special-case” medical knowledge.
Figure 7 shows a revised COGENT model in which a generalised decision procedure for
diagnostic decision making has been separated from specific knowledge about diseases and
symptoms. This decision procedure can be viewed as a generalised strategy for proposing
diagnostic hypotheses, anticipating symptoms and selecting questions which can be used to
differentiate any set of alternative diagnoses. Figure 8 shows this generalised diagnostic
strategy. The decision process now consists of only 5 rules, one for each distinct function (1.
generate hypotheses; 2. generate anticipations; 3. select question to verify a hypothesis; 4.
select question to differentiate two hypotheses; 5. make a diagnosis). Instead of having a
large number of specialised medical rules in which the decision process is implicit, the
decision process is now explicit.
The the specialised domain knowledge is stored in a separate knowledge base, as a
collection of facts like “tonsillitis is associated with dysphagia”. Logically the two models are
exactly equivalent though there are a number of well known advantages to the latter kind of
representation, including:

Diagnostic expertise can be simply increased by learning additional facts, such as “peptic
ulcer is associated with vomiting” or “chicken pox is associated with spots”. Once these
facts are added to the knowledge base they can be used immediately without having to
reprogram the decision procedure.

The medical knowledge is represented “declaratively” rather than “procedurally”. In the
first model the medical facts required to do the task are embedded in the rules which form
the diagnosis procedure, so this knowledge cannot be used for any purpose other than
diagnosis. When encoded as distinct facts, as in the second model, the knowledge can be
reused for other purposes. Another use would be to introduce an additional process which
would use the same medical knowledge base in order to explain a decision, for example.
Figure 8: A first-order implementation of the decision procedure. Each rule is a “first-order” inference rule
which is evaluated by matching the antecedent terms with the contents of an associated memory. Variables
(capitalised words, such as Disease) are instantiated by the matching process, as when the Disease variable
is instantiated by “tonsillitis”. The rules are all set to be “refracted” which is to say they will only be applied
once with any specific instantiation, though they can be applied any number of times with different
instantiations.
Apart from the reduction in rule numbers from the original model there are a number of other
changes to the form of the rules in the revised model. First, COGENT processes can
manipulate any number of buffers. Each condition of a rule, therefore, refers explicitly to a
buffer, here “working memory” or “knowlege base”. This means that COGENT will test the
conditions of the rules by attempting to retrieve matching terms from the named buffer.
Second, the rules include variables, such as Disease, Symptom and so on. The syntactic
form of a rule’s premisses constrains the form of the terms they can be matched in the buffer.
In the terminology of mathematical logic the first model is limited to propositional reasoning,
while the latter has the greater power of the first-order predicate calculus.
Figure 9: Execution of the first-order implementation of the model. The behaviour of the model is
indistinguishable from the propositional implementation of the decision procedure.
Dysphagia
Vomiting
Headache
Earache
Pyrexia
No
question
Dysphagia
D
V
Vomiting
DV
Headache
D
V
Earache
D
V
Pyrexia
DV
Diagnostic accuracy by comparison with the true presenting disease:
Average number of questions before giving a diagnosis:
D 2.3;
D 80 %; V 92 %; overall 83%
V 3.1; av = 2.7
Table 9: Summary analysis for the first-order model showing that its behaviour is effectively
identical to the behaviour to the propositional model.
Returning to our general theme we have predicted that even apparently major changes to the
representation need not imply any change in the behaviour of the model, so long as we do not
alter either the medical knowledge content or the memory retrieval parameters,. Figure 9
shows the state of working memory at the end of a diagnostic trial; it is identical to that shown
in Figure 4, and indeed the dynamic processing of working memory by the first-order decision
procedure is the same as for the propositional procedure for all simulated patients. This is
reflected in the summary data in Table 6.
4. Discussion
We have suggested that the information processing mechanisms which many cognitive
scientists seek to understand must underpin decision making as well as other cognitive skills,
and that an understanding of their role is likely to form part of a complete theory of human
judgement. The 1980 study of decision making suggested that an understanding of memory
retrieval mechanisms, symbolic reasoning, and the interaction between them could make a
significant contribution to our understanding of human decision making under uncertainty. The
additional simulation experiments reported here underline this point, particularly the
demonstration that assumptions about memory access can be crucial in achieving accurate
predictions about decision making behaviour.
Decision making can be productively viewed as a process of applying symbolic knowledge,
both specialised and general knowledge, to symbolic data. This contrasts with classical
algebraic models which view judgement as a form of numerical calculation. On the contrary
the present approach, in which medical knowledge is represented as qualitative inference
rules, leads to a model which appears both powerful and intuitively appealing as an account
of how decision makers form hypotheses, seek evidence, take decisions and so forth.
The equivalence of the propositional and first-order representations is particularly striking.
From a purely logical point of view the result is not surprising (any theorem in propositional
logic is a theorem in first-order logic) but from a cognitive psychological point of view it raises
some interesting issues.
Hastie and Pennington (1995) remark that a reliable observation from several decades of
research on mental representations is that people are “concrete thinkers”; we prefer taskspecific, concrete representations of the world and rarely rely on abstract representations.
The decision procedure in the propositional model reported here, is implemented as a set of
concrete, “special case” rules. The first-order decision procedure, however, abstracts a
generalised decision making process from knowledge of specific diseases and symptoms.
One possible conclusion from this is that the level of abstraction which people actually use
may not be easy to identify empirically, at least in the present type of task, since the concrete
and abstract process models can produce indistinguishable behaviour. (In fact it is possible to
carry out a further abstraction step, in which strategic knowledge of how to take any kind of
decision is abstracted from the relatively specific knowledge of how to take diagnosis
decisions in medicine and still exactly simulate the behaviour of our subjects 6.)
This raises the general issue of distinguishability of theories, which has been debated in a
number of areas of cognitive psychology. John Anderson and others argue that
representations cannot, in principle, be distinguished empirically, for example, since any
representation of sufficient richness can in principle mimic any other, while Zenon Pylyshin
6
This is not reported here because the further abstraction step appears to require the
introduction of a number of very general “mentalistic” concepts, such as goals and plans,
which are intuitively plausible but for which we have no empirical justification.
and Steven Kosslyn have argued that specific cognitive representation (propositional and
analogical representations respectively) make distinct and empirically testable predictions.
A question which arises from the indistinguishability of propositional and first-order knowledge
representations concerns whether human knowledge is encoded declaratively or procedurally
and whether the cognitive implementations of these two types of memory are different. The
encoding and retrieval procedures for these knowledge types have been claimed to involve
different mechanisms, which are manifested in differences in people’s ability to introspect on
their knowledge, the occurrence of different retrieval errors, and so forth. If there are such
differences they are not evident in the COGENT model.
The distinction between procedural and declarative knowledge has found a role in
understanding other kinds of cognitive skills, and even that different representations may be
used at different stages in development and at different levels of expertise. For example
Karmiloff-Smith (1992) has argued that development, and perhaps learning, involves a
process of “representational redescription”, whereby an initial procedural representation is
revised as learning progresses and is eventually replaced by a declarative representation as
mastery in the domain is attained. Recalling some of the a priori advantages of the first-order
representation this issue might have important theoretical implications for our understanding
of human decision making, and practical consequences for training decision makers.
On a somewhat related point Hastie and Pennington suggest that “special case” knowledge is
more frequently acquired than abstract, generalised knowledge. We are inclined to agree with
this, but any conclusion that this is a fundamental property of human cognition does not
necessarily follow. Hastie and Pennington themselves accept that “deliberately trained
expertise” may well include abstractions. In the light of the possible indistinguishability of the
propositional and first-order models we need to be cautious about strong claims regarding
whether decision makers naturally use, or eschew, abstractions. The empirical observation of
a bias towards concrete representations may be correct, but that might be an artifact of the
learning process or other aspects of experience. We tend to acquire specific knowledge in
specific situations, of course, but often we simply do not have the time, inclination or
sufficiently varied examples to discover any generalisations that may exist. This would be a
much more modest conclusion than a claim that human decision processes are not naturally
organised for exploiting abstractions.
Finally we suggest that computational modelling tools like COGENT can make a significant
contribution to the understanding of decision making, particularly complex, practical decision
making. Psychological processes are subtle and difficult to understand, and the behaviour we
observe is often the expression of many interacting processes. As a consequence theories
are often based on relatively simple experimental manipulations, whose analysis is
mathematically tractable, but the conclusions may not extrapolate to complex, real world
tasks. Among our own objectives are the desire to develop a general theory of complex
decision making, which can have practical value. The particular “real world” that we are
primarily concerned with is medicine; the problems we wish to address include: how to design
computer systems to help medical professionals in their routine decision making and patient
management (e.g. Fox and Das, 1996) and how to communicate information about risks and
uncertainties in ways that people can understand (e.g. Fox et al, 1995). Our experience to
date suggests that these design objectives are considerably helped by insights into how
people make decisions and not just what decisions they will take.
5. Conclusions
An eclectic approach to theories of human judgement and decision making is needed,
combining insights from the algebraic, representational and information processing
perspectives. We agree with Busemeyer et al’s observation that “cognitive scientists generally
find that decision theories fail to provide sufficient depth of explanation, and decision theorists
generally find that cognitive theories are too complex”. We hope that the methods and results
presented here provide some support for our belief that cognitive modelling in general, and
tools like COGENT in particular, can help to ameliorate such problems, and bring the JDM
communities closer together.
6. References
Beach L R Image theory: decision making in personal and organizational contexts.
Chichester, England: John Wiley, 1990.
Busemeyer J, Hastie R and Medin D L (eds) Decision making from a cognitive perspective
San Diego: Academic Press, 1995.
Cooper R C and Shallice T, “Soar and the case for unified theories of cognition”, Cognition,
55(2), 115-149, 1995.
Cooper R C, Fox J, Farringdon J and Shallice T “A systematic methodology for cognitive
modelling” Artificial Intelligence Journal, August 1996 (in press)
Fishburn P C “Normative theories of decision making under risk and under uncertainty” In
Bell, Raife and Tversky (eds) Decision Making, Cambridge: Cambridge University
Press,1988
Fox J “Making decisions under the influence of memory” Psychological Review, 1980
Fox J “Automating assistance for safety critical decisions” Phil. Trans. Roy. Soc. Lond. B,
327, 447-593, 1990.
Fox J “On the necessity of probability: reasons to believe and grounds for doubt” in G Wright
and P Ayton (eds) Subjective Probability, Chichester: John Wiley, 1994.
Fox J, Clark D A, Glowinski A J and O’Neil M “Using predicate logic to integrate qualitative
reasoning and classical decision theory” IEEE Trans. Systems, Man and Cybernetics,
20, 347-357, 1990.
Fox J and Das S “A unified framework for hypothetical and practical reasoning (2): lessons
from medical applications” in D M Gabbay and H J Ohlbach (eds) Practical
Reasoning, Berlin: Springer Verlag, 1996.
Fox J, Hardman D, Krause P, Ayton P, Judson, P “Risk assessment and communication: a
a cognitive engineering approach” in A Macintosh and C Cooper (eds) Applications
and innovations in expert systems III, SGES Publications: ISBN 1 899621 03 2, 1995
Gigerenzer G “Why the distinction between single-event probabilities and frequencies is
important for psychology (and vice versa)” in G Wright and P Ayton (eds)
Subjective Probability, Chichester: John Wiley, 1994.
Hastie R and Pennington N “Cognitive approaches to judgment and decision making” in
Busemeyer J, Hastie R and Medin D L (eds) Decision making from a cognitive
perspective, San Diego: Academic Press, 1995.
Karmiloff-Smith A “Beyond modularity: a developmental perspective on cognitive science”
Cambridge, MA: MIT Press.
Shanteau J “Psychological characteristics of expert decision makers” in J Mumpower (ed)
Expert judgement and expert systems, NATO ASI Series, vol F35, 1987.
Wallsten, T S (ed) Cognitive processes in choice and decision behavior, Hillsdale NJ:
Erlbaum, 1980.
Weber, E U, Goldstein W M and Barlas S “And let us not forget memory: the role of memory
processes and techniques in the study of judgement and choice” in Busemeyer J,
Hastie R and Medin D L (eds) Decision making from a cognitive perspective
San Diego: Academic Press, 1995.