Analysis of Complexity in
Cognitive Tasks
Graeme S. Halford
School of Psychology
University of Queensland
gsh@psy.uq.edu.au
Acknowledgments

This work has been developed in collaboration
with Bill Wilson, Steve Phillips, Glenda Andrews,
John Bain, Andrew Neal, Murray Maybery, Keith
Holyoak, Thomas Suddendorf, Paula Irving,
Christine Boag, Julie McCredden, Damian Birney,
Geoffrey Goodwin, and many other people have
contributed in some way. Their contributions are
gratefully acknowledged. We also acknowledge
the contribution of Air Services Australia to our
analyses of air traffic control situations.

The problem of what makes tasks complex occurs
in many areas of cognitive psychology:
–
–
–
–

cognitive development
mathematics and science education
animal cognition
industrial situations (e.g. air traffic control)
Comparison of task difficulties depends on solving
this problem

A transitive inference problem such as the
following might seem rather simple:
– Tom is happier than Mark
– Peter is happier than Tom
– Who is happiest?

To answer this we might order the elements:
Peter
Tom
Peter
Mark
Tom
Tom
Mark



Integration of premises produces a measurable
processing load effect.
What is the source of this?
Answer: We have to consider both premises to
assign an element to a slot.
Peter
Tom
Peter
Mark
Tom
Tom
Mark


How do we quantify this load?
Premise integration entails assigning entities to
three slots.
– It relates three variables
– The task entails processing a ternary relation.

We can quantify complexity of tasks in terms of
the relations they entail.
– More complex tasks entail processing more complex
relations.

There are three requirements to establish its
validity:
Requirements of relational
complexity theory:
1. Tasks with the same complexity should be
similarly difficult.
2. Tasks with lower levels of relational complexity
should be simpler, while tasks with higher levels of
relational complexity should be harder, given that
materials, procedure and knowledge are constant
over levels.
3. Performance should be predicted by tasks of the
same level of relational complexity from different
domains.

Task difficulty can vary for many reasons:
– declarative and procedural knowledge
– method of presentation, and strategy adopted.



Transitivity is one of a set of tasks that are difficult for
young children.
The causes of this have been controversial, but it is fair to
say that, when allowance has been made for the many
hypotheses in the literature, there is a component of the
difficulty that requires to be explained.
Our research suggests that the complexity that is inherent
in the task is one source of this difficulty.

Another task that has been similarly difficult is class
inclusion.

The classical question is to ask children: are there
more blue things or more circles?
Children under about 5 years typically say there are
more circles. There are a number of known sources
of false negatives, but when these are corrected (e.g.
using the procedures of Hodkin, 1987 or Halford &
Leitch, 1989) the task is still difficult for young
children.


What is the cause of this difficulty?
Blue
things
Superordinate
class
Included
In
Subordinate
classes
Included
In
Circles
Triangles
Complement
Of
•There is a superordinate class (blue), a subordinate class (circles) and a
complementary subordinate class (triangles).
•Class inclusion, like transitivity, is a ternary relation.
• The three classes must be assigned to the appropriate slots in the hierarchy.
•Blue is assigned to the superordinate slot because it includes triangles and
circles.
•This imposes a processing load, that has been verified by Halford & Leitch
(1989).

Transitivity and class inclusion are superficially different,
yet both entail ternary relations:
Transitivity
>
A
Class
Inclusion
B
>
>
C
Fruit
Included-in
Apples
Included-in
Non-Apples
Complement-of
• Transitivity and class inclusion were both originally Piagetian
tasks, but
• Other paradigms that entail ternary relations and are similarly
difficult for young children: E.g. concept of mind.
• The the relation
between a property of an
object, and the person’s
percept, is modulated by
a third variable, the
viewing condition.
 The
concept of mind
task entails relating three
variables, and is ternary
relational. Consequently
a structural complexity
effect was predicted
ahead of evidence
(Halford, Children’s
Understanding: The
Development of Mental
Models, 1993) and has
since been confirmed.



Concerning the second requirement for
relational complexity theory, if we can
reduce the relational complexity of a task,
holding materials and procedure constant,
then difficulty should be correspondingly
reduced.
We have done this successfully in several
concept domains.
The procedure for transitive inference is
shown in the following Figure:
Transitive Inference
Premises
blue
red
yellow
green
purple
blue
green
red
green
green
red
blue
Binary
red
blue
Ternary
Premises
blue
red
yellow
green
purple
blue
green
red
green
green
red
blue
Binary

red
blue
Ternary
The ternary relational task is to first place green and blue. It has to be
inferred that green is above red and red is above blue, therefore the order
is green, red, blue. This is a ternary relation. Once green and blue are in
place, the child is asked to place red. This is very easy if the correct order
has already been mentally constructed, and serves as a check on guessing
or other lower level strategies.
Premises
blue
red
yellow
green
purple
blue
green
red
green
green
red
blue
Binary

red
blue
Ternary
The binary relational task is to first place green and red. This can be
inferred from the single premise green above red, and entails processing
only a binary relation. Then the child is asked to place blue. This can be
done by processing the binary relation, red above blue.
Premises
blue
red
yellow
green
purple
blue
green
red
green
green
red
blue
Binary

red
blue
Ternary
Notice that binary and ternary tasks entail placing the same three blocks,
using the same set of premises. The percentages of children at each age
succeeding at the .05 level are shown in the following figure:
Children’s performance:
transitive inference
120
100
95.8
93.3
100
96.7
83.3
86.7
80
Percent
children 60
succeeding
40
71.4
66.7
Binary
Ternary
46.7
20
6.4
0
4
5
6
Age (years)
7
8



The comparatively small change in procedure
produced a dramatic difference in performance,
especially for the younger children (Andrews, 1997).
Relational complexity produces powerful effects on
performance.
Relational complexity can also be manipulated with
classification tasks.
The category induction task (Gelman & Markman,
1987) is binary relational, because it entails a
category and its complement. If we tell a child that
(say) cats climb trees, and ask them whether the
attribute tree-climbing applies to instances of a
complementary category. The structures of category
induction and class inclusion are in this figure:
Structures of category induction and class
inclusion
Complementary
Basic
Basic
A
Complementary
Subordinate
Subordinate
Cold water fish:
have bones
Salmon: Swim
upstream
B
Sharks
Trout: Have
dark flesh
Assessing Category Induction
and Class Inclusion


To make them comparable, the tasks have to be
assessed by the same method. Both can be
assessed by property inference (Johnson, Scott &
Mervis, 1997).
We used familiar categories, but unfamiliar
attributes (Halford, Andrews & Jensen, 1998).
Within-level questions
Cold water fish:
have bones
Salmon: Swim
upstream
2. Basic to basic:
Would this cold-water fish
have bones?
Sharks
Trout: Have
dark flesh
1. Basic to
complementary-basic:
Do all sharks have bones?
3. Subordinate to
complementary-subordinate:
4. Subordinate to
subordinate:
Do all trout swim upstream?
Would this salmon swim
upstream?
Within-level questions
Cold water fish:
have bones
Salmon: Swim
upstream
2. Basic to basic:
Would this cold-water fish
have bones?
Sharks
Trout: Have
dark flesh
1. Basic to
complementary-basic:
Do all sharks have bones?
3. Subordinate to
complementary-subordinate:
4. Subordinate to
subordinate:
Do all trout swim upstream?
Would this salmon swim
upstream?
Predictions: Relational Complexity Theory

Relational complexity theory predicts that
category induction should be easier than class
inclusion. To test this hypothesis, the tasks must
be assessed by procedures that are strictly
comparable. This can be done by property
inference:
Predictions: Relational Complexity
Theory
Cold water fish:
have bones
Salmon: Swim
upstream
Subordinate to complementarysubordinate:
Do all trout swim upstream?



Sharks
Trout: Have
dark flesh
Subordinate to subordinate:
Would this salmon swim
upstream?
Property Inference 1: An attribute should generalise more to instances
of the same category than to instances of a complementary category
(within level inference): e.g. swimming upstream is more likely to be
true of other salmon…….,
than of non-salmon (trout).
Getting both these questions right should indicate understanding the
relation between a category and its complement - a binary relation.
Between - level questions
Cold water fish:
have bones
Salmon: Swim
upstream
Sharks
Trout: Have
dark flesh
1. Basic to subordinate:
Do all salmon have bones?
2. Subordinate to basic: Do
all coldwater fish swim
upstream?
3. Basic to complementarysubordinate:
4. Complementarysubordinate to basic:
Do all trout have bones?
Do all coldwater fish have
dark flesh?
Between - level questions
Cold water fish:
have bones
Salmon: Swim
upstream
Sharks
Trout: Have
dark flesh
1. Basic to subordinate:
Do all salmon have bones?
2. Subordinate to basic: Do
all coldwater fish swim
upstream?
3. Basic to complementarysubordinate:
4. Complementarysubordinate to basic:
Do all trout have bones?
Do all coldwater fish have
dark flesh?
Predictions: Relational Complexity Theory
Cold water fish:
have bones
Salmon: Swim
upstream
1. Basic to subordinate:
Do all salmon have bones?
Sharks
Trout: Have
dark flesh
2. Subordinate to basic: Do all
coldwater fish swim upstream?

Property Inference 2: An attribute should generalise more to instances of
a lower level category than to instances of a higher level category: e.g.
attributes of coldwater fish should generalise to salmon.

But not necessarily the reverse.

Getting both these questions right should indicate understanding of
relations between hierarchical categories.
This is a between-level inference, and corresponds to a ternary relation.
The task is designed so questions are matched within and between levels.






The hypothesis that ternary relational reasoning would
be harder than binary relational was confirmed in two
experiments.
The third requirement for relational complexity theory
is that performance should be predicted by tasks of the
same relational complexity in other domains.
We have shown this to be true in numerous experiments.
In the category experiment just outlined, we found ternary
relational performance was predicted by transitivity and
class inclusion, both ternary relational tasks, the first from
a different domain.
In another study we obtained similar results for concept of
mind, which is also ternary relational, as noted above.
In other experiments we have found that relational
complexity scores in 6 domains loaded on a single factor
that accounted for 80 percent of the age related variance in
fluid intelligence (Andrews & Halford, submitted).
Relational Complexity Metric




The essence of our theory is that complexity of relations
processed in the same decision (in parallel) is a good way
to define cognitive complexity.
Relational complexity refers to the number of entities
related. It corresponds to number of slots or arity of
relations.
A binary relation has two slots: e.g.
Larger-than(_______, _______) has a slot for a larger
entity and one for a smaller entity. Each slot can be filled
in a variety of ways:
Larger-than(elephant, mouse)
Larger-than(mountain, molehill)
Larger-than(ocean-liner, rowing-boat)
Complexity of relations can be defined by the number
of slots:
Unary relations have one slot: e.g. class membership, as
in dog(Fido).
Binary relations have two slots: e.g. larger(elephant,
mouse).
Ternary relations have three slots: e.g. addition(2,3,5).
Quaternary relations: e.g. proportion (2/3 = 6/9).


A slot corresponds to a variable or dimension.
An N-ary relation is a set of points in N-dimensional space.






The relational complexity metric is very general. It has been applied to
– child and adult cognition
– higher animals such as chimpanzees
– is currently being applied to dolphins.
– industrial contexts, including air traffic control.
Relational complexity theory encompasses declarative and procedural
knowledge
Processing capacity is an enabling factor, but concept acquisition is
knowledge acquisition, and cognitive development is experience
driven.
There is a broad correspondence between levels of relational
complexity and the phenomena that Piaget attributed to stages
However, there is no suggestion that concepts of a given level are all
acquired synchronously. Concept acquisition is a function of
experience, given that the relevant processing capacity is available.
Relational complexity applies to explicitly represented relations. It
does not apply to tasks that can be processed by perceptual or
associative processes, such as transitivity of choice (Chalmers &
McGonigle, 1984; Wynne, 1995).
Capacity limits

A quaternary relation is the most complex that
adults can process in parallel
– (though a minority of people can probably process
quinary relations under optimal conditions)

This is a soft limit
– Processing capacity is not all or none
– Increased complexity produces increased errors and
decision times, rather than sudden catastrophic failure.

In order to handle more complex concepts,
mechanisms for reducing processing loads are
required. These are:

Conceptual chunking involves recoding concepts into
less complex relations. However there is a temporary
loss of access to chunked relations.
– For example, speed = distance/time, is a ternary relation,
but speed can be recoded into a unary relation,
speed(60kph) as when speed is indicated by the position of
a pointer on a dial.
– However the chunked representation does not permit us to
answer questions such as “How does speed change if we
cover the same distance in half the time?”

Segmentation entails breaking tasks into less complex
steps, which can be processed serially. Strategies and
algorithms are common ways of doing this:
– e.g. adding one column at a time in multidigit addition.

Chunking and segmentation skills are important
components of expertise.




Complexity estimates are based on the mental models used
to represent a concept, taking chunking and segmentation into
account.
Variables cannot be chunked if relations between them must be
considered.
Assumptions about chunking and segmentation are made
explicit and applied consistently across domains.
A useful heuristic is that relational complexity cannot be
reduced if the variables interact. This is analogous to analysis
of variance:
– Interacting variables must be interpreted jointly.
– A mathematical procedure for determining effective relational
complexity is described by Halford et al. (Behavioral & Brain Sciences,
1998, 21(6), 803-864, Section 3.4.3). If a relation can be decomposed
into simpler relations, then recomposed without loss of information,
effective complexity is equivalent to the less complex relation.
Applications of relational
complexity theory



Now we consider some applications of the theory,
beginning with work on air traffic control in
collaboration with Christine Boag and Andrew
Neal.
Two air traffic control situations are shown. The
first has fewer aircraft, but was reported as a
heavy load by the controller. The second contains
a lot of aircraft, but is a simple situation for a
controller.
The first situation entails more complex relations
between aircraft.
Display A
AERODROME A
MEL
TJS
KRD
LAO
AERODROME D
AERODROME B
AERODROME C
TMK
RDS
TO
AERODROME E
Display B
LAT
AERODROME A
TAK
DJT
SRD
AERODROME D
WEF
AERODROME B
KAL
SDK
AERODROME C
TJT
EWF
MTJ
RMS
TWE
TO
AERODORME E

The theory has also been applied to data from
chimpanzees. The relational match-to-sample task
requires participants to discriminate the relation
same/different between a pair of objects.
Sample
Choices


XX
YY
WY
Choice of YY is correct.
Chimpanzees succeed, if they have been taught symbols for
the relations.
– This indicates they can process binary relations.
– We have work in progress with dolphins on this task, in
collaboration with Thomas Suddendorf and Paula Irving.
Cardinality

Understanding that counting yields the
cardinal value of the set requires a mental
model that incorporates the principles of
counting.
Successor Model

In the successor model, numerals are assigned to
objects.
next
numbers
one
next
two
next
three
next
four
items
next
next
next
(a) Successor Model
next
five
Sets Model

In the sets model, numerals are assigned to sets,
but it does not distinguish items that have been
counted from those that have not.
next
numbers
one
next
two
next
three
next
four
sets
large
r
large
large
r
r
(b) Sets Model
large
r
five
Inclusion Model

In the inclusion model numbers are assigned to sets, which
include (a) an item not previously counted, and (b) the set of all
items previously counted (this set is empty on the first item).
next
numbers
one
next
two
next
next
three
four
five
sets
included in
included in
(c) Inclusion Model
included in
included in


The inclusion model enables recognition that the last number represents the
cardinal value of the set - the cardinal word principle.
Also, because it distinguishes previously counted objects from the object
currently counted, it makes the order in which objects are counted irrelevant.
next
numbers
one
next
two
next
next
three
four
five
sets
included in
included in
(c) Inclusion Model
included in
included in

Understanding cardinality was assessed using
three question types:
How many: Children should say the last number they
mentioned in counting S, and should not recount.
Show me X: Children should indicate all the objects in
the set, not just the last object.
Reverse count: Asking children how many there would
be if counted in the opposite direction.
Fractions: Quaternary Relation

Fractions and proportions cause difficulty for
young children
– (for reasons that never appear to have been completely
explained)

Proportion is a quaternary relation.
Fractions: Quaternary relation
1
3
=
2
6
1
4

<
2
6
1
3
<
3
6
5
5
>
7
8


The examples alongside show
that comparison of fractions
entails working with four
dimensions, because a change in
either numerator or either
denominator can affect the
comparison.
Comparison of integers requires
working with only two
dimensions.
The powerful effect of structural
complexity has been largely
overlooked.

In order not to be misunderstood:
– There is nothing to prevent younger children from
understanding fraction concepts that entail simpler relations.
(e.g. half a pie entails only binary relations, and should be
understandable by two year olds).
– Notice that the predicted age is actually earlier than appears to
have been observed.

A good relational complexity analysis frequently
uncovers unrecognised potential;
– e.g. it predicts that even two year olds should be able to make
balance scale judgements based on either weight or distance,
though not both.
– This is more optimistic than current norms, and has been
confirmed empirically (Halford& Dalton, 1995)

Relational Complexity Metric
Piagetian Stage
Cognitive Processes
Typical Tasks
Early Sensori-motor
No Accessible representation
(Elemental association)
Conditioning,
linear transitivity of choice
PDP Implementation
Input
Sensori-motor
Computed,
Non-structural (configural
association)
Conditional discrimination,
prototype,
circular transitivity
Input
Output
Hidden
Output
A rg ( ta il -w a g g in g )
Preconceptual
Intuitive
Unary Relations
Binary Relations,
Univariate functions
Match-to-sample,
identity position integration,
category label distinct from category
R
(d o g )
Relational match to sample,
A not-B,
complementary categories
lo v
R (
e s)
A r g2
A r g1
( j a ne )
( j o e)
Concrete Operational
Ternary Relations,
Binary Operations,
Bivariate Functions
Transitive inference,
hierarchical categories,
concept of mind
at
Re l
i on
Ar g2
Formal
Quaternary Relations,
composition of binary operation
Proportion, Balance scale
Quinary Relations
Doubly-centre embedded sentences,
Tower of Hanoi
Ar g1
Ar g n


The relational complexity metric forms a single scale from
the most basic psychological process, elemental
association, to the most complex level of which humans
are capable.
Sample concepts at each level are shown, together with:
– the corresponding Piagetian stage and:
– the neural net architecture that is characteristic of each level.



There is a unique empirical indicator for each level, and
each level imposes requirements on neural net models,
over and above requirements of lower levels.
This provides converging evidence that the levels are
genuinely distinct.

A complexity metric is essential for orderly
interpretations of findings in Psychology.
– Without it, questions such as whether infants or animals
have similar cognitive capabilities to adult humans are
inherently unanswerable.


To investigate these questions systematically, we
need ways of equating, or differentiating between,
levels of cognitive functioning.
We suggest that the relational complexity theory
makes a start in providing such a metric.
Abstract
Techniques are outlined for analysis of cognitive complexity in general
cognition, cognitive development, mathematics education, reasoning tasks,
psychometric test items, and industrial decision making, especially command
and control. Complexity in cognitive tasks can be assessed by analysing the
number of entities that have to be related in a single representation. This
corresponds to the number of interacting variables that are processed in
parallel. Adult humans are typically limited to processing quaternary relations
(I.e. to relating four entities, or four variables) in parallel. More complex
concepts are processed by segmentation (breaking a task into components
small enough not to exceed capacity, and which are processed serially) or
conceptual chunking (collapsing to a smaller number of variables, which
reduces processing load, but makes some relations inaccessible). Processing
load analyses are based on the principle that interacting variables cannot be
processed serially. Neural net models of higher cognitive processes offer
explanations for processing loads. Complexity analysis also leads to
discovery of new capabilities. A single complexity metric is presented for all
levels of cognition.