A Conceptual Complexity Metric Based on Representational Rank
Graeme S. Halford
Psychology
University of Queensland
Queensland 4072 Australia
gsh@psy.uq.edu.au
William H. Wilson
Computer Science & Engineering
University of New South Wales
Sydney NSW 2052 Australia
billw@cse.unsw.edu.au
Steven Phillips
Information Science Division
Electrotechnical Laboratory
1-1-4 Umezono, Tsukuba 305 Japan
stevep@etl.go.jp
Abstract
A conceptual complexity metric based on representational rank is proposed. Rank is the
number of entities that are bound into a representation, and is related to the number of
dimensions, which is a measure of complexity. Each rank corresponds to a class of neural
nets. The ranks and typical concepts which belong to them, are: Rank 0, elemental
association; Rank 1, content-specific representations and configural associations; Rank 2,
unary relations, class membership, variable-constant bindings; Rank 3, binary relations,
proportional analogies; Rank 4, ternary relations, transitivity and hierarchical classification;
Rank 5, quaternary relations, proportion and the balance scale. Rank 6, quinary relations.
Rank 0 can be performed by 2-layered nets, rank 1 by 3-layered nets, and ranks 2-6 by tensor
products of the corresponding number of vectors. All animals with nervous systems perform
rank 0, vertebrates perform rank 1, primates perform rank 2-3, but ranks 4-6 are uniquely
human. Rank also increases with age.
We want to propose a theory of conceptual complexity in cognition of humans and higher
animals, and to show how properties of major classes of cognitive tasks can be derived from the
theory. There are systematic differences between psychological processes depending on their
complexity. For example, tasks that are performed by using basic processes such as association have
different properties from tasks that require higher cognitive processes.
Representational rank corresponds to the number of components of a representation, given that
the components retain their identity when bound in a more complex representation. The ranks are
summarised in Figure 1. Each rank comprises an equivalence class of cognitive processes of equal
structural complexity, and higher ranks correspond to more complex tasks. Each representational
rank can be identified with empirical indicators, and with a class of neural nets which can be used to
predict properties of the levels. Ranks 0 and 1 are associative while Ranks 2-6 meet the criteria for
symbolic thought (Halford, Wilson, & Phillips, submitted; Phillips & Halford, in press; Phillips,
Halford, & Wilson, 1995).
Rank 0 corresponds to Elemental associations, which comprise links between pairs of entities:
E1  E2
They are Rank 0 because they can be modelled by nets without representation other than input
and output, and they can be implemented by 2-layered nets. Rank 0 can be assessed by any
associative learning test, and can be performed by virtually all animals with nervous systems.
Rank 1 corresponds to Configural associations, which entail links in which one cue is
modified by another. They have the form:
E1, E2  E3
Rank 1 would be indicated by configural learning, without isomorphic transfer, by
representation of objects, without symbolic representation, and by inferences that do not go beyond
the immediate spatio-temporal frame. This level of performance has been demonstrated in a number
of mammals (Pearce, 1994; Rescorla, Grau, & Durlach, 1985; Rudy & Sutherland, 1989) and may
also be possible in other vertebrates. It can be modeled by 3-layered nets.
Figure 1. Ranks 0-6, with schematic neural
nets.
Rank 2 corresponds to unary
relations which are the lowest level of
relational representation. A unary relation
R(x), is a binding between a relation
symbol and an argument symbol. In
Figure 1 the input and output layers are
omitted for Ranks 2-6, and only the
representation is shown. As the
representation
comprises
two
components, which retain their identity
when composed into a more complex
structure, it is Rank 2. An example would
be the proposition happy(John). This
consists of a binding between happy and
John. In the neural net model of Halford
et al. it would be represented as
VhappyVJohn The transition from Rank 1
to Rank 2 can be envisaged by imagining
the hidden layer at Rank 1 (Figure 1)
being divided into two components which
are then connected so as to form a matrix
as shown for Rank 2.
Retrieval
is
performed
by
computing the dot product of the
representation with a retrieval cue. If the
retrieval cue is happy the output is John,
equivalent to asking “Who is happy?” If
the retrieval cue is John the output is
happy , equivalent to asking “What is
John’s state?” Thus the representation has
the omni-directional access property of
relational
knowledge,
which
is
considered basic to higher cognition
(Halford et al., submitted; Phillips et al.,
1995; Wilson, Halford, & Phillips,
submitted). These retrieval operations are
implemented mathematically by Vhappy
*(VhappyVJohn) = VJohn and VJohn
*(VhappyVJohn) = Vhappy, where * is the
inner product.
Rank 2 would be indicated by a
performance that entailed representing
the binding between an object and its
location (failure to do this results in the A
not-B error, as explained in Halford,
3
1993, pp. 52-54, 56), by symbolic representation of object, and requiring participants to
category membership, which is indicated by two choose between an apple and a pear, then
performances, understanding word reference, and transferring to an isomorphic task in
isomorphic transfer of match-to-sample. The latter is
demonstratedby showing (say) an apple as sample
which (say) the sample is a hammer, and the choices are a banana and a hammer. Transfer of matchto-sample implies they can represent a binding between a category representation (e.g. hammer) and
representation of an object. The task can be performed by higher monkeys (Premack, 1983) and can
be represented by a Rank 2 tensor product.
Rank 3 corresponds to binary relations, which represent common states and actions in the
world, such as larger(whale,dolphin), or loves(Joe,Jenny). They can also be interpreted as a
univariate functions, f(a) = b, or as unary operators; e.g. change-sign comprises the set of ordered
pairs {(x, -x)}. Rank 3 would be indicated by proportional analogies A:B::C:D with binary relations.
Rank 3 appears to have been demonstrated with chimpanzees (Premack, 1983) using binary
relational match-to-sample task, shown in Figure 2. This requires choice of a pair of objects that has
the same relation as the sample (e.g. if the sample is XX, they should choose AA rather than BC. If
the sample is XY, they should choose BC rather than AA).
RELATIONAL MATCH-T0-SAMPLE
SAMPLE:
XX
CHOICE*:
YY
XY
Analogy: O-same(XX) maps to O-same(YY)
CHOICE*:
YY
XY
AB
Analogy: O-different(XY) maps to O-different(AB)
*Correct choice underlined
Figure 2. Relational match-to-sample task.
This implies a form of analogical reasoning based on binary relations, a Rank 3 representation
(Halford et al., submitted; Holyoak & Thagard, 1995). Other indicators include isomorphic transfer
of 2-object discrimination (object discrimination learning set), and discrimination based on weight or
distance (but not both) in the balance scale (Halford & Dalton, 1995).
Rank 4 corresponds to ternary relations such as “love-triangle”, which is a relation between
three people. They can be interpreted as bivariate functions, and binary operations. For example, the
binary operation of arithmetic addition consists of the set of ordered triples of +{. . , (3,2,5), . . ,
(5,3,8), . . , . . } and is a ternary relation. Many cognitive tasks that cause difficulty for young
children, including transitivity and class inclusion, are ternary relations (Halford, 1992; Halford,
1993; Halford et al., submitted). Configural learning tasks are ternary relations, in that they consist of
sets of ordered triples: e.g. conditional discrimination tasks, in which cue-response contingencies
depend on background consist of triples of the form (cue,background,response). Because transfer
between isomorphs requires representation of structure, then isomorphic transfer of conditional
discrimination would constitute a test for ternary relations. In this way relatively high level symbolic
4
competence can be tested using an object-discrimination task which is procedurally comparable to
those used for configural association.
Transitivity and class inclusion are superficially dissimilar, to each other and to the other
concepts we have been considering, yet at the relational level they share some important properties.
The essence of transitivity is to integrate the premises into an ordered triple. For example, given the
premises: Bill is happier than Peter, Peter is happier than Tom, most of us would arrange Bill, Peter,
Tom from left to right, or top to bottom. Integrating the premises produces a processing load,
corresponding to a subjective experience of effort caused by the need to consider both premises
jointly (Maybery, Bain, & Halford, 1986).
Class inclusion entails representing the relations between the superordinate set B and the
complementary subsets, A and A'. Given the problem in which apples and pears are included in fruit,
the important thing is to identify which set is the superordinate. Fruit is not inherently a
superordinate. Had the problem been fruit and meat included in food, fruit would have been a
subordinate. Fruit is a superordinate because it includes two subordinate sets, apples and non-apples.
Similarly, apples is a subordinate because it is included in fruit, and there is at least one other class
that is included in fruit. The superordinate is identified by its relations to the other two sets. This
entails processing a ternary relation.
Transitivity and class inclusion have a common degree of structural complexity, because the
core of both is to process ternary relations. This imposes a high processing load for both children and
adults. Relational complexity is a major cause of the difficulty that children have with both these
concepts, and with many others that entail processing ternary relations (Halford, 1993; Halford et al.,
submitted).
In transitivity we have found that three- and four-year old children perform at near ceiling level
when they only have to process one binary relation at a time. However when they have to integrate
two binary relations into a ternary relation, their performance drops to approximately chance level.
Other aspects of the task were held constant in these comparisons (Andrews, 1996; Andrews &
Halford, submitted).
Rank 5 corresponds to quaternary relations. Proportion, a/b = c/d, and comparison of moments
on the balance scale (McClelland, 1995; Siegler, 1981) is a quaternary relation. The complexity of
quaternary relations is illustrated by Figure 3, where four dimensions have to be considered to decide
if a proportion holds.
1
3
--=
--2
6
1
--2
1
--3
5
---
<
4
--6
<
3
--6
>
5
---
5
7
8
Figure 3. Comparisons between fractions, illustrating that both numerators and both
denominators must be processed, entailing representation interaction between four dimensions.
Rank 6 corresponds to quinary relations, but there are no good tests for this level at present.
Rank and processing capacity
Higher rank tasks impose higher processing loads (Halford, Maybery, & Bain, 1986; Halford et
al., submitted; Maybery et al., 1986). Empirical evidence (Halford, 1993; Halford et al., 1994;
Halford et al., submitted) indicates that human adults are normally restricted to processing quaternary
relations in parallel (i.e. relating four variables in parallel), and that only a minority process quinary
relations. Consequently, empirical tests for Rank 6 are not as well developed as for Ranks 0-5.
Representational rank increases with age, as follows: Rank 0 should be possible in utero when
the nervous system forms, Rank 1 is observed at 4 months, Rank 2 at 1 year, Rank 3 at 2 years, Rank
4 at 5 years, and Rank 5 at 11 years. The phenomena that Piaget attributed to stages correspond in
approximate fashion to the ranks in Figure 1 (Halford, 1993).
Concepts too complex to be processed in parallel (more than Rank 5 for most adults, on
existing evidence) are handled by segmentation (decomposition into smaller segments that can be
processed serially) and conceptual chunking (recoding representations into lower rank, but at the
cost of making some relations inaccessible). For example, velocity = distance/time, is a ternary
relation, and is Rank 4, but can be recoded to Rank 2, a binding between a variable and a constant
(Halford, 1993; Halford et al., 1994; Halford et al., submitted). Low ranks impose low processing
loads, which is why they tend to be effortless and parallel, but they are also inaccessible to other
cognitive processes. The reason higher cognitive tasks tend to be serial (Norman, 1986) is that they
entail high rank representations, which impose high processing loads that must be reduced by
segmentation.
Representational rank and neural nets
The ranks correspond to certain distinctions between neural net architectures. Two-layered nets
can process elemental associations but not configural associations. Three-layered nets can process
configural associations and can compute any computable function, but they do not typically embody
the criteria for relational knowledge. The most important limitation is that they cannot produce
transfer between isomorphs (Phillips & Halford, in press). The best way of representing relations in
neural nets is currently a major topic of research, but certain requirements can be specified: There
must be a relation symbol and a representation for each argument (entity that is related), these
representations should retain their identity when bound in a more complex structure, and properties
of relational representation, such as omni-directional access (Halford et al., submitted) must be
maintained. In our model as indicated above, and illustrated schematically in Figure 1, the relation
symbol and each argument are represented by vectors. A matrix of the vectors binds the
representation. This is usually achieved by computing the tensor product of the vectors. This model
does represent structure and can mediate isomorphic transfer.
Neural nets that represent relations incur a computational cost, and complexity analysis shows
that this cost increases exponentially with number of entities related. This offers a natural
explanation for the processing load observed in relational tasks, and for the limits to our processing
capacity.
Conclusion
6
The concept of representational rank provides a conceptual complexity metric, and helps to
clarify the difference between higher cognitive processes and more basic associative processes.
Higher cognitive processes are slow, serial, effortful, flexible, content independent, explicit,
cognitively accessible, and dynamic. Associative processes are fast, parallel, automatic, not under
strategic control, implicit, not cognitively accessible, and dependent on input. Yet, despite their
importance, the basis for these distinctions has never been clear.
We propose that the crucial factor is whether structure is represented and, if so, the complexity
of that structure. Ranks 0 and 1, corresponding to elemental and configural association, permit even
quite complex tasks to be performed, but there is no representation of structure. The clearest
objective indicator of this is failure of transfer to isomorphs. This is really a form of analogical
reasoning, and requires structure to be represented. Feedforward nets can model these processes but
do not mediate isomorphic transfer.
Structure is represented at Ranks 2-6, and rank provides a good measure of structural
complexity. Neural nets based on tensor products of vectors provide good models of these structures,
and provide a natural explanation for processing loads that have been observed empirically.
Representational rank is a metric for conceptual complexity, both in evolution and
development. Elemental association (Rank 0) is virtually universal to all animals with nervous
systems, while present evidence suggests that configural association (Rank 1) occurs only in higher
animals, and may be confined to vertebrates. Ranks 2-6 process explicit relations, are symbolic and
are confined to primates. The structures represented increase with phylogenetic level, so
chimpanzees can process rank 3 whereas no monkeys appear to perform above rank 2.
Representational rank increases with age in humans, and provides an indicator of development.
7
References
Andrews, G. (1996, August). Assessment of relational reasoning in children aged 4 to 8
years. Paper presented at the Conference of the International Society for Study of Behavioral
Development, Quebec City.
Andrews, G., & Halford, G. S. (submitted). Children's ability to make transitive inferences:
The importance of analogical mapping and structural complexity.
Halford, G. S. (1992). Analogical reasoning and conceptual complexity in cognitive
development. Human Development, 35, 193-217.
Halford, G. S. (1993). Children's understanding: the development of mental models.
Hillsdale, NJ: Erlbaum.
Halford, G. S., & Dalton, C. (1995). Performance on the balance scale by two-year old
children (ERIC Document ED 385 355).
Halford, G. S., Maybery, M. T., & Bain, J. D. (1986). Capacity limitations in children's
reasoning: A dual task approach. Child Development, 57, 616-627.
Halford, G. S., Wilson, W. H., Guo, J., Gayler, R. W., Wiles, J., & Stewart, J. E. M. (1994).
Connectionist implications for processing capacity limitations in analogies. In K. J. Holyoak & J.
Barnden (Eds.), Advances in connnectionist and neural computation theory, Vol. 2: Analogical
connections (pp. 363-415). Norwood, NJ: Ablex.
Halford, G. S., Wilson, W. H., & Phillips, S. (submitted). Processing capacity defined by
relational complexity: Implications for comparative, developmental, and cognitive psychology.
Holyoak, K. J., & Thagard, P. (1995). Mental leaps. Cambridge, MA: MIT Press.
Maybery, M. T., Bain, J. D., & Halford, G. S. (1986). Information processing demands of
transitive inference. Journal of Experimental Psychology: Learning, Memory and Cognition, 12(4),
600-613.
McClelland, J. L. (1995). A connectionist perspective on knowledge and development. In T.
Simon & G. S. Halford (Eds.), Developing Cognitive Competence: New Approaches to Cognitive
Modelling (pp. 157-204). Hillsdale, NJ: Erlbaum.
Norman, D. A. (1986). Reflections on cognition and parallel distributed processing. In J. L.
McClelland & D. E. Rumelhart (Eds.), Parallel distributed processing. Explorations in the
microstructure of cognition: Vol. 2. Psychological and biological models (pp. 531-546). Cambridge,
MA: MIT Press.
Pearce, J. M. (1994). Similarity and discrimination: A selective review and a connectionist
model. Psychological Review, 101(4), 587-607.
Phillips, S., & Halford, G. S. (in press). Systematicity: Psychological evidence with
connectionist implications. Proceedings of the Nineteenth Annual Conference of the Cognitive
Science Society.
Phillips, S., Halford, G. S., & Wilson, W. H. (1995, July). The processing of associations
versus the processing of relations and symbols: A systematic comparison. Proceedings of the
Seventeenth Annual Conference of the Cognitive Science Society, Pittsburgh, PA, 688-691.
Premack, D. (1983). The codes of man and beasts. The Behavioral and Brain Sciences, 6,
125-167.
Rescorla, R. A., Grau, J. W., & Durlach, P. J. (1985). Analysis of the unique cue in
configural discriminations. Journal of Experimental Psychology: Animal Behavior Processes, 11(3),
356-366.
8
Rudy, J. W., & Sutherland, R. J. (1989). The hippocampus is necessary for rats to learn and
remember configural discriminations. Behavioral Brain Research, 34, 97-109.
Siegler, R. S. (1981). Developmental sequences within and between concepts. Monographs
of the Society for Research in Child Development, 46, 1-84.
Wilson, W. H., Halford, G. S., & Phillips, S. (submitted). The properties of higher cognitive
processes and how they can be modelled in neural nets.
9