Identifying Repeated Patterns
of Behavior in Time
Magnus S. Magnusson
Research Professor
Human Behavior Laboratory
University of Iceland
www.hbl.hi.is
Life and Repetition
of Spatio/Temporal Patterns
Francis CRICK:
“Another key feature of
biology is the existence of
many identical examples
of complex structures.”

Some examples:
 DNA
–


(Crick and Watson discovered the
double helix structure of DNA.)
Behavior
–
(1989, p. 138.)
Words, gestures and
patterns of these
Clothing
–

Base pairs, genes,
chromosomes, genomes
Shoes, hats, coats,..
Urban environments
–
Houses, streets, cars,
shops, books, radios...
Behavior is Patterns
- often hidden patterns
“Behavior consists of patterns in
time. Investigations of behavior
deal with sequences that, in
contrast to bodily characteristics, are
not always visible.”
Opening words of Eibl-Eibesfeldt’s Ethology: The Biology of
Behavior, 1970, p. 1; {Emphasis added.}
Self-organization –
The Emergence of Patterns
Bénard cells
Visible or hidden
From Scott Kelso, 1997
“Emergence” often needs to be
assisted

“It is rarely, if ever, the case that the
appropriate notion of pattern is extracted
from the phenomenon itself using minimally
biased procedures. Briefly stated, in the
realm of pattern formation ‘patterns’ are
guessed and then verified.” Crutchfiled, J.,
1993. (Here cited from Solé & Goodwin,
2000, p. 20).
New Research Directions

Much theoretical and methodological thinking within
the behavioral sciences (and statistics) stems from
the time before cheap powerful computers and
advanced software development tools

Highly complex search patterns and algorithms can
now be developed and applied

Behavioral scientists can aim for new kinds of
discoveries
Basic Viewpoint and Task

Behavior is more structured than is perceived
directly or through standard data analysis methods

To fully disclose its structure new pattern types and
detection methods are needed to complement
existing ones

The discovery of hidden patterns is of considerable
importance for theoretical and practical reasons
Architechture vs. Structure
a simple philosophy

Search algorithms should correspond to the
structure of the phenomenon being studied

Even the most sophisticated and powerful
square detection algorithm is not adequate for
the detection of planetary orbits

An imperfect ellipse detection algorithm would
be preferable
Some Basic Questions




What kinds of hidden significant structure
exists in behavior?
How to characterize such structure?
How to discover such hidden structure?
How to discover effects of independent
variables on such structure?
Sequences and Patterns
A sequence:



“1. an arrangement of two or
more things in successive
order”
“3. an action or event that
follows another or others”
“Maths. a. an ordered set of
numbers or other
mathematical entities in oneto-one correspondence with
the integers 1 to n” - Collins.
A pattern:
 “1. an arrangement of
repeated or corresponding
parts, decorative motives
etc...”
 “Most mathematicians
define Mathematics as the
science of patterns….”
A pattern (shape, form) may not be a sequence but may still include
one. Detecting a pattern may thus mean detecting a sequence.
Behavior as Repeated Patterns

Linguistics: repeated hierarchical/syntactic patterns

Ethology:

Behaviorism: repeated real-time contingencies

Anthropology, Social Psychology and more: scripts, plans, routines,
repeated hierarchical/syntactic patterns
strategies, rituals, ceremonies, etc.

The importance of repeated patterns in behavior is widely accepted

The recognition of the “hiddenness” of some such patterns is needed

New pattern definitions with corresponding detection algorithms and
tools (software) are required
Verbal and Nonverbal are One

“The activity of man constitutes a structural whole, in
such a way that it cannot be subdivided into neat
“parts” or “levels” or “compartments” insulated in
character, content, and organization from other
behavior. Verbal and nonverbal activity is a unified
whole, and theory and methodology should be
organized or created to treat it as such.” Pike(1960,
p. 2). {Emphasis added.}
Repetition versus Uniqueness
“.. a conversation, … a complex system of
relationships which nonetheless may be
understood in terms of general principles
which are discoverable and generally
applicable, even though the course of any
specific encounter is unique (cf. Kendon
1963, Argyle and Kendon 1967).”
(Kendon, 1990, p. 4). (Emphasis added.)
Towards a General Pattern Type
Different Time Scales and Content
These patterns of patterns have aspects in common:
 How do you do?
 How do you do? Very well, thank you.
 Pass me the salt, Jack. Jack, passes the salt.
 If..then..else
 Dinner: Sit down..take an entrée..take a main
course..take dessert..drink coffee..stand up
 Rituals, ceremonies, routines, genes, poems,
hospital operations, conferences, classes, football
matches, strikes, and melodies
Common Structural Aspects

Fixed order and “significantly invariant
distances” between components

Hierarchical/syntactic structure

Self similarity / scale independence
Such Patterns are Often Hard to Spot
1
2
d ek w akb c d k w k d e wkakb ckd w
d ek w akb c d k w k d e wkakb ckd w
1 and 2 are identical data sets
T
T
A Dinner...a Time-Flexible Pattern of Patterns
The six sub-patterns are separated by relatively freely
structured intervals of relatively fixed length.
(From Magnusson, 2005.)

Patterns and Causation

Very well, thank you
–

an earlier word is usually not considered as a
cause of any word following it within such intraindividual patterns
How do you do? Very well, thank you
–
an earlier part of some inter-individual patterns
may be seen as a likely cause of a later part of the
same pattern
The T-patterns

A t-pattern is a particular set of event-types recurring in
the same order (and/or concurrently) with
“significantly similar distances” between them on a
single dimension.

T-patterns have a scale-independent and hierarchical
structure (often syntactically constrained -- “grammar”)

T-patterns may occur randomly, but they often occur in
cycles and even when their elements do not
The Data: Series of Points on 1-Dim.

An event-type may be an actor’s (agent’s) beginning or ending of a
particular behavior.

It may also be a base in a DNA molecule or an amino (recid.) in a
protein.
a
a
a
aa
a
Sets of such series form multivariate point series to
which all T-pattern definitions refer exclusively.
Multivariate Point Series
The Basic Data Type
A .
.
.
. .. .
.
..
B ..
.
.
. .
C
.
.
..
.
D
.
.
E .. .
. . .
....
. . . ..
F
.
.
G . ..
. .
.
.
.
_______________________________________
t1
t2
Recursive T-Pattern Definition
with the event-type as the simplest T-pattern
The T-pattern is an ordered set of T-patterns (X):
X1 ≈dt1 X2 ≈dt2 . . Xi-1 ≈dti-1 Xi . . ≈dtm-1 Xm
that recurs with significantly similar time distances,
≈dt, between its elements relative to the zero
hypothesis (fiction) of constant probability per unit
time for each Xi = NXi / observation time
Towards a Detection Algorithm
Searching for Critical Intervals [d1, d2]
Comparing Series A and B
Detected
Critical (distance)
Interval (window)
A
B
d1 d2
Repeatedly, an A is followed by a B within approximately the same distance
Critical Intervals and Binary Trees

Any T-pattern Q = X1 X2..Xm can be split into
a pair of shorter ones related by a critical
interval:

QLeft [d1, d2] QRight
Recursively, QLeft and QRight can each be split
until the whole pattern X1..Xm is expressed as
the terminals of a binary-tree
Bottom-up Detection
Patterns Grow & Compete

The bottom-up algorithm detects patterns gradually
from event types, as pairs of pairs, i.e., as binary trees

It detects critical interval relations between the
occurrence series of event types and/or already
detected patterns and then connects these to form
longer patterns (trees).

Many binary-trees may correspond fully or partly to the
same pattern so all detected patterns are automatically
compared and only the most complete (longest)
patterns are kept.
Completeness Competition partial and equivalent trees







(( A B) ((C D) (E F)))
(( A ((B
C) D)) (E F))
(( A B)
(D (E F ))
(( A
C)
F)
(B
E)
(B
D)
(A
F)
Behavior Record - Example
Two Children (Blue & Red) Play With One Toy for 13.5 Min; 81 Series
Time in 1/15 s
New Pattern Presentation
for Complex Patterns
Nature’s Symmetries are Approximate

Different instances of the same t-pattern may
have quite different internal intervals

Still the relationship is always the same, the
Critical Interval Relationship.

Should this be called relative translation
symmetry?
Statistical Validation
Statistical Validation Types
Standard Statistical Methods
Inadequate for T-pattern Detection

Multivariate statistical methods: look for clouds of points in ndimensional space rather than for syntactic structures on one
dimension

Time series analysis looks for trends or waves rather than
hierarchical discontinuities occurring irregularly

Sequential analysis may look for a priori unlikely time sequences
but involves no concept of complex repeated 1-D shapes or
patterns. It may therefore detect a multitude of sequential relations
without ever detecting such underlying patterns
25 min of Children’s Dyadic
Problem Solving
Data from published studies by Beaudichon and Magnusson.
Univariate Bursts
Doctor-patient facial interaction
data coded with FACS
Interindividual T-patterns in DoctorPatient Facial Interactions
Data from the Psychiatric Hopitals in Geneva, V. Haynal et al.
Wild-type Male vs. wild-type Mated Female
#P = 3
Wild-type Male vs. Mature wild-type Virgin Female
#P = 4
Wild-type Male vs. Immature wild-type Female
Drosophila
interactions
B. Arthur’s data
and patterns
#P = 8
The T-System or T-model
A System of Mathematically Defined Terms

The t-pattern type is the basis of a growing
system of terms for the description of temporal
structure in complex behavioral processes

Corresponding detection algorithms have been
developed and implemented in the THEME
software

See patternvision.com & noldus.com
Extending the T-model
Building on the Critical Interval and T-pattern Concepts








Markers & Indicators
Composition
+/- Associates; Satellites & taboos
Gravity- and repulsion zones
Packets
Packet markers
Drifters
T-kappa
T-markers

A t-marker of a t-pattern occurs almost exclusively as
a part of that pattern

A t-marker’s occurrence thus indicates that a
particular t-pattern is occurring

A marker that occurs early in a pattern predicts the
rest of the pattern

A marker occurring late in the pattern retrodicts the
earlier part of the pattern
T-associates

A positive or negative associate of a T-pattern is: some
behavior that is not a part of that pattern, but occurs
within or around its occurrences significantly more (or
less) often than expected by chance

Associates may occur only, always, sometimes or
even never within or near their corresponding T-pattern

The “only and always” case is called a t-satellite

The (almost) never case is called a t-taboo
The T-packet Structure
A T-pattern with its Associates and Zones

An instance of a t-packet showing two t-associate instances

The gravity zone, [ t1, t2 ], of a t-pattern extends from the earliest to
the last positive associate

The negative gravity or repulsion zone (not shown) is similarly the
interval within which any behaviors tend not to occur

T-packets are simultaneously sequential and non-sequential
structures
Neurones as Interacting and Networking
(Social) Organisms
Neuronal T-data Including Breathing
(from Nicol, Kendrick, Magnusson, 2005)
A Breathing Related Neuronal T-pattern
N = 15
Len= 12
Dur = 266482
%Dur = 88
Log10( P(Template)) = -2,88
Log10( P(Template Occurrence)) = -43,18
T-Paths are T-patterns in Space
-- A neuronal T-pattern on an Electrode Grid
Origins
Greener: higher probability of being the origin (first node)
DNA and its “Ticking” Backbone
Cyclical and Combinatorial
Cycles, base-pairs, codons; DNA->tRNA->Protein->>>Phenotype->..
Figure from A.J.F. Griffiths et al. 1999, p. 30.
Genes and Genomes as T-patterns
Non-coding or “Free” Segments Between and Within Genes
Figure from A.J.F. Griffiths et al. 1999, p. 33.
A Dinner...a Time-Flexible Pattern of Patterns
The six sub-patterns are separated by relatively freely
structured intervals of relatively fixed length.
(From Magnusson, 2005.)

Thank You