Predictive coding and active inference
Karl Friston, University College London
How much about our interaction with – and experience of – our world can be deduced from basic
principles? This talk reviews recent attempts to understand the self-organised behaviour of
embodied agents, like ourselves, as satisfying basic imperatives for sustained exchanges with the
environment. In brief, one simple driving force appears to explain many aspects of action and
perception. This driving force is the minimisation of surprise or prediction error that – in the context of
perception – corresponds to Bayes-optimal predictive coding. We will look at some of the
phenomena that emerge from this principle; such as hierarchical message passing in the brain and
the perceptual inference that ensues. I hope to illustrate the ensuing brain-like dynamics using
models of bird songs that are based on autonomous dynamics. This provides a nice example of how
dynamics can be exploited by the brain to represent and predict the sensorium that is – in many
instances – generated by ourselves. I hope to conclude with an illustration that illustrates the tight
relationship between communication and active inference about the behaviour of self and others.
Overview
The anatomy of inference
predictive coding
graphical models
canonical microcircuits
Birdsong
perceptual categorization
omission related responses
sensory attenuation
a birdsong duet
“Objects are always imagined as being present in the field of
vision as would have to be there in order to produce the same
impression on the nervous mechanism” - von Helmholtz
Hermann von Helmholtz
Richard Gregory
Geoffrey Hinton
The Helmholtz machine and the
Bayesian brain
Thomas Bayes
Richard Feynman
“Objects are always imagined as being present in the field of
vision as would have to be there in order to produce the same
impression on the nervous mechanism” - von Helmholtz
Hermann von Helmholtz
sensory impressions…
Richard Gregory
Plato: The Republic (514a-520a)
Bayesian filtering and predictive coding
 t   D        
changes in expectations are predicted changes and (prediction error) corrections

  s  g ( )
prediction error
Minimizing prediction error
sensations – predictions
Prediction error
Action
Perception
Change sensations
Change predictions
Generative models
A simple hierarchy
v(3)
v (2)
 x(2)
x (2)
v(2)
v (1)
 x(1)
x (1)
 v(1)
v (0)
what
where
Sensory
fluctuations
From models to perception
A simple hierarchy
(3)
v(3)
v
Ascending
prediction errors
Generative model
Dx(i )  f (i ) ( x(i ) , v (i ) )  x(i )
(2)

v (2)
v
(2)

 x(2)
x
(2)
x (2)
x
(2)

 v(2)
v
(1)

v (1)
v
x(1)x(1)
(1)
x (1)
x
(1)
v(1)
v
(0)

v (0)
v
v (i 1)  g (i ) ( x(i ) , v (i ) )  v(i )
Descending
predictions
ModelPredictive
inversion (inference)
coding
Expectations:
Predictions:
Prediction errors:
v(vi()i ) DDv(vi()i )vv F( i )( s , ( i ),) v( i 1)
x(ix()i ) DDx(ix()i )xx F(i )( s , (i ), )
g (i )  g (i ) (  x( i ) , v( i ) )
f (i )  f (i ) (  x( i ) , v( i ) )
 v(i )   (vi ) v( i )   (vi ) ( v( i 1)  g ( i ) )
 x(i )   (xi ) x( i )   (xi ) (D  x( i )  f ( i ) )
Canonical microcircuits for predictive coding
Haeusler and Maass: Cereb. Cortex 2006;17:149-162
Bastos et al: Neuron 2012; 76:695-711
David Mumford
Predictive coding with reflexes
Action
a   a s   v(1)
oculomotor
signals
reflex arc
proprioceptive input
pons
Perception
retinal input
Errors (superficial pyramidal cells)
frontal eye fields
geniculate
 (i )
Top-down or descending
predictions
v(i )   (vi ) (v(i 1)  g (i ) ( x(i ) , v(i ) ))
 x(i )   (xi ) (D x(i )  f (i ) (  x(i ) , v(i ) ))
Expectations (deep pyramidal cells)
Bottom-up or ascending
prediction error
visual cortex
 (i )
v(i )  Dv(i )   v (i )   ( i )   v(i 1)
 x(i )  D x(i )   x (i )   (i )
Interim summary
Hierarchical predictive coding is a neurobiological plausible scheme that the brain
might use for (approximate) Bayesian inference about the causes of sensations
Predictive coding requires the dual encoding of expectations and errors, with
reciprocal (neuronal) message passing
Much of the known neuroanatomy and neurophysiology of cortical architectures is
consistent with the requisite message passing
Hermann von Helmholtz
“It is the theory of the sensations of hearing to which the theory of music
has to look for the foundation of its structure." (Helmholtz, 1877 p.4)
‘Helmholtz, H. (1877). “On the Sensations of Tone as a Physiological Basis for the Theory of Music", Fourth German edition,;
translated, revised, corrected with notes and additional appendix by Alexander J. Ellis. Reprint: New York, Dover Publications
Inc.,1954
Overview
The anatomy of inference
predictive coding
graphical models
canonical microcircuits
Birdsong
perceptual categorization
omission related responses
sensory attenuation
a birdsong duet
Generating bird songs with attractors
 v1 
v 
v2 
Hidden causes
f (1)
18 x2(1)  18 x1(1)

 (1) (1)

 v1 x1  2 x3(1) x1(1)  x2(1) 
 2 x (1) x (1)  v (1) x (1)

2
3
 1 2

Syrinx
Sonogram
Frequency
Higher vocal center
Hidden states
0.5
1
1.5
time (sec)
prediction and error
20
15
Predictive coding and message passing
10
5
0
-5

(1)
v
10
20
30
40
50
60
causal states
Descending predictions
20
 x(1)
stimulus
15
10
5000
4500
s (t )
4000
 v(1)
Ascending prediction error

3500
3000
2000
0.2
0.4
0.6
time (seconds)
0.8
15
10
5
0
-5
10
20
30
40
50
0
-5
-10
hidden states
20
2500
(1)
x
5
60
10
20
30
40
50
60
Frequency (Hz)
Perceptual categorization
Song a
Song b
time (seconds)
Song c
Sequences of sequences
Higher vocal center
Syrinx
y1
y2
v1(1)
v2(1)
Sonogram
Frequency (KHz)
Area X
0.5
18 x  18 x

18 x  18 x





 32 x1(2)  2 x3(2) x1(2)  x2(2)  f (1)  v1(1) x1(1)  2 x3(1) x1(1)  x2(1) 
 2 x (2) x (2)  8 x (2)

 2 x (1) x (1)  v (1) x (1)

2
3
3 3
 1 2

 1 2

(2)
2
f (2)
g
(2)
(2)
1
 x2(2)  v1(1) 
  (2)    (1) 
 x3  v2 
(1)
2
g
(1)
(1)
1
 x2(1)   y1 
  (1)    
 x3   y2 
1
1.5
Time (sec)
Frequency (Hz)
4500
4000
3500
3000
2500
without last syllable
4500
Frequency (Hz)
4000
3500
3000
percept
2500
0.5
100
1
Time (sec)
percept
1.5
0.5
ERP (prediction error)
100
0
-50
-100
1
Time (sec)
1.5
with omission
50
LFP (micro-volts)
50
LFP (micro-volts)
omission and
violation of
predictions
stimulus (sonogram)
Stimulus but no percept
0
Percept but no stimulus
-50
500
1000
1500
peristimulus time (ms)
2000
-100
500
1000
1500
peristimulus time (ms)
2000
Active inference: creating your own sensations
Higher vocal
centre
Motor commands
(proprioceptive predictions)
Hypoglossal Nucleus
Corollary discharge
(exteroceptive predictions)
Thalamus
Area X
Active inference and sensory attenuation
percept
5000
Frequency (Hz)
4500
4000
3500
3000
2500
0.2
0.4
0.6
0.8
1
time (sec)
1.2
1.4
1.6
1.8
1.6
1.8
2
1.6
1.8
2
First level expectations (hidden states)
40
30
20
10
0
-10
0
0.2
0.4
0.6
0.8
1
time (seconds)
1.2
1.4
Second level expectations (hidden states)
50
40
30
20
10
0
0
0.2
0.4
0.6
0.8
1
time (seconds)
1.2
1.4
Active inference and sensory attenuation
percept
5000
Frequency (Hz)
4500
Mirror neuron system
4000
3500
3000
2500
0.2
0.4
0.6
0.8
1
time (sec)
1.2
1.4
1.6
1.8
1.6
1.8
2
1.6
1.8
2
First level expectations (hidden states)
80
60
40
20
0
-20
-40
0
0.2
0.4
0.6
0.8
1
time (seconds)
1.2
1.4
Second level expectations (hidden states)
60
50
40
30
20
10
0
-10
0
0.2
0.4
0.6
0.8
1
time (seconds)
1.2
1.4
percept
5000
Frequency (Hz)
4500
4000
3500
3000
2500
1
2
3
4
time (sec)
5
6
7
6
7
8
6
7
8
First level expectations (hidden states)
100
50
0
-50
0
1
2
3
4
time (seconds)
5
Second level expectations (hidden states)
80
60
40
20
0
-20
-40
0
1
2
3
4
time (seconds)
5
percept
5000
Frequency (Hz)
4500
4000
3500
Active inference and communication
3000
2500
1
2
3
4
time (sec)
5
6
7
6
7
8
6
7
8
First level expectations (hidden states)
100
50
0
-50
0
1
2
3
4
time (seconds)
5
Second level expectations (hidden states)
80
60
40
20
0
-20
-40
0
1
2
3
4
time (seconds)
5
Hermann von Helmholtz
"There is nothing in the nature of music itself to determine the pitch of the
tonic of any composition...In short, the pitch of the tonic must be chosen so
as to bring the compass of the tones of the piece within the compass of the
executants, vocal or instrumental.” (Helmholtz, 1877 p. 310)
‘Helmholtz, H. (1877). “On the Sensations of Tone as a Physiological Basis for the Theory of Music", Fourth German edition,;
translated, revised, corrected with notes and additional appendix by Alexander J. Ellis. Reprint: New York, Dover Publications
Inc.,1954
Thank you
And thanks to collaborators:
Rick Adams
Andre Bastos
Sven Bestmann
Harriet Brown
Jean Daunizeau
Mark Edwards
Xiaosi Gu
Lee Harrison
Stefan Kiebel
James Kilner
Jérémie Mattout
Rosalyn Moran
Will Penny
Lisa Quattrocki Knight
Klaas Stephan
And colleagues:
Andy Clark
Peter Dayan
Jörn Diedrichsen
Paul Fletcher
Pascal Fries
Geoffrey Hinton
James Hopkins
Jakob Hohwy
Henry Kennedy
Paul Verschure
Florentin Wörgötter
And many others