Framework for Live
Algorithms
Tim Blackwell
Michael Young
Goldsmiths College, London
www.timblackwell.com
Organised Sound 9(2): 123–136
The context
Consider (for now, because it’s
easier) improvised performance
i.e. free of compositional
directives such as harmony, rhythm,
form…
Conventionally such improvisations
are explorations of timbre and
texture
The context
Consider (for now, because it’s
easier) improvised performance
i.e. free of compositional
directives such as harmony, rhythm,
form…
Conventionally such improvisations
are explorations of timbre and
texture
What is a Live Algorithm?
A Live Algorithm is
Interactive
Autonomous
Ideas generator
Idiosyncratic
Comprehensible
Interactive
Sympathetic, supportive, makes
appropriate contributions, tacets
Autonomous
Not merely automatic, mechanical and
predictable. Must be comprehensibly
interactive and capable of novelty
Ideas machine
Contradictory, individualistic
source of novelty and leadership
Idiosyncratic
Contributions limited by instrument
and experience but can be unusual,
individualistic
Comprehensible
Suitable for collaboration, at least
by humans. Not opaque, but not too
transparent either
And, importantly…
Closure
…knows when to stop!
Principles
Expectation of generating form might
not be necessary
Local events can be structuring
For example, spatio-temporal self-organisation depends only on
local interactions of a certain complexity and on positive and
negative feedback.
Here, the space is (interpreted as) a musical/sonic parameter
space at some level.
Parameterisations possible at micro (sample, grain), mini
(note) and meso (phrase) levels.
The problem of emergence
might necessitate a need a multi-level system
Knowledge of music rules might not be
necessary
LA need not be aware of what it is doing
In the Y model, LAs and humans interact
with meaningless sounds, populating an
inert environment.
This builds on a former XY model, which
expresses the paradox of interaction
(In our nature-inspired systems, The
sounds are organised by stigmergy)
Knowledge of music rules might not be
necessary
LA need not be aware of what it is doing
Desirability of an Interactive Model and a
Conceptual Architecture
In the Y model, LAs and humans interact
with meaningless sounds, populating an
inert environment.
This builds on a former XY model, which
expresses the paradox of interaction
(In our nature-inspired systems, The
sounds are organised by stigmergy)
The XY model
The Y model
y
A
B
Conceptual (and Actual?) Architecture
P: Listening, analysing
Typically thins degrees of freedom; from
real time
YIN -> p(t) -> pi
F(p): Ideas engine
Patterning in a hidden space H;
generative/algorithmic/iterative
xi -> xi+1
Q: Interpretation, Playing, synthesizing
Typically expands d.o.f.; into real time
xi+1 -> q(t) -> YOUT
PfQ Architecture
Analysis
is typically projection from one level to
another, higher level. E.g. samples Y to
event parameters p
P: Y(t) -> p(t) -> pi
We observe that this projection is
determined by cultural, personal, genrespecific and even political forces
P might be a map into H and hence hidden
state x is a possible parameterisation of
the sonic environment.
Interpretation
is the process of relating internal states to event
parameterisations, and eventually to sound
Q: xi+1 -> q(t) -> YOUT
A technical complication surrounds the relationship
between real and algorithmic time
It is unlikely that both will flow at the same rate: the
computational update time xi -> xi+1 might not correspond
with the desired time interval between events Y(t),
Y(t+Dt), ...
Internal states might be sampled at a given rate
irrespective of iterative time or timing information
could even derive from x itself
Information might be derived from averages over a
population {x1, x2, x3, ...}
All these complications and possibilities are
represented by a single interpretative function Q
Ideas generator
F is parameterised by p and determines
state flow of hidden variables x
F(p): xi -> xi+1
xi+1 determined F, p(t) and xi
F need not derive from any musical
concern, and this may be advantageous
F only concerns patterning and structure
Autonomous
1. Dx ≠ 0 even if Dp = 0
2. Dx = 0 even if Dp ≠ 0
1. This is usual, expresses state flow in
a dynamical system
2. Harder to achieve, but could arrange
for Q( x  [x-dx, x+dx] ) = q.
Alternatively, include dissipation and reenergising so that Dx  0 between
injections of energy (determined by S p?)
Idiosyncratic
Q and P concern the mappings to and
from sound – herein lies the
machine's idiosyncrasies and
characteristics
Can be quite limited, since
interactivity and autonomy are the
major prerequisites
Potentially P, Q may cross levels
Interactive
autonomy + modification of system state
Analysis parameters p ensure interaction i.e. Y
affects, but does not fully determine, x
For example, p might be an attractor in H. In a
dynamic system, x orbits p, but precise trajectory
depends on initial conditions
Alternatively, hidden variables x can be regarded
as parameters which select mappings Fx: p -> q
The collaborative interface is the asynchronous
map
Q Fx P = Mx
YOUT(t+Dt) = Mx YIN(t)
Comprehensible:
Transparency > Comprehensibilty > Opacity
Very transparent if p  H, Q = P-1 and dim(H) is
small enough.
Easy to establish correlations between Yhuman
(incoming sound, deposited in the environment by a
human) and YLA (outgoing sound deposited by the
LA). Might become too predictable unless enhanced
by some stochastic adjustment (a degenerate
solution)
Very opaque if QFP is very complex and dim(H) is
big.
Very hard to establish correlations or any measure
of causal relations between Yhuman and YLA. Too
unpredictable and therefore not collaborative
Closure:
Raises issues of knowledge of form
Could argue that LA should not be deprived
of knowing the modus operandi of the
performance
In humans, closure is influenced by time
constraints, visual cues and an increasing
capacity to interpret gestures as possible
sonic cadences
However, in an LA closure might possibly
emerge from lower level dynamics e.g.
self-organised criticality
Other desirable properties
Memory
Dynamical states do not conventionally posses this
Short term (repetitions, anticipations based on
recognition..)
>>> attractor persistence, pheromone trails
E.g. Swarm Techtiles, Ant Colony simulations
Long term (draws on previous musical engagements)
>>> current states could have memories of previous
states…
E.g. Particle Swarms (particles have a memory and
participate in social networks)
…AND the sequence of conditions that caused these
states (a contextual memory)
E.g. ??
Links with existing research fields:
P: Y -> p
Real-time music analysis and informatics
Q: x -> Y
Real-time audio synthesis
F
Generative Music (Neural, Cellular
Autonoma, Genetic Algorithms, Swarms
Links with existing paradigms
Live Electronics:
ideas engine F replaced by human volition. State
flow is adjustment of “controls” x
Live Coding
F = I and Q is adjusted (in software) manually
Generative/algorithmic music:
Set Dp = 0 (turn P off)
Y(t) = F(xi) = FN( x0 )
The system is self-contained - the composer
chooses F and the initial condition, x(0)
Reactive Systems
In a reactive system, YIN will necessarily trigger
certain transitions xi -> xi+1
Mx = QFP express a causal and necessary chain – a
rule-based system
It is conjectured that a reactive system might not
be sufficient for a Live Algorithm because it is
not sufficiently interactive
This touches on various issues in cognitive science and machine
intelligence
Possibly, a large enough rule set might do the job, and might also be
indistinguishable from a discrete dynamical system (i.e. an algorithmic
model of a continuous DS)
It is hard to see how we might achieve this complexity without drawing
on inspiration from dynamical, and other natural, systems
(Our) Experience
Swarm Music:
Very transparent, P and Q operate at the
note level
Swarm Granulator:
Less transparent, functions largely at the
grain level, although some level-crossing
Swarm Tech-tiles
P actually expands Y into a 2D landscape
F integrates ALife and optimisation
Q is a rendering of parts of the landscape
visited by x
Argrophyllax
P: FFT analysis
f: stochastic function
H: Fourier space
Q: coefficients of inverse FFT transform Q
Evaluation
Irrelevance of Turing Test? TT is more
relevant for the performers than the
listeners. A metric: do the performers
find the LA to be stucturing?
Applicability:
The attributes of a LA should be useful in
other musical contexts
E.g. intelligent effects pedal, synth
plug-ins, accompaniment programs, genre
improvisation