Life’s longing for itself
Speculations on propagation, prediction,
purpose and progress
Amsterdam, October 2004
(Updated version of Ulam Lectures given Sept.
2002)
J. Doyne Farmer
Santa Fe Institute
Your children are not your children.
They are the sons and daughters of Life’s
longing for itself.
They come through you but not from you,
And though they are with you yet they
belong not to you.
…
For life goes not backward nor tarries with
yesterday.
Kahlil Gibran, The Prophet
Lecture 1: Propagation
•
•
•
•
•
•
•
What are complex systems?
The history of the mechanistic view
Entropy and information
What is a machine?
Organisms and artifacts evolve
How might the first copy machine have been built?
The symbiosis of human population and technology
Main points of lecture 1
• Why is the world populated with functional
structures?
– Propagation implies prevalence.
• Not reducing universe to “just mechanics”
– Through self-organization, machines are capable of
far more than previously thought.
• Biological life, human artifacts, and human
societies all evolve.
– Relationship becoming increasingly intimate.
Lecture 2: Prediction
•
•
•
•
Prediction, action and survival
Methods of prediction
Limits to prediction and their loopholes
Personal history:
– Roulette
– financial markets
• A mechanistic model of a market.
• How prediction makes reality more subjective
– Markets manias, and other social schizophrenias.
Lecture 3: Purpose and progress
•
•
•
•
•
The progress debate
The purposeful arrow of time
What is progress?
Happiness vs. purpose
A few speculations about the future
Prediction, action, and survival
• Prediction is a prerequisite for purposeful
behavior.
• Purposeful behavior consists of three parts
– Sensation
– Prediction
– Action
• Purposeful behavior (and therefore prediction)
exists because it is useful for propagation.
Prediction, action and survival
QuickTime™ and a
Sorenson Video 3 decompressor
are needed to see this picture.
Lecture 2: Prediction
•
•
•
•
Prediction, action and survival
Methods of prediction
Limits to prediction and their loopholes
Personal history:
– Roulette
– financial markets
• A mechanistic model of a market.
• How prediction makes reality more subjective
– Markets manias, and other social schizophrenias.
Methods of prediction
• How is the model constructed?
– First principles vs. empirical
• What does the model predict?
– Dynamical vs. contemporaneous predictions
• Modeling paradigm
– Deterministic vs. random processes
Astrological prediction of stock prices
Fibonnaci predicts social trends!
How is the model constucted?
• First principles. Based on an understanding
of the world.
– Requires high degree of sophistication
• Empirical. Build a model automatically by
fitting historical data.
– Simple organisms: Hard-wired models, tuned by
evolution
– Complex organisms: Further tuning by experience
What does the model predict?
• Dynamical systems:
– Predict the future based on the past
• Contemporaneous models:
– Relate one property of the world to another
property of the world at the same time.
– Useful in simplifying description of the world.
Modeling paradigm
• Deterministic
– World is described by a single point in state
space. Completely determines future. Rule
that does this is called a dynamical system.
• Random
– Evolution of future states is not determined by
present states.
Dynamical prediction
• Key idea is state space. A state is a list of
numbers that gives the information needed
to determine the future. If you have N such
numbers, it is useful to think of them as
defining an N-dimensional space.
• E.g. from Newton’s laws, knowing forces,
position and velocity are sufficient to
determine future motion. Position and
velocity are the state.
How do bacteria do it?
• Don’t know in detail.
– Must predict concentration
– Involves measuring the concentration at
different points in time and comparing.
– If concentration is increasing, keep swimming
– Otherwise tumble and/or eat
• State space:
– One number: Rate of change of concentration
– (concentration now – concentration earlier)
Lecture 2: Prediction
•
•
•
•
Prediction, action and survival
Methods of prediction
Limits to prediction and their loopholes
Personal history:
– Roulette
– financial markets
• A mechanistic model of a market.
• How prediction makes reality more subjective
– Markets manias, and other social schizophrenias.
Limits to prediction
• Limits to prediction come from complicated
geometry of state space, which causes
nearby states to diverge rapidly.
• Produce behavior that looks random, even
in purely deterministic setting.
• Small uncertainties in initial measurements
are amplified, limiting predictability even
when model is known.
On Hurricane Charlie
You can’t plan for the unforeseen. God
doesn’t follow the linear directions of
computer models. And these are powerful
storms that don’t behave in any kind of way
that you can say with certainty where they
are going.
Jeb Bush
Poincare’ on Fortuitous phenomena
A very small cause which escapes our notice determines a
considerable effect that we cannot fail to see, and then we say
that the effect is due to chance. If we knew exactly the laws
of nature and the situation of the universe at the initial
moment, we could predict exactly the situation of that same
universe at a succeeding moment. But even if it were the
case that the natural laws had no longer any secret for us, we
could still only know the initial situation approximately. If
that enabled us to predict the succeeding situation with the
same approximation, that is all we require, and we should say
that the phenomenon had been predicted, that it is governed
by laws. But it is not always so; it may happen that small
differences in initial conditions produce very great ones in the
final phenomena. A small error in the former will produce an
enormous error in the latter. Prediction becomes impossible,
and we have the fortuitous phenomenon.
Chaos is a double-edged sword
• On one hand, long-term behavior is effectively
random (though dynamics are deterministic)
• On the other hand, short-term behavior is
predictable if model is known.
– Systems otherwise believed random become
predictable in the short term.
– Simple mechanical oscillators, transition fluid flows,
sunspots, timing of ice ages, …
(joint work with John “Sid” Sidorowich).
Rolling ball on a circular track with
counter-spinning inside track
Roulette
• Classical physics problem Newton could have solved.
– Measuring position and velocity at a given time determines
future motion. Wind resistance is main force
rate of change of velocity  constant  (velocity)
– complication due to tilt
• Prediction is difficult because of:
– circularity of wheel (like taking remainder in division)
– imperfections in track and ball creates “turbulence”
– Chaotic bouncing on cups
2
Shoe computer
Shoe +
computer
Histogram + battery boat
Copy machines
Illustration of two
methods of prediction
• Roulette provided a good illustration of two
methods of prediction:
– Version (1) based on first principles
– Version (2) based on empirical method
– (1) was more accurate but less robust
Making predictions can alter the future
• After the book The Eudeamonic Pie was published
in 1984
– Nevada passed a law against using computers to predict
the outcome of “a game”.
– Huxley roulette wheel company designed a new
roulette wheel with lower cups and more elastic balls.
– Winning players who place bets at the last minute are
immediately asked to take their business elsewhere.
– Altered the rest of my life
Limits to short term prediction
• Limits to short-term prediction come from
The curse of dimensionality
Data needed to build a good model increases
exponentially with dimension of the state space
• Even worse: High dimensionality, chaos, ability
to measure only some variables, means that some
systems are fundamentally random, even for very
short term prediction
– Casdagi, Eubank, Farmer, Gibson (1991)
• Weather, the economy, …
– high dimensional + chaotic
What about free will?
The brain as a dynamical system
Prediction can make the world less predictable
• Market efficiency: most economists believe that future
price movements are fundamentally unpredictable.
– If there are patterns in prices, profit-seeking behavior of
participants will eliminate them.
– E.g. if people think the price is going to rise, more people
will buy, which drives the price up, so the price rise happens
before it is possible to take advantage of it.
– The future becomes the present
• Effect of predictions complicates dynamics.
• Result: unpredictable prices -- “market efficiency”
• To first approximation a good model
Prediction Company
(cofounded in 1991 with Norman Packard)
• (Empirical, dynamical prediction, random process)
• Manages money under exclusive relationship with
United Bank of Switzerland (Warburg Dillon Reed)
• “Cerebellar” approach to market forecasting
– empirically searches for patterns in historical data
– keys are feature extraction, law of large numbers
– little understanding of origin of patterns
– relies on abundant past data, stationary conditions.
• Trading is fully automated (no human decisions).
Harvard Business Review, 1992
Nonetheless
• Profits are limited
– Market has friction -- trading changes prices
– Particularly felt with frequent trading
– Market is “pretty efficient” - like any business
Lecture 2: Prediction
•
•
•
•
Prediction, action and survival
Methods of prediction
Limits to prediction and their loopholes
Personal history:
– Roulette
– financial markets
• A mechanistic model of a market.
• How prediction makes reality more subjective
– Markets manias, and other social schizophrenias.
Mechanistic properties of markets
• Market institutions shape our behavior
• Neoclassical economic models assume
perfect rationality of agents
• We explore alternative: Random behavior
– Make a physics style model
– Agents make random orders at random times
Order driven market
• Two fundamental types of orders
– Market order: buy or sell given number shares
at best available price
– Limit order: buy or sell given number shares at
a specified price. Does not guarantee execution!
• Patient traders use limit orders; impatient
traders use market orders
Patient trading
• Patient traders place non-marketable
limit orders that do not lead to an
immediate transaction
• Non-marketable limit orders accumulate
• Limit order book is a storage device
Limit Order
BUY / SELL
# OF SHARES
VOLUME
LIMIT PRICE
BID
NEW ASKASK
price ($)
Impatient trading
Market Order
BUY / SELL
# OF SHARES
VOLUME
Market order:
• An order to buy or sell up to a given volume
• No limit price is defined
• Executed immediately
• Often causes unfavorable price impact
BID
ASK
NEW ASK
price ($)
Order cancellation
VOLUME
Limit order cancellations:
• Limit orders can be cancelled by the owner
• Market defined expiration
price ($)
The Basic Model
Limit
order arrival: unit size, in time & price; 
Market
order arrival: unit size, random in time; 
Cancellation:
Assume
Random in time (like radioactive decay); 
prices are continuous.
Depth of the book np,t tells how many shares are in the book at
price p at a given time t.
SELL LIMIT ORDERS
BUY MARKET ORDERS

0
BUY LIMIT ORDERS

SELL MARKET
ORDERS
ASK
n( p, t )
BID
( p, t )


p
Parameters of model
Three fundamental dimensional quantities:
shares, price, time
Five parameters:
  limit order rate, shares/(pr ice  time)
  market order rate, (shares/ti me)
  order cancellati on rate, (1/time)
  typical order size (shares)
dp  tick size (price)
Results in simple formulas predicting volatility, liquidity, spread, ..
London Stock Exchange data set
Continuous double auction
Execution priority of limit orders:
• Price priority: lower sell / higher buy limit prices
• Time priority: applicable only for limit orders with same price
SELL
VOLUME
PRIORITY
VOLUME
LIMIT ORDERS
SPREAD
(BEST) BID
BUY
(BEST) ASK
PRIORITY
price ($)
QuickTime™ and a
Video decompressor
are needed to see this picture.
Predicted
price
diffusion
rate
 5 / 21/ 2 1/ 2
2

Top 10 Russian jokes, Oct. 23, 2003
с сайта "Немецкая волна"
http://www.dw-world.de/russian/0,3367,2212_A_985770_1_A,00.html
Ученые-экономисты давно стараются понять закономерности, которым
подчиняются биржевые курсы, и используют для этого математические
модели. На протяжении многих десятилетий такие модели исходили из
представлений о брокерах как об аналитиках с выдающимися умственными
способностями, обладающих исчерпывающей информацией о рынке и
действующих исключительно рационально. Однако удовлетворительно описать
реальные изменения биржевых курсов эти модели оказались не в состоянии.
Значительно успешнее справляется с этой задачей новая модель,
предложенная Дойном Фармером (J. Doyne Farmer), сотрудником Института
Санта-Фе в штате Нью-Мексико. Она базируется на предположении, что
брокеры Ц полные Ђидиотыї, действующие совершенно случайно и к тому же
лишенные какой бы то ни было информации. Сравнив данные, рассчитанные на
основе этой модели, с реальными курсами лондонской фондовой биржи за
период с 1998-го по 2000-й годы, ученые выявили очень высокую степень
совпадения
Lecture 2: Prediction
•
•
•
•
Prediction, action and survival
Methods of prediction
Limits to prediction and their loopholes
Personal history:
– Roulette
– financial markets
• A mechanistic model of a market.
• How prediction makes reality more subjective
– Markets manias, and other social schizophrenias.
The 2nd millenium technology bubble
(NASDAQ)
Second millenium tech bubble (CISCO)
Wheat price in Munich (1815-1820)
from: Hidden Collective
Factors in Speculative
Trading, by Bertrand M.
Roehner (2001)
Monthly wheat price in Paris (1691-1694)
from: Hidden Collective
Factors in Speculative
Trading, by Bertrand M.
Roehner (2001)
Price of Bananas
from: Hidden Collective
Factors in Speculative
Trading, by Bertrand M.
Roehner (2001)
Fortune teller problem
Predicting the future
influences the future
Feedback between prediction and reality
… if a dream can tell the future it can also thwart
that future. For God will not permit that we shall
know what is to come. He is bound to no one that
the world shall unfold just so upon its course and
those who by some sorcery or by some dream
might come to pierce the veil that lies so darkly
over all that is before them may serve by just that
vision to cause that God should wrench the world
from its heading and set it upon another course
altogether and then where stands the sorcerer?
Where the dreamer and his dream?
Cormac McCarthy, The Crossing
Dmitriy Cherkashin
Seth Lloyd
A simple game with feedback between
perception and reality
• Assume N agents bet on event at time t.
– e.g. a horse race.
• Odds based on net wager on each outcome.
• Allow outcome to be influenced by odds.
– financial markets provide a good example.
– supply and demand are inherently subjective.
• Simplest case is coin flip
– only two outcomes
– coin can be biased -- bias can depend on odds
probabilit y tails  (1 - probabilit y heads)
Examples where perception can
influence reality
•
•
•
•
•
Gambling
Economics
Politics
Global climate change
Personal achievement
Agent strategies
• Assume N+ 1 players bet
(0, 1/N, 2/N, …, 1) of their money on heads
• Give them all equal wealth to start with.
Purely objective reality
Self-defeating prophesy
Self-defeating prophesy
Bias of coin vs. time
Weak self-fulfilling prophesy
Bias of coin vs. bias of predictions
Weak self-fulfilling prophesy
Bias of coin vs. time
Perfectly self-fulfilling
Perfectly self-fulfilling prophesy
Bias of coin vs. time
Over self-fulfilling
Over self-fulfilling
Bias of coin vs. time
Perfectly self-fulfilling prophesy
Bias of coin vs. time
Feedback and reality
• If you are able to create reality, reality can
become unstable due to feedback.
• You must make yourself part of the
prediction; must also model others.
• Danger of schizophrenia
Creating reality
Guys like you are ‘in what we called the
reality-based community’, defined as people
who ‘believe that solutions emerge from your
judicious study of discernable reality. That’s
not the way the world really works anymore.
We’re an empire now, and when we act we
create our own reality.’
A senior advisor to G.W. Bush, to Ron Susskind
Summary
• Prediction is a key element of purposeful
behavior coming out of propagation.
• Predictions come in many shapes and sizes
• As the sophistication of predictions
increases, their affect on the environment
can make prediction more difficult, and
make dynamics unstable.
Thanks
Else Neeft