00adBARYAM_29412
9/5/00 7:26 PM
Page 1
0
Overview:
The Dynamics of Complex Systems —
Examples, Questions, Methods and Concepts
0.1
The Field of Complex Systems
The study of complex systems in a unified framework has become recognized in recent years as a new scientific discipline, the ultimate of interdisciplinary fields. It is
strongly rooted in the advances that have been made in diverse fields ranging from
physics to anthropology, from which it draws inspiration and to which it is relevant.
Many of the systems that surround us are complex. The goal of understanding
their properties motivates much if not all of scientific inquiry. Despite the great complexity and variety of systems, universal laws and phenomena are essential to our inquiry and to our understanding. The idea that all matter is formed out of the same
building blocks is one of the original concepts of science. The modern manifestation
of this concept—atoms and their constituent particles—is essential to our recognition of the commonality among systems in science. The universality of constituents
complements the universality of mechanical laws (classical or quantum) that govern
their motion. In biology, the common molecular and cellular mechanisms of a large
variety of organisms form the basis of our studies.However, even more universal than
the constituents are the dynamic processes of variation and selection that in some
manner cause organisms to evolve. Thus, all scientific endeavor is based, to a greater
or lesser degree, on the existence of universality, which manifests itself in diverse ways.
In this context,the study of complex systems as a new endeavor strives to increase our
ability to understand the universality that arises when systems are highly complex.
A dictionary definition of the word “complex” is: “consisting of interconnected
or interwoven parts.” Why is the nature of a complex system inherently related to its
parts? Simple systems are also formed out of parts. To explain the difference between
simple and complex systems, the terms “interconnected” or “interwoven” are somehow essential.Qualitatively, to understand the behavior of a complex system we must
understand not only the behavior of the parts but how they act together to form the
behavior of the whole. It is because we cannot describe the whole without describing
each part, and because ea ch part must be described in relation to other parts, that
complex systems are difficult to understand. This is relevant to another definition of
“complex”: “not easy to understand or analyze.” These qualitative ideas about what a
complex system is can be made more quantitative. Articulating them in a clear way is
1
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 1
Short / Normal / Long
00adBARYAM_29412
2
9/5/00 7:26 PM
Page 2
O ve r v i e w
both essential and fruitful in pointing the way toward progress in understanding the
universal properties of these systems.
For many years, professional specialization has led science to progressive isolation of individual disciplines.How is it possible that well-separated fields such as molecular biology and economics can suddenly become unified in a single discipline?
How does the study of complex systems in general pertain to the detailed efforts devoted to the study of particular complex systems? In this regard one must be careful
to acknowledge that there is always a dichotomy between universality and specificity.
A study of universal principles does not replace detailed description of particular
complex systems. However, universal principles and tools guide and simplify our inquiries into the study of specifics. For the study of complex systems,universal simplifications are particularly important. Sometimes universal principles are intuitively
appreciated without being explicitly stated. However, a careful articulation of such
principles can enable us to approach particular systems with a systematic guidance
that is often absent in the study of complex systems.
A pictorial way of illustrating the relationship of the field of complex systems to
the many other fields of science is indicated in Fig. 0.1.1. This figure shows the conventional view of science as progressively separating into disparate disciplines in order to gain knowledge about the ever larger complexity of systems. It also illustrates
the view of the field of complex systems, which suggests that all complex systems have
universal properties. Because each field develops tools for addressing the complexity
of the systems in their domain, many of these tools can be adapted for more general
use by recognizing their universal applicability. Hence the motivation for crossdisciplinary fertilization in the study of complex systems.
In Sections 0.2–0.4 we initiate our study of complex systems by discussing examples, questions and methods that are relevant to the study of complex systems.Our
purpose is to introduce the field without a strong bias as to conclusions, so that the
student can develop independent perspectives that may be useful in this new field—
opening the way to his or her own contributions to the study of complex systems. In
Section 0.5 we introduce two key concepts—emergence and complexity—that will
arise through our study of complex systems in this text.
0.2
Examples
0.2.1 A few examples
What are complex systems and what properties characterize them? It is helpful to start
by making a list of some examples of complex systems. Take a few minutes to make your
own list. Consider actual systems rather than mathematical models (we will consider
mathematical models later). Make a list of some simple things to contrast them with.
Examples of Complex Systems
Governments
Families
The human body—physiological perspective
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 2
Short / Normal / Long
00adBARYAM_29412
9/5/00 7:26 PM
(a)
Page 3
Biology
Chemistry
Mathematics
Physics
Computer Science
Simple systems
Philosophy
Sociology
Anthropology
Psychology
Economics
Socio Biology
logy
Mat
Com hemati
pute cs
r Sc
ienc
e
Psychology
s
Economic
Physics
y
soph
Philo y
olog
hrop
Ant
Chemistry
Complex systems
(b)
Simple systems
Figure 0.1.1 Conceptual illustration of the space of scientific inquiry. (a) is the conventional
view where disciplines diverge as knowledge increases because of the increasing complexity
of the various systems being studied. In this view all knowledge is specific and knowledge is
gained by providing more and more details. (b) illustrates the view of the field of complex
systems where complex systems have universal properties. By considering the common properties of complex systems, one can approach the specifics of particular complex systems from
the top of the sphere as well as from the bottom.
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 3
Short / Normal / Long
00adBARYAM_29412
4
9/5/00 7:26 PM
Page 4
O ve r v i e w
A person—psychosocial perspective
The brain
The ecosystem of the world
Subworld ecosystems: desert, rain forest, ocean
Weather
A corporation
A computer
Examples of Simple Systems
An oscillator
A pendulum
A spinning wheel
An orbiting planet
The purpose of thinking about examples is to develop a first understanding of the
question, What makes systems complex? To begin to address this question we can start
describing systems we know intuitively as complex and see what properties they share.
We try this with the first two examples listed above as complex systems.
Government
• It has many different functions:military, immigration,taxation,income distribution, transportation, regulation. Each function is itself complex.
• There are different levels and types of government: local, state and federal; town
meeting, council,mayoral. There are also various governmental forms in different countries.
Family
•
•
•
•
•
It is a set of individuals.
Each individual has a relationship with the other individuals.
There is an interplay between the relationship and the qualities of the individual.
The family has to interact with the outside world.
There are different kinds of families: nuclear family, extended family, etc.
These descriptions focus on function and structure and diverse manifestation.
We can also consider the role that time plays in complex systems. Among the properties of complex systems are change, growth and death, possibly some form of life cycle. Combining time and the environment, we would point to the ability of complex
systems to adapt.
One of the issues that we will need to address is whether there are different categories of complex systems. For example, we might contrast the systems we just described with complex physical systems: hydrodynamics (fluid flow, weather), glasses,
composite materials, earthquakes. In what way are these systems similar to or different from the biological or social complex systems? Can we assign function and discuss
structure in the same way?
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 4
Short / Normal / Long
00adBARYAM_29412
9/5/00 7:26 PM
Page 5
E xa m p l e s
5
0.2.2 Central properties of complex systems
After beginning to describe complex systems,a second step is to identify commonalities. We might make a list of some of the characteristics of complex systems and assign each of them some measure or attribute that can provide a first method of classification or description.
• Elements (and their number)
• Interactions (and their strength)
• Formation/Operation (and their time scales)
• Diversity/Variability
• Environment (and its demands)
• Activity(ies) (and its[their] objective[s])
This is a first step toward quantifying the properties of complex systems.Quantifying
the last three in the list requires some method of counting possibilities. The problem
of counting possibilities is central to the discussion of quantitative complexity.
0.2.3 Emergence: From elements and parts to complex systems
There are two approaches to organizing the properties of complex systems that wil l
serve as the foundation of our discussions. The first of these is the relationship between elements,parts and the whole. Since there is only one property of the complex
system that we know for sure — that it is complex—the primary question we can ask
about this relationship is how the complexity of the whole is related to the complexity of the parts. As we will see, this question is a compelling question for our understanding of complex systems.
From the examples we have indicated above, it is apparent that parts of a complex system are often complex systems themselves. This is reasonable, because when
the parts of a system are complex, it seems intuitive that a collection of them would
also be complex. However, this is not the only possibility.
Can we describe a system composed of simple parts where the collective behavior is complex? This is an important possibility, called emergent complexity. Any complex system formed out of atoms is an example. The idea of emergent complexity is
that the behaviors of many simple parts interact in such a way that the behavior of the
whole is complex.Elements are those parts of a complex system that may be considered simple when describing the behavior of the whole.
Can we describe a system composed of complex parts where the collective behavior is simple? This is also possible, and it is called emergent simplicity. A useful
example is a planet orbiting around a star. The behavior of the planet is quite simple,
even if the planet is the Earth, with many complex systems upon it. This example illustrates the possibility that the collective system has a behavior at a different scale
than its parts. On the smaller scale the system may behave in a complex way, but on
the larger scale all the complex details may not be relevant.
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 5
Short / Normal / Long
00adBARYAM_29412
6
9/5/00 7:26 PM
Page 6
O ve r v i e w
0.2.4 What is complexity?
The second approach to the study of complex systems begins from an understanding
of the relationship of systems to their descriptions. The central issue is defining quantitatively what we mean by complexity. What, after all, do we mean when we say that
a system is complex? Better yet, what do we mean when we say that one system is more
complex than another? Is there a way to identify the complexity of one system and to
compare it with the complexity of another system? To develop a quantitative understanding of complexity we will use tools of both statistical physics and computer science—information theory and computation theory. According to this understanding,
complexity is the amount of information necessary to describe a system. However, in
order to arrive at a consistent definition,care must be taken to specify the level of detail provided in the description.
One of our targets is to understand how this concept of complexity is related to
emergence—emergent complexity and emergent simplicity. Can we understand why
information-based complexity is related to the description of elements,and how their
behavior gives rise to the collective complexity of the whole system?
Section 0.5 of this overview discusses further the concepts of emergence and
complexity, providing a simplified preview of the more complete discussions later in
this text.
0.3
Questions
This text is structured around four questions related to the characterization of complex systems:
1. Space: What are the characteristics of the structure of complex systems? Many
complex systems have substructure that extends all the way to the size of the system itself. Why is there substructure?
2. Time: How long do dynamical processes take in complex systems? Many complex
systems have specific responses to changes in their environment that require
changing their internal structure. How can a complex structure respond in a reasonable amount of time?
3. Self-organization and/versus organization by design: How do complex systems
come into existence? What are the dynamical processes that can give rise to complex systems? Many complex systems undergo guided developmental processes
as part of their formation. How are developmental processes guided?
4. Complexity: What is complexity? Complex systems have varying degrees of complexity. How do we characterize/distinguish the varying degrees of complexity?
Chapter 1 of this text plays a special role. Its ten sections introduce mathematical
tools. These tools and their related concepts are integral to our understanding of complex system behavior. The main part of this book consists of eight chapters,2–9. These
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 6
Short / Normal / Long
00adBARYAM_29412
9/5/00 7:26 PM
Page 7
Questions
7
chapters are paired.Each pair discusses one of the above four questions in the context
of a particular complex system. Chapters 2 and 3 discuss the role of substructure in
the context of neural networks. Chapters 4 and 5 discuss the time scale of dynamics
in the context of protein folding. Chapters 6 and 7 discuss the mechanisms of organization of complex systems in the context of living organisms. Chapters 8 and 9 discuss complexity in the context of human civilization. In each case the first of the pair
of chapters discusses more general issues and models. The second tends to be more
specialized to the system that is under discussion. There is also a pattern to the degree
of analytic, simulation or qualitative treatments. In general,the first of the two chapters is more analytic, while the second relies more on simulations or qualitative treatments. Each chapter has at least some discussion of qualitative concepts in addition
to the formal quantitative discussion.
Another way to regard the text is to distinguish between the two approaches summarized above. The first deals with elements and interactions. The second deals with
descriptions and information. Ultimately, our objective is to relate them, but we do so
using questions that progress gradually from the elements and interactions to the descriptions and information. The former dominates in earlier chapters, while the latter is important for Chapter 6 and becomes dominant in Chapters 8 and 9.
While the discussion in each chapter is presented in the context of a specific
complex system, our focus is on complex systems in general. Thus, we do not attempt (nor would it be possible) to review the entire fields of neural networks, protein folding, evolution, developmental biology and social and economic sciences.
Since we are interested in universal aspects of these systems, the topics we cover
need not be the issues of contemporary importance in the study of these systems.
Our approach is to motivate a question of interest in the context of complex systems using a particular complex system, then to step back and adopt a method of
study that has relevance to all complex systems. Researchers interested in a particular complex system are as likely to find a discussion of interest to them in any one
of the chapters, and should not focus on the chapter with the particular complex
system in its title.
We note that the text is interrupted by questions that are, with few exceptions,
solved in the text. They are given as questions to promote independent thought about
the study of complex systems. Some of them develop further the analysis of a system
through analytic work or through simulations. Others are designed for conceptual development. With few exceptions they should be considered integral to the text, and
even if they are not solved by the reader, the solutions should be read.
Q
uestion 0.3.1 Consider a few complex systems. Make a list of their elements, interactions between these elements, the mechanism by which
the system is formed and the activities in which the system is engaged.
Solution 0.3.1 The following table indicates properties of the systems that
we will be discussing most intensively in this text. ❚
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 7
Short / Normal / Long
00adBARYAM_29412
8
9/5/00 7:26 PM
Page 8
O ve r v i e w
System
Element
Interaction
Formation
Activity
Proteins
Amino Acids
Bonds
Protein folding
Enzymatic
activity
Nervous system
Neural networks
Neurons
Synapses
Learning
Behavior
Thought
Physiology
Cells
Chemical
messengers
Physical support
Developmental
biology
Movement
Physiological
functions
Life
Organisms
Reproduction
Competition
Predation
Communication
Evolution
Survival
Reproduction
Consumption
Excretion
Human
economies
and societies
Human Beings Communication
Technology
Confrontation
Cooperation
Social evolution
Same as Life?
Exploration?
Table 0.3.1: Complex Systems and Some Attributes
0.4
Methods
When we think about methodology, we must keep purpose in mind.Our purpose in
studying complex systems is to extract general principles.General principles can take
many forms. Most principles are articulated as relationships between properties—
when a system has the property x, then it has the property y. When possible, relationships should be quantitative and expressed as equations. In order to explore such relationships, we must construct and study mathematical models. Asking why the
property x is related to the property y requires an understanding of alternatives. What
else is possible? As a bonus, when we are able to generate systems with various properties, we may also be able to use them for practical applications.
All approaches that are used for the study of simple systems can be applied to the
study of complex systems. However, it is important to recognize features of conventional approaches that may hamper progress in the study of complex systems. Both
experimental and theoretical methods have been developed to overcome these difficulties. In this text we introduce and use methods of analysis and simulation that are
particularly suited to the study of complex systems. These methods avoid standard
simplifying assumptions, but use other simplifications that are better suited to our
objectives. We discuss some of these in the following paragraphs.
• Don’t take it apart. Since interactions between parts of a complex system are essential to understanding its behavior, looking at parts by themselves is not sufficient. It is necessary to look at parts in the context of the whole. Similarly, a complex system interacts with its environment, and this e nvironmental influence is
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 8
Short / Normal / Long
00adBARYAM_29412
9/5/00 7:26 PM
Page 9
C o nc e pts : Emer ge nc e a n d co mp l ex i t y
9
important in describing the behavior of the system. Experimental tools have been
developed for studying systems in situ or in vivo—in context. Theoretical analytic
methods such as the mean field approach enable parts of a system to be studied
in context. Computer simulations that treat a system in its entirety also avoid
such problems.
• Don’t assume smoothness. Much of the quantitative study of simple systems makes
use of differential equations. Differential equations,like the wave equation,assume
that a system is essentially uniform and that local details don’t matter for the behavior of a system on larger scales. These assumptions are not generally valid for
complex systems.Alternate static models such as fractals, and dynamical models including iterative maps and cellular automata may be used instead.
• Don’t assume that only a few parameters are important. The behavior of complex
systems depends on many independent pieces of information. Developing an understanding of them requires us to build mental models. However, we can only
have “in mind” 7±2 independent things at once. Analytic approaches, such as
scaling and renormalization,have been developed to identify the few relevant parameters when this is possible. Information-based approaches consider the collection of all parameters as the object of study. Computer simulations keep track
of many parameters and may be used in the study of dynamical processes.
There are also tools needed for communication of the results of studies.
Conventional manuscripts and oral presentations are now being augmented by video
and interactive media. Such novel approaches can increase the effectiveness of communication,particularly of the results of computer simulations. However, we should
avoid the “cute picture” syndrome, where pictures are presented without accompanying discussion or analysis.
In this text, we introduce and use a variety of analytic and computer simulation
methods to address the questions listed in the previous section. As mentioned in the
preface, there are two general methods f or studying complex systems. In the first, a
specific system is selected and each of the parts as well as their interactions are identified and described. Subsequently, the objective is to show how the behavior of the
whole emerges from them. The second approach considers a class of systems (ensemble), where the essential characteristics of the class are described,and statistical analysis is used to obtain properties and behaviors of the systems. In this text we focus on
the latter approach.
0.5
Concepts: Emergence and Complexity
The objectives of the field of complex systems are built on fundamental concepts—
emergence, complexity—about which there are common misconceptions that are addressed in this section and throughout the book.Once understood,these concepts reveal the context in which universal properties of complex systems arise and specific
universal phenomena, such as the evolution of biological systems, can be better
understood.
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 9
Short / Normal / Long
00adBARYAM_29412
10
9/5/00 7:26 PM
Page 10
O ve r v i e w
A complex system is a system formed out of many components whose behavior
is emergent,that is,the behavior of the system cannot be simply inferred from the behavior of its components. The amount of information necessary to describe the behavior of such a system is a measure of its complexity. In the following sections we
discuss these concepts in greater detail.
0.5.1 Emergence
It is impossible to understand complex systems without recognizing that simple atoms
must somehow, in large numbers, give rise to complex collective behaviors. How and
when this occurs is the simplest and yet the most profound problem that the study of
complex systems fa ces. The p roblem can be approached first by developing an understanding of the term “emergence.” For many, the concept of emergent behavior
means that the behavior is not captured by the behavior of the parts. This is a serious
misunderstanding. It arises because the collective behavior is not readily understood
from the behavior of the parts. The collective behavior is, however, contained in the
behavior of the parts if they are studied in the context in which they are found. To explain this,we discuss examples of emergent properties that illustrate the difference between local emergence—where collective behavior appears in a small part of the system—and global emergence—where collective behavior pertains to the system as a
whole. It is the latter which is particularly relevant to the study of complex systems.
We can speak about emergence when we consider a collection of elements and the
properties of the collective behavior of these elements. In conventional physics, the
main arena for the study of such properties is thermodynamics and statistical mechanics. The easiest thermodynamic system to think about is a gas of particles. Two
emergent properties of a gas are its pressure and temperature. The reason they are
emergent is that they do not naturally arise out of the description of an individual particle. We generally describe a particle by specifying its position and velocity. Pressure
and temperature become relevant only when we have many particles together. While
these are emergent properties, the way they are emergent is very limited. We call them
local emergent properties. The pressure and temperature is a local property of the gas.
We can take a very small sample of the gas away from the rest and still define and measure the (same) pressure and temperature. Such properties,called intensive in physics,
are local emergent properties. Other examples from physics of locally emergent behavior are collective modes of excitation such as sound waves, or light propagation in
a medium. Phase transitions (e.g., solid to liquid) also represent a collective dynamics
that is visible on a macroscopic scale, but can be seen in a microscopic sample as well.
Another example of a local emergent property is the formation of water from
atoms of hydrogen and oxygen. The properties of water are not apparent in the properties of gasses of oxygen or hydrogen. Neither does an isolated water molecule reveal
most properties of water. However, a microscopic amount of water is sufficient.
In the study of complex systems we are particularly interested in global emergent
properties. Such properties depend on the entire system. The mathematical treatment
of global emergent properties requires some effort. This is one reason that emergence
is not well appreciated or understood. We will discuss global emergence by summariz-
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 10
Short / Normal / Long
00adBARYAM_29412
9/5/00 7:26 PM
Page 11
C onc e pt s : E merge nc e a nd c omp lex i t y
11
ing the results of a classic mathematical treatment, and then discuss it in a more general manner that can be readily appreciated and is useful for semiquantitative analyses.
The classic analysis of global emergent behavior is that of an associative memory
in a simple model of neural networks known as the Hopfield or attractor network.
The analogy to a neural network is useful in order to be concrete and relate this model
to known concepts. However, this is more generally a model of any system formed
from simple elements whose states are correlated. Without such correlations, emergent behavior is impossible. Yet if all elements are correlated in a simple way, then local emergent behavior is the outcome. Thus a model must be sufficiently rich in order to capture the phenomenon of global emergent behavior. One of the important
qualities of the attractor network is that it displays global emergence in a particularly
elegant manner. The following few paragraphs summarize the operation of the attractor network as an associative memory.
The Hopfield network has simple binary elements that are either ON or OFF. The
binary elements are an abstraction of the firing or quiescent state of neurons. The elements interact with each other to create correlations in the firing patterns. The interactions represent the role of synapses in a neural network. The network can work
as a memory. Given a set o f preselected patterns, it is possible to set the interactions
so that these patterns are self-consistent states of the network—the network is stable
when it is in these firing patterns. Even if we change some of the neurons, the original pattern will be recovered. This is an associative memory.
Assume for the moment that the pattern of firing represents a sentence, such as
“To be or not to be,that is the question.”We can recover the complete sentence by presenting only part of it to the network “To be or not to be, that” might be enough. We
could use any part to retrieve the whole,such as,“to be,that is the question.” This kind
of memory is to be contrasted with a computer memory, which works by assigning an
address to each storage location. To access the information stored in a par ticular location we need to know the address. In the neural network memory, we specify part
of what is located there, rather than the analogous address: Hamlet, by William
Shakespeare, act 3, scene 1, line 64.
More central to our discussion,however, is that in a computer memory a particular bit of information is stored in a particular switch. By contrast,the network does
not have its memory in a neuron. Instead the memory is in the synapses. In the model,
there are synapses between each neuron and every other neuron. If we remove a small
part of the network and look at its properties,then the number of synapses that a neuron is left with in this small part is only a small fraction of the number of synapses it
started with. If there are more than a few patterns stored, then when we cut out the
small part of the network it loses the ability to remember any of the patterns, even the
part which would be represented by the neurons contained in this part.
This kind of behavior characterizes emergent properties. We see that emergent
properties cannot be studied by physically taking a system apart and looking at the
parts (reductionism). They can,however, be studied by looking at each of the parts in
the context of the system as a whole. This is the nature of emergence and an indication of how it can be studied and understood.
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 11
Short / Normal / Long
00adBARYAM_29412
12
9/5/00 7:26 PM
Page 12
O ve r v i e w
The above discussion reflects the analysis of a relatively simple mathematical
model of emergent behavior. We can,however, provide a more qualitative discussion
that serves as a guide for thinking about diverse complex systems. This discussion focuses on the properties of a system when part of it is removed. Our discussion of local emergent properties suggested that taking a small part out of a large system would
cause little change in the properties of the small part, or the properties of the large
part.On the other hand, when a system has a global emergent property, the behavior
of the small part is different in isolation than when it is part of the larger system.
If we think about the system as a whole, rather than the small part of the system,
we can identify the system that has a global emergent property as being formed out of
interdependent parts. The term “interdependent” is used here instead of the terms
“interconnected” or “interwoven” used in the dictionary definition of “complex”
quoted in Section 0.1, because neither of the latter terms pertain directly to the influence one part has on another, which is essential to the properties of a dynamic system.
“Interdependent” is also distinct from “interacting,” because even strong interactions
do not necessarily imply interdependence of behavior. This is clear from the macroscopic properties of simple solids.
Thus, we can characterize complex systems through the effect of removal of part
of the system. There are two natural possibilities. The first is that properties of the part
are affected, but the rest is not affected. The second is that properties of the rest are affected by the removal of a part. It is the latter that is most appealing as a model of a
truly complex system. Such a system has a collective behavior that is dependent on the
behavior of all of its parts. This concept becomes more precise when we connect it to
a quantitative measure of complexity.
0.5.2 Complexity
The second concept that is central to complex systems is a quantitative measure of
how complex a system is. Loosely speaking, the complexity of a system is the
amount of information needed in order to describe it. The complexity depends on
the level of detail required in the description. A more formal definition can be understood in a simple way. If we have a system that could have many possible states,
but we would like to specify which state it is actually in, then the number of binary
digits (bits) we need to specify this particular state is related to the number of states
that are possible. If we call the number of states Ω then the number of bits of information needed is
log2(Ω)
I
(0.5.1)
To understand this we must realize that to specify which state the system is in, we must
enumerate the states. Representing each state uniquely requires as many numbers as
there are states. Thus the number of states of the representation must be the same as
the number of states of the system. For a string of N bits there are 2N possible states
and thus we must have
Ω
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
2N
(0.5.2)
Pg. No. 12
Short / Normal / Long
00adBARYAM_29412
9/5/00 7:26 PM
Page 13
C o nc e pts : E merge nce an d co mpl ex i t y
13
which implies that N is the same as I above. Even if we use a descriptive English text
instead of numbers,there must be the same number of possible descriptions as there
are states, and the information content must be the same. When the number of possible valid English sentences is properly accounted for, it turns out that the best est imate of the amount of information in English is about 1 bit per character. This means
that the information content of this sentence is about 120 bits, and that of this book
is about 3 106 bits.
For a microstate of a physical system, where we specify the positions and momenta of each of the particles, this can be recognized as proportional to the entropy
of the system, which is defined as
S
k ln(Ω)
k ln(2)I
(0.5.3)
where k 1.38 10 23 Joule/˚Kelvin is the Boltzmann constant which is relevant to
our conventional choice of units. Using measured entropies we find that entropies of
order 10 bits per atom are typical. The reason k is so small is that the quantities of matter
we typically consider are in units of Avogandro’s number (moles) and the number of
bits per mole is 6.02 1023 times as large. Thus, the information in a piece of material is of order 1024 bits.
There is one point about Eq.(0.5.3) that may require some clarification. The positions and momenta of particles are real numbers whose specification might require
infinitely many bits. Why isn’t the information necessary to specify the microstate of
a system infinite? The answer to this question comes from quantum physics, which is
responsible for giving a unique value to the entropy and thus the information needed
to specify a state of the system. It does this in two ways. First, it tells us that microscopic states are indistinguishable unless they differ by a discrete amount in position
and momentum—a quantum diff erence given by Planck’s constant h. Second, it indicates that particles like nuclei or atoms in their ground state are uniquely specified
by this state,and are indistinguishable from each other. There is no additional information necessary to specify their internal structure. Under standard conditions, essentially all nuclei are in their lowest energy state.
The relationship of entropy and information is not accidental, of course, but it is the
source of much confusion. The confusion arises because the entropy of a physical system is largest when it is in equilibrium. This suggests that the most complex system is a
system in equilibrium. This is counter to our usual understanding of complex systems.
Equilibrium systems have no spatial structure and do not change over time. Complex
systems have substantial internal structure and this structure changes over time.
The problem is that we have used the definition of the information necessary to
specify the microscopic state (microstate) of the system rather than the macroscopic
state (macrostate) of the system. We need to consider the information necessary to
describe the macrostate of the system in order to define what we mean by complexity. One of the important points to realize is that in order for the macrostate of the
system to require a lot of information to describe it,there must be correlations in the
microstate of the system. It is only when many microscopic atoms move in a coherent fashion that we can see this motion on a macroscopic scale. However, if many
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 13
Short / Normal / Long
00adBARYAM_29412
14
9/5/00 7:26 PM
Page 14
O ve r v i e w
microscopic atoms move together, the system must be far from equilibrium and the
microscopic information (entropy) must be lower than that of an equilibrium system.
It is helpful, even essential, to define a complexity profile which is a func tion of
the scale of observation. To obtain the complexity profile, we observe the system at a
particular length (or time) scale,ignoring all finer-scale details. Then we consider how
much information is necessary to describe the observations on this scale. This solves
the problem of distinguishing between a microscopic and a macroscopic description.
Moreover, for different choices of scale, it explicitly cap tures the dependence of the
complexity on the level of detail that is required in the description.
The complexity profile must be a monotonically falling function of the scale.This
is because the information needed to describe a system on a larger scale must be a subset of the information needed to describe the system on a smaller scale—any finerscale description contains the coarser-scale description. The complexity profile characterizes the properties of a complex system. If we wish to point to a particular
number for the complexity of a system,it is natural to consider the complexity as the
value of the complexity profile at a scale that is slightly smaller than the size of the system itself. The behavior at this scale includes the movement of the system through
space, and dynamical changes of the system that are essentially the size of the system
as a whole. The Earth orbiting the sun is a useful example.
We can make a direct connection between this definition of complexity and the
discussion of the formation of a complex system out of parts. The complexity of the
parts of the system are described by the complexity profile of the system evaluated on
the scale of the parts. When the behavior of the system depends on the behavior of the
parts, the complexity of the whole must involve a description of the parts, thus it is
large. The smaller the parts that must be described to describe the behavior of the
whole, the larger the complexity of the entire system.
0.6
For the Instructor
This text is designed for use in an introductory graduate-level course, to present various concepts and methodologies of the study of complex systems and to begin to develop a common language for researchers in this new field. It has been used for a onesemester course, but the amount of material is large, and it is better to spread the
material over two semesters.A two-semester course also provides more opportunities
for including various other approaches to the study of complex systems, which are as
valuable as the ones that are covered here and may be more familiar to the instructor.
Consistent with the objective and purpose of the field,students attending such a
course tend to have a wide variety of backgrounds and interests. While this is a positive development, it causes difficulties for the syllabus and framework of the course.
One approach to a course syllabus is to include the introductory material given
in Chapter 1 as an integral part of the course. It is better to interleave the later chapters with the relevant materials from Chapter 1. Such a course might proceed:1.1–1.6;
2; 3; 4; 1.7; 5; 6; 7; 1.8–1.10; 8; 9. Including the materials of Chapter 1 allows the dis-
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 14
Short / Normal / Long
00adBARYAM_29412
9/5/00 7:26 PM
Page 15
For the instructor
15
cussion of important mathematical methods,and addresses the diverse backgrounds
of the students. Even if the introductory chapter is covered quickly (e.g., in a onesemester course),this establishes a common base of knowledge for the remainder of
the course. If a high-speed approach is taken,it must be emphasized to the students
that this material serves only to expose them to concepts that they are unfamiliar with,
and to review concepts for those with prior knowledge of the topics covered.
Unfortunately, many students are not willing to sit through such an extensive (and intense) introduction.
A second approach begins from Chapter 2 and introduces the material from
Chapter 1 only as needed. The chapters that are the most technically difficult,and rely
the most on Chapter 1,are Chapters 4 and 5. Thus, for a one-semester course,the subject of protein folding (Chapters 4 and 5) could be skipped. Then much of the introductory material can be omitted, with the exception of a discussion of the last part of
Section 1.3,and some introduction to the subject of entropy and information either
through thermodynamics (Section 1.3) or information theory (Section 1.8), preferably both. Then Chapters 2 and 3 can be covered first, followed by Chapters 6–9, with
selected material introduced from Chapter 1 as is appropriate for the background of
the students.
There are two additional recommendations.First,it is better to run this course as
a project-based course rather than using graded homework. The varied backgrounds
of students make it difficult to select and fairly grade the problems. Projects for individuals or small groups of students can be tailored to their knowledge and interests.
There are many new areas of inquiry, so that projects may approach research-level
contributions and be exciting for the students. Unfortunately, this means that students may not devote sufficient effort to the study of course material,and rely largely
upon exposure in lectures. There is no optimal solution to this problem. Second,if it
is possible,a seminar series with lecturers who work in the field should be an integral
part of the course. This provides additional exposure to the varied approaches to the
study of complex systems that it is not possible for a single lecturer or text to provide.
# 29412 Cust: AddisonWesley Au: Bar-Yam
Title: Dynamics Complex Systems
Pg. No. 15
Short / Normal / Long