Lecture 11
• Network terminology and structural characteristics
• Motifs (patterns of directed -and non-directed- links and a
connection to function)
• A Complex system representation: hierarchy of function
• Coarse-Graining (abstractions of function hierarchically
described) and PGNM
• Return to modularity discussion
• (Introduction to models (lecture 12 material as time permits)
• Electric Power student team report #1
Professor C. Magee, 2006
Page 1
Network Analysis Terminology -notated
• node (vertex), link (edge) CM2,
• size, sparseness, metrics CM2
• degree, average degree, degree
sequence DW4
• directed, simple DW4
• geodesic, path length, graph
diameter DW4
• transitivity (clustering), DW10
connectivity, reciprocity CM6
• centrality (degree, closeness,
betweenness, information,
eigenvector) CM6
• prestige, acquaintance CM6
• ideal graphs (star, line, circle,
team) CM6
• degree distribution, power laws,
exponents, truncation, CM6
• Models (random, “small world”,
poisson, preferential attachment)
• constraints, rewiring DW7
• Hierarchy DW7, JM8&9, CM11
• community structure, cliques,
homophily, assortative mixing,
degree correlation coefficient
DW10
• motifs, coarse- graining CM11
• navigation, search, epidemics and
cascades
• self-similarity, scale-free, scalerich DW10,CM11
• dendograms, cladograms and
relationship strength
Professor C. Magee, 2006
Page 2
Motifs
• Milo et al. first extended the concept beyond sociological
networks in a 2002 article in Science titled: “Network Motifs:
Simple building blocks of Complex Networks”,
• They defined motifs in this paper as patterns of
interactions that occur at significantly higher rates in an
actual network than in randomized networks and
developed an algorithm for extracting them from (directed)
networks
Professor C. Magee, 2006
Page 3
A
A
5
B
B
4
2
1
7
9
11
12
13
Motif:
1
8
16
10
11
14
12 13
4
3
6
3
2
7
1
8
16
9
10
2
15
11
12
13
5
4
3
6
14
2
7
1
7
1
8
16
8
16
9
15
14
10
Real Network
15
5
15
14
7
4
5
6
2
9
16
8
10
3
6
3
6
4
5
11 12
13
15
14
9
10
11
12
13
Randomized Network
Schematic of network motif detection. Motifs are found in the real
network (A) much more frequently than in a ensemble of random networks (B)
Professor C. Magee, 2006
Page 4
Figure by MIT OCW.
Motifs b
• Milo et al. first extended the concept in a 2002 article in
Science titled: “Network Motifs: Simple building blocks of
Complex Networks”,
• They define motifs as patterns of interactions that are
significantly higher than in randomized networks
• They studied 19 networks (in six different classes)
• For 2 gene transcription networks they found that the
two different transcription systems showed the same
motifs
Professor C. Magee, 2006
Page 5
Network
Nodes
Edges
Nreal Nrand +
_ SD Z score Nreal Nrand +
_ SD Z score Nreal Nrand +
_ SD Z score
X
X
Gene
Regulation
(transcription)
Y
Y
Feedforward
loop
Bi-fan
Z
W
Z
E. coli
424
S. cerevisiae* 685
519
1,052
40
70
_3
7+
_4
11 +
10
14
203
1812
_ 12
47 +
_ 40
300 +
13
41
Figure by MIT OCW.
The number of times these two motifs occur is more than 10 standard
deviations greater than their mean number of appearances in randomized
networks. None of the other 13 three node possible patterns or any other of
the 199 4 node possible patterns appear more than the mean plus 2 standard
deviations of their appearance in randomized networks
Professor C. Magee, 2006
Page 6
Concentration of Feedforward Loop
0.015
Real
Random
0.01
0.005
0
150
200
250
300
Subnetwork Size
350
400
Dependence of Detectability of the 3 Node Motif on the Size of the E. coli
Network Studied
Professor C. Magee, 2006
Page 7
Figure by MIT OCW.
Motifs c
• Milo et al. first extended the concept in a 2002 article in
Science titled: “Network Motifs: Simple building blocks of
Complex Networks”,
• They define motifs as patterns of interactions that are
significantly higher than in randomized networks
• They studied 19 networks (in six different classes)
• For 2 gene transcription networks they found that the
two different transcription systems showed the same
motifs
• For 8 electronic circuits (in 2 classes), they found
Professor C. Magee, 2006
Page 8
Network
Nodes
Edges
Nreal Nrand +
_ SD Z score Nreal Nrand +
_ SD Z score Nreal Nrand +
_ SD Z score
X
X
Electronic Circuits
(forward logic chips)
Y
Y
Feedforward
loop
X
Bi-fan
Z
Y
Biparallel
Z
W
W
Z
$15850
$38584
$38417
$9234
$13207
10,383
20,717
23,843
5,844
8,651
14,240
34,204
33,661
8,197
11,831
Electronic Circuits
(digital fractional
multipliers)
$208
$420
$838 +
+
122
252
512
424
413
612
211
403
_2
2+
_3
10 +
_2
3+
_1
2+
_1
2+
X
189
399
819
285
120
400
140
225
Three-node
feedback
loop
Y
Z
10
20
40
_1
1+
_1
1+
_1
1+
9
18
38
1040
1739
2404
754
4445
_1
1+
_2
6+
_1
1+
_1
1+
_1
1+
X
Y
Z
W
4
10
22
_1
1+
_1
1+
_1
1+
1200
800
2550
1050
4950
Bi-fan
3.8
10
20
480
711
531
209
264
_1
2+
_2
9+
_2
2+
_1
1+
_1
2+
X
Y
Z
W
5
11
23
335
320
340
200
200
Four-node
feedback
loop
_1
1+
_1
1+
_1
1+
5
11
25
The extremely high ratios for the motifs in these cases (even at small size) can probably be interpreted as evidence
of design intent and for these small technological systems the importance of available modules in such systems
(lecture 18) probably accounts for the reuse of the "motifs" in the variety of circuits of the same class.
Professor C. Magee, 2006
Page 9
Figure by MIT OCW.
The extremely high ratios for the motifs in these cases (even at small
size) can probably be interpreted as evidence of design intent and
for these small technological systems the importance of available
modules in such systems (lecture 18) probably accounts for the
reuse of the same “motifs” in the variety of circuits of the same class.
Professor C. Magee, 2006
Page 9
Motifs d
• Milo et al. first extended the concept in a 2002 article in Science
titled: “Network Motifs: Simple building blocks of Complex
Networks”,
• They define motifs as patterns of interactions that are
significantly higher than in randomized networks
• They studied 19 networks (in six different classes)
• For 2 gene transcription networks they found that the two
different transcription systems showed the same motifs
• For 8 electronic circuits (in 2 classes), they found
reproducible motifs at high concentration for each class of
circuit studied
• One interesting conclusion is that the technique can be applied to
networks with variable nodes and links
• A second interesting conclusion coming from comparison of
neurons, genes, food webs and electronic circuits is
• “Information processing seems to give rise to significantly
different structures than does energy flow.” The possible
relevance to lecture 7 and past Whitney work is intriguing and
addressing it would involve a research question
Professor C. Magee, 2006
Page 10
Motifs e
• In the software tools and example data section of the web site,
you can now find “mfinder manual”
• This entry has links to mfinder which is software (free
download) for detecting motifs on networks (PC, Windows
XP and Linux versions available)
• Also comes with mDraw which allows visualization of
results of mfinder.
• Also contains network randomization methods
• Biological, electronic (and social networks) have been found to
have motifs and in many cases, the motifs have been valuable
in understanding such systems.
• Why might electronic and biological networks in particular
show motifs? What factors or constraints are important in
these systems?
Professor C. Magee, 2006
Page 11
Function on a local scale
• The motifs shown for the electronic circuits (and the biological
systems) seem to show evidence of functionality imbedded
within the network and pursuing a hierarchy of function within
technological networks is one interesting avenue suggested by
this work.
• The following slides are a brief discussion of an approach used
to estimate complexity of various systems attending to
hierarchy, interactions and function (Masters thesis of PierreAlain Martin)
• After that, we will return to looking at hierarchies of motifs on
various levels which is called coarse-graining in the literature
(hierarchy of function?)
Professor C. Magee, 2006
Page 12
Aspects of Complexity
•
•
•
•
•
Number of elements (scale)
Number of interactions
Patterns of interactions
Number and interaction of hierarchical levels
Scope
• # of functions (and their interactions)
• # of time scales (and their interactions)
• Feedback and diverse time delays
• # of spatial scales (and their interactions)
• In our “network approximations”, we have deliberately started with
the simple end (to do otherwise risks immediate non calculability)
and the question is how much complexity must be added for these to
be useful in our systems for our purposes.
Professor C. Magee, 2006
Page 13
The drivers of Complexity in model and
representation discussed
•
•
•
•
CIPD
Number of elements
Number of links (JM idealization)
Number of basic functions (new here)
Hierarchy of these basic functions (new here)
Professor C. Magee, 2006
Page 14
Functional Classification Matrix
Process/Operand
Matter
(M)
Energy
(E)
Information (I)
Value
(V)
Transform or Process (1)
GE Polycarbonate
Manufacturing
Plant
Pilgrim Nuclear
Power Plant
Intel Pentium V
N/A
Transport or Distribute (2)
FedEx Package
Delivery
US Power Grid
System
AT&T
Telecommunication
Network
Intl Banking
System
Store or
House (3)
Three Gorge Dam
Three Gorge Dam
Boston Public
Library (T)
Banking Systems
Exchange or Trade (4)
eBay Trading
System (T)
Energy Markets
Reuters News
Agency (T)
NASDAQ
Trading System
(T)
Control or Regulate (5)
Health Care
System of France
Atomic Energy
Commission
International
Standards
Organization
US Federal
Reserve (T)
Professor C. Magee, 2006
Page 15
System Representation
System
CIPD
Subsystems
Professor C. Magee, 2006
Page 16
Convention
Elements
Transform or
Process
Transport or
Distribute
Store or
House
Exchange or
Trade
1
2
Control or
Regulate
Level 1
Level 2
Links
Directional
Bidirectional
CIPD
3
4
Level 3
Level 4
I – Information
E – Energy
M – Matter
V – Value
Professor C. Magee, 2006
Page 17
5
Level 5
I
M
I
V
I
E
I
Reference Decomposition
I
I
V
I
I
Level 1
I
V
I
Level 2
I
E
I
E
V
I
Level 3
CIPD
Professor C. Magee, 2006
Page 18
M
I
I
M
I
V
I
E
I
Reference Decomposition
M
I
I
E
V
I
Reference Decomposition
The Reference Decomposition is a map of interconnected mono-functional elements
CIPD
Professor C. Magee, 2006
Page 19
Recommended complexity metric
Counting only the non-identical elements
Ln ((Vj + 1) x (H j + 1))
L
NI
C = ×∑Ej
E j
Ln (4)
Normalized Density
Product
Microscopic
V is the number of basic functions in the system
H is the number of hierarchical layers to decompose
the system to monofunctional elements
CIPD
Professor C. Magee, 2006
Page 20
I
E
I
Filter
Switch
E
I
E
Switch
RCMS
Remote Control
&
Monitoring
System
I
I
Aircraft Target
Channel RGN
I
I
Interface
Board
I
I
I
I
I
Interference
Processing
I
Interface
I
Switch
I
Attenuator
E
Converter
E
I
E
I
E Feed Horn
Reflector
Assembly
E
WaveguideM
Connection
E
E
Power
Supply
Control
E
Attenuator
E
Converter
E
Switch
E
Attenuator
E
Pedestal
E
Attenuator
E
E
E
E
E
Circulator
E
E
E
I
E
Attenuator
Multi-channel
Transmission Encoder
B
E
E
E
E
E
Power Module
E
I
Combiner
Power Module
I
Power Module
Preamplifier
B
E
Attenuator
E
Converter
E
Divider
I
I
E
Switch
E
E
Power Module
I
I
E
Attenuator
E
Switch
E
E
I
E
E
I
E
I
Power Module
Interface &
Safety Board B
Dispatcher
1->4
E
E
I
I
Switch
Attenuator
I
E
E
RCMS
Interface Board
RCVR
PROT
Switch
I
Interface B
I
E
I
Power Module
I
Attenuator
Interface A
I
I
M
Fan
I
E
Attenuator
Encoder
A
BITE
E
Power
Supply
E
Power Module
I
Interface &
Safety Board A
E
Dispatcher
1->2
Power Supply
E B
I
I
E
Power Supply
A
E
I
I
Preamplifier
A
I
E
Fan
Dispatcher
1->4
I
RCVR
PROT
E
Switch
I
Driver
Coupler
E
RCVR
PROT
Switch
Attenuator
E
I
I
Interface
I
I
I
Output Interface
Voyager Synchronizer
I
Interface
Process
I
I
I
I
Formatting
I
I
CFAR
I
Max
Threshold
I
I
Theoretical
Range
Threshold
Clutter MapI Modulus
Integration
Tables
I
I
I
Adaptative
Analogous
STC
Bank Select
Thresholding I
Doppler Filter
Propagation
Compensation
Phase/Amplit.
I Phase/Amplit.
I
I
I
IFilter
I
6 Level Filter
Compensation
Compensation I
Selection Selection Map
Measurement
Ground
I
I Clutter
I
Echo Detection
Filtering
I
I
Digital
Pulse
I
I
Switch
Compression
Compression
I
I
Switch I
Switch
I Receiv
OsIProc
Chrono.
I
Interface
Formatting
I
Scan to Scan
Integration
I
I
I
I
Digital
Switch
Mixer Mixer
Gener0 I
I
I
I
Convert
DAC I
I MixerConvert
I
I
I
PSU-2500S
Power Supply Unit
I
Merge
I Format
I
Switch
Filter
Os
I
I
1 ISwiOs tch x
2I
Switch
Os
SwiI
I
I
1
Os tch x
I
2
Convert I
I
I
Filter
E
E
Weather Data
Processing
I
E
Switch
E
Aircraft Target
Data Processing
Weather Target
Channel RGN
I
I
Filter
I
I
Switch
Output Video
E
Low Voltage
Power Supply
RCVR
PROT
Low Voltage
Power Supply
Power Module
E
E
E
E
I
E
I
Output Data
Formatting
Output Video
Aircraft Target
Data Processing
E
I
I
I
I
E
E
Filter
I
E
Filter
I
Switch
Switch
Main
Distribution
AE 2000
E
E
E
Weather Target
Channel RGN
I
I
Filter
I
I
I
Switch
Aircraft Target
Channel RGN
Filter
E
I
OsI
I
1 Swix
I
I
I I
Os tch
Os ProcReceiv
2I
Switch
Os
Chrono.
1 SwiI
Os tchI x
2I
I
Interface
I
I Convert I
Board
Digital
I
Switch
Mixer Mixer
Gener0
I
I
I
I I
I
ConvertMixerConvert
DAC
I
I
PSU-2500S
Power Supply Unit
I
I
Output Interface
Formatting
I
I
I
I
CFAR
Threshold
I
I
I
Clutter Map Modulus
I
Adaptative
I
Bank Select
Doppler Filter
I I
Phase/Amplit.
Phase/Amplit.
ICompensation
I
Compensation
Measurement
I
I
E
E
Switch
I
Switch
I
I
I
Voyager Synchronizer
I
Echo Detection
I
I
I
Digital
Compression
I
Switch
I
I
Interference
Processing
I
Professor C. Magee, 2006
Page 21
Interface
I
I
I
I
I
BITE
I
I
E
Power
Supply
M
Fan
I
Driver
I
Coupler
C (STAR 2000) =
CIPD
Ln ((Vj + 1) x (H j + 1))
L
NI
×∑Ej
E j
Ln (4)
Professor C. Magee, 2006
Page 22
= 350.20
Complexity Estimation and Technological
System Representation by Networks
• Martin’s thesis work may be a superior way to calculate
complexity and worked well for the two cases he studied.
• Much more application to other systems is needed to determine
its utility.
• For today’s lecture purpose, we introduce it to allow discussion
of node differentiation by function and by hierarchical level.
In addition, we want to note the possible utility of the
representation developed in that work as a basis for developing
more effective (yet tractable) network models for technological
systems.
Professor C. Magee, 2006
Page 23
Self-similarity and self-dissimilarity
• Wolpert and Macready(2000) introduced the concept of selfdissimilarity as a complexity metric
• They defined self-dissimilarity as “the variability of interaction
patterns of a system at different spatio-temporal scales”
• Wolpert and Macready invented relatively elaborate methods
for statistically applying their concept and demonstrate it only
through numerical simulations.
• Itzkovitz et. al (2004) have recently developed a method they
call “coarse-graining” based on their prior work on motifs.
This method also assesses self-dissimilarity and has been
applied to biological and technological networks.
Professor C. Magee, 2006
Page 24
Coarse-Graining
• Itzkovitz et. al. investigate Coarse-Graining as an objective
means for “reverse-engineering” that can be applied even when
the lower level functional units are unknown (biological
focus).
• The coarse-grained version of a network is a new network with
fewer elements. This is achieved by replacing some of the
original nodes by CGU’s (patterns of node interactions at the
level being examined-motifs chosen somewhat differently).
• Itzkovitz et. al. apply simulated annealing to arrive at an
optimum set of CGU’s (minimize the “vocabulary” of CGU's
and the complexity of the chosen CGU’s while maximizing the
coverage of the original network by the coarse-grained
description).
Professor C. Magee, 2006
Page 25
Optimal selection of CGU’s
• Complexity defined (number of “ports”for a node -equivalent
to JM )
• The number of ports in the network (system) covered by a
motif group selected
• A scoring function which can be maximized to optimize
coverage and favors CGU’s which have high coverage and
many internal nodes (and few external mixed nodes) is
Professor C. Magee, 2006
Page 26
Coarse-Graining
• Itzkovitz et. al. investigate Coarse-Graining as an objective
means for “reverse-engineering” that can be applied even when
the lower level functional units are unknown (biological
focus).
• The coarse-grained version of a network is a new network with
fewer elements. This is achieved by replacing some of the
original nodes by CGU’s (patterns of node interactions at the
level being examined-motifs chosen somewhat differently).
• Itzkovitz et. al. apply simulated annealing to arrive at an
optimum set of CGU’s (minimize the “vocabulary” of CGU's
and the complexity of the chosen CGU’s while maximizing the
coverage of the original network by the coarse-grained
description).
• Applying this algorithm to an electronic circuit..
Professor C. Magee, 2006
Page 27
a
c
a
b
b
d
e
c
d
e
GND
Figure by MIT OCW.
Transistor level map of an 8 bit binary counter used in a digital fractional multiplier. Highlighted is a sub-graph that
represents the transistors that make up one NOT gate. Examining possible motifs up to 6 nodes shows…
Professor C. Magee, 2006
Page 28
93
68 48
12
10
17 17
10
9
8 13
10
9
13
9
13
13
13 11
9
9
6
5
7
5
4
4
Figure by MIT OCW.
Two sets of possible optimal motif-based CGU’s. The solid boxes choice can be arranged to
arrive at a “gate-level coarse-graining”
Professor C. Magee, 2006
Page 29
Gate Level
(P=99, E=153)
Transistor Level
(P=516, E=686)
Figure by MIT OCW.
In the transistor level, nodes represent transistor junctions. In the gate
level, nodes are CGU’s, made of transistors, each representing a logic gate.
Shown is the CGU that corresponds to a NAND gate. Re-applying the
coarse-graining optimization sequentially yields 2 more levels..
Professor C. Magee, 2006
Page 30
Counter Level
(P=42, E=56)
Flip-Flop
Level
(P=59, E=97)
CGU
1
CGU 1
CGU
2
CGU
1
CGU
3
CGU
1
CGU
2
CGU 3
CGU 2
CGU
4
CGU
3
CGU 2
D Q
D Q
D Q
D Q
Q
Q
Q
Q
Gate level
(P=99, E=153)
Figure by MIT OCW.
Professor C. Magee, 2006
Page 31
Coarse-Graining b
• Itzkovitz et. al. investigate Coarse-Graining as an objective
means for “reverse-engineering” that can be applied even when
the lower level functional units are unknown (biological
focus).
• The coarse-grained version of a network is a new network with
fewer elements. This is achieved by replacing some of nodes
by GCU’s (patterns of node interactions at the level being
examined.
• Itzkovitz et. al. apply simulated annealing to arrive at an
optimum set of GCU’s (minimize the “vocabulary” of GCU’s
while maximizing the coverage of the original network by the
coarse-grained description).
• Applying this algorithm to an electronic circuit, one finds a
four level description which has variable functional
significance and self-dissimilarity at each level
Professor C. Magee, 2006
Page 32
NAND
/NOT
AND
OR
NOR
Gate Level
Nodes are SIUs made of
transistors
/
D-flipflop+gate
Flip-Flop Level
D
Q
Q
Nodes are gates or SIUs made
of gates
Counter Level
CGU 1,2,3
CGU 4
Nodes are gates or CGUs made
of gates+flip-flops
Figure by MIT OCW.
Professor C. Magee, 2006
Page 33
Self-dissimilarity at multiple levels in the electronic circuit.
This change of patterns with level apparently applies to all
biological and technological networks studied thus far.
Professor C. Magee, 2006
Page 33
Coarse-Graining c
• Itzkovitz et. al. investigate Coarse-Graining as an objective means
for “reverse-engineering”.
• The coarse-grained version of a network is a new network with
fewer elements.
• Itzkovitz et. al. apply simulated annealing to arrive at an optimum
set of GCU’s
• Applying this algorithm to an electronic circuit, one finds a four
level description which has variable functional significance and selfdissimilarity at each level
• Note the fundamental difference between Coarse-Graining and
algorithms for detection of community structure:
• Community structure algorithms try to optimally divide networks
into sub-graphs with minimal interconnections but these subgraphs are distinct and complex
• Coarse-Graining seeks a small dictionary of simple sub-graph
types in order to elucidate the function of the network in terms
of recurring building blocks
Professor C. Magee, 2006
Page 34
Coarse-Graining c
• Itzkovitz et. al. investigate Coarse-Graining as an objective means
for “reverse-engineering”.
• The coarse-grained version of a network is a new network with
fewer elements.
• Itzkovitz et. al. apply simulated annealing to arrive at an optimum
set of GCU’s
• Applying this algorithm to an electronic circuit, one finds a four
level description which has variable functional significance and selfdissimilarity at each level
• Note the fundamental difference between Coarse-Graining and
algorithms for detection of community structure:
• Community structure algorithms try to optimally divide networks
into sub-graphs with minimal interconnections (modularity1)
but these sub-graphs are distinct and complex
• Coarse-Graining seeks a small dictionary of simple sub-graph
types in order to elucidate the function of the network in terms
of recurring building blocks (modularity 2)
Professor C. Magee, 2006
Page 35
Different Definitions of “Modular” or
“Module” (modified from Whitney)
• You can see different elements and the places where they join
(common engineering view particularly in ME but also in
economics): H. Simon’s near-decomposability(1962).
• Each item does a specific thing (form-function, genotypephenotype in a one-to-one relationship) (Suh, Altenberg and
also many biological papers discussing modularity)
• You need only know how to use them and don’t need to know
what’s inside (common engineering view particularly in EE
and software)
• Interconnectedness is concentrated inside them (Alexander,
software design)
• Their links to the outside are standardized , or simple and few
(Alexander)
• Modules can be replaced in a system arbitrarily preserving (but
possibly modifying) function: (Plug and Play intuition)
Professor C. Magee, 2006
Page 36
Different Definitions of “Modular” or
“Module”
• You can see different elements and the places where they join
(modularity 1)
• Each item does a specific thing (form-function, genotypephenotype in a one-to-one relationship) (Suh, Altenberg)
(modularity 2)
• You need only know how to use them and don’t need to know
what’s inside (modularity 2)
• Interconnectedness is concentrated inside them
(Alexander)(software design) (modularity 1)
• Their links to the outside are standardized (modularity 2), or
simple and few (Alexander) (modularity 1)
Professor C. Magee, 2006
Page 37
“Better” Definition(s) of Modularity
•
•
•
•
Modularity 1:
• The system can be decomposed into subunits to arbitrary depth
• These subunits can be dealt with separately (to some degree)
• In different domains, such as design, manufacturing, use, errorcorrection, recycling
Modularity 2:
• The functions of the system can be associated with clusters of
physical elements
• in the limit one function:one module
• These elements operate (somewhat) independently
• (They do not have to be physically contiguous)
Merged definition
• The intuitive “Plug and Play” requires both definitions to be
operable (without qualifications in brackets)
There are physical constraints in moderately higher power systems that
prevent modularity without significant qualification (Whitney)
Professor C. Magee, 2006
Page 38
Coarse-Graining d
• Itzkovitz et. al. investigate Coarse-Graining as an objective
means for “reverse-engineering”.
• The coarse-grained version of a network is a new network with
fewer elements.
• Itzkovitz et. al. apply simulated annealing to arrive at an
optimum set of GCU’s
• Applying this algorithm to an electronic circuit, one finds a
four level description which has variable functional
significance and self-dissimilarity at each level
• Note the fundamental difference between Coarse-Graining and
algorithms for detection of community structure
• Note that motifs and coarse-graining have thus far only been
applied to fairly simple technological systems
• Monofunctional from the Martin-Magee perspective and
easily functionally modularized
Professor C. Magee, 2006
Page 39
Research Questions
• Should we apply community structure separation and coarsegraining at different levels for improved understanding (and
design?) of complex technological (and biological) systems?
• Does the form-function relationship only work for relatively
simple systems?
• To what degree, do the two kinds of modularity apply to
different levels of abstraction and/or power/information
differences?
• Hypotheses:
• For a truly complex system, one has multiple functions that
cannot be separately decomposed.
• For such a system, one might decompose (sequentially) by
community structure (interaction density) to the level of
mono-functionality (Martin-Magee representation arrived
at objectively)
• One then could examine the resulting mono-functional
systems by coarse-graining looking for more basic formfunction relationships
Professor C. Magee, 2006
Page 40
Self-similarity and self-dissimilarity b
• Wolpert and Macready(2000) introduced the concept of selfdissimilarity as a complexity metric
• Self-dissimilarity is defined as “the variability of interaction
patterns of a system at different spatio-temporal scales”
• Note that as defined this definition is in a sense counter to the
notion (often loosely defined) of “scale free” which implies (at
least seems to) the notion that structure is repetitive at various
scales
• Wolpert and Macready invented relatively elaborate methods
for statistically applying their concept and demonstrate it only
through numerical simulations
• Itzkovitz et. al (2004) have recently developed a method they
call “coarse-graining” based on their prior work on motifs.
This method also assesses self-dissimilarity and has been
applied to biological and technological networks.
Professor C. Magee, 2006
Page 41
A
B
Hierarchical Organization of
Modularity in Metabolic
Networks
E. Ravasz, A. L. Somera,
D. A. Mongru, Z. N. Oltvai,
A.-L. Baraba´si
SCIENCE VOL 297
30 AUGUST 2002 p 1551
C
A.
B.
C.
Scale free
Modular, not scale free
Nested modular, scale free
Professor C. Magee, 2006
Page 42
Figures by MIT OCW.
Probabilistic Generation of Network
Motifs (PGNMs)
• Based on idealized block models arrived at by assigning roles
to nodes (limited number of roles) and defined relationships
between nodes of differing roles (types). A modified scoring
function is used
• Where g includes generalizations of the CGU’s according to
the block model idealization (an example of generalizations
relative to the BiFan motif is shown in the next slide)
Professor C. Magee, 2006
Page 43
Role 1
1
2
Bifan
Role 2
3
4
[ [
0
0
0
0
0
0
0
0
1
1
0
0
1
1
0
0
G1
Role 1
Role 2
1
G2
2
3
4
5
[ [
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
1
1
0
0
1
1
1
0
0
Role 1
Role 2
1
3
2
4
5
[ [
0
0
0
0
0
0
0
0
0
0
1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
Figure by MIT OCW.
Comparison of this idealization to two cases is shown in
the following slide
Professor C. Magee, 2006
Page 44
Probabilistic Generation of Network
Motifs (PGNMs) II
Role 1
Role 2
[ [
0
1
0
0
A
G2
G1
Role 1
Role 2
1
2
3
4
[ [
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
6
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
1
1
1
0
0
0
7
1
1
0
1
0
0
0
1
1
1
1
0
0
0
Role 1
1
2
Role 1
3
4
[ [
Role 2
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
6
1
1
0
0
0
0
0
1
1
0
0
0
0
0
0
0
1
1
0
0
0
0
0
1
1
0
0
0
7
0
0
1
1
0
0
0
Figure by MIT OCW.
Professor C. Magee, 2006
Page 45
Self-similarity and self-dissimilarity c
• The scale-free modular example is self-similar.
• However, the electronic circuits and biological systems studied
by Itzkovitz et. al are not scale free in that even though the
modules are consistent with one another at a given scale, the
patterns are dissimilar (in the Wolpert/Macready sense) at
different scales (or levels of agglomeration)
• Note that the electronic circuit systems shown to be “scalerich” by Itzkovitz et. al. show power law degree relationships
so the use of the term “scale-free” when power laws is
observed is nonsensical. The lack of correlation between
structure and power laws was mentioned in lecture 6.
• Li et. al and Doyle et al have introduced the phrase “scalerich” partly in response to the work by Itzkovitz and have
developed some other metrics (related to degree correlation, r)
and we will return to this theme in a later discussion of models
of the Internet
Professor C. Magee, 2006
Page 46
References
• 1. R. Milo, S. Shen-Orr, S. Itzkovitz, N. Kashtan, D.
Chklovskii, U. Alon, “Network Motifs: Simple building blocks
of Complex Networks”, Science, Vol. 298, pp 824-827, (2002)
• 2. Pierre-Alain Martin, “A Framework for Quantifying
Complexity and Understanding its Sources: Application to two
Large-Scale Systems” SM thesis, MIT, 2004.
• 3. D. H. Wolpert and W. Macready, “Self-dissimilarity: An
empirically observable complexity Metric”, Unifying themes
in complex systems, New England Complex Systems
Institute(2000). A second paper found on the NASA Moffet
web site has similar information and is titled “Selfdissimilarity as a high dimension complexity measure” (2004?)
• 4. S. Itzkovitz, R. Levitt, N. Kashtan, R. Milo, M. Itzkovitz
and U. Alon, “Coarse-Graining and Self-Dissimilarity of
Complex Networks”, (Oct. 2004)
Professor C. Magee, 2006
Page 47