“Ergodic” (Invariant) Measures Applied to nDimensional, Lag Embeddings of Expanding and Mixing,
Biological Dynamical Systems
Arnold J. Mandell M.D. Multi Modal Imaging Laboratory, MMIL
Department of Psychiatry, UCSD and Fetzer-Franklin Fund
Ornstein theorem: Most/All suitably normalized measures made
on chaotic dynamical systems are equivalent to their informational
entropy, h (which has no single or definitive algorithmic
computation).
Frobenius-Perron theorem: Square matrices with non-negative
entries have at at least one positive eigenvalue, λ ≥ 0 and log λ ≈ h.
Pesin-Young-Mandell (conservation of) : For uniformly & nonuniformly hyperbolic systems, the topological entropy, hT , varies as
the product of the capacity dimension, Dc & the leading Lyapounov
exponent, Λ 1 . hT = Dc Λ1.
Bifurcation
Entropy
Fractal
Leading Lyapounov
Nonlinear Dynamics
Chaos
Google Book Title Word Count From 1750 to 2005.
Note decline of relevant words beginning in the vicinity of 2000.
Adoption of new science characteristically comes late to biology
Dynamical systems as qualitative nonlinear orbital
behavior (in for example, phase portraits)
“The qualitative theory of differential equations”
see classic book by VI Arnold,(1983) global
behavior
Statistical mechanics’ central point is the dependence
on microscopic variables. Distributions/moments
versus .
“Invariant Measure”
“Ergodic theory of dynamical systems”
Topological. metric and nonuniform entropy, the
dimensions, the expansion exponents, graph
theoretic exponents….see Ergodic Theory by
Cornfield, Fommin &Sinai (1982)
Quasi-periodic and
chaotic van der pol.
Phase Portraits
Stretch, fold,
shuffle.
Points get out of
order;
“mixing”Lyapoun
ov > 0
Generic membrane
equations
Chaotic?
Irrational winding
on the torus
Lyapounov ≤ 0
Multiply
periodic
?
Mandell and Selz, 1990
In place of n-tori of stable periodic orbits, we may have:
Homoclinic Orbits Which Join Unstable Fixed Points to Themselves
Recurrence via unstable periodic orbits of increasing lengths and selfsimilar structure. “sizes” are hierarchical and scale with a power law.
“A small disc centered near a homoclinic point includes infinitely many periodic
points of different periods” Poincare; Smale; Yorke; etc.
X
Intersection of
stable and
unstable
manifolds of
unstable
fixed point
WU
Ws
A two dimensional section showing
the unstable and stable manifolds
of an orbital flow meeting at a “homoclinic” point.
UNSTABLE PERIODIC ORBITS, UPOs, (HOMOCLINIC TO
UNSTABLE FIXED POINTS) OF HIPPOCAMPAL NEURON INTERSPIKE
INTERVALS (period one)
Stable and unstable
manifolds support
chaotic
Hippocampal
neuron spikes
dynamical
systems
Interspike
Aattractors. intervals
So et al, 1998 (Steve Schiff’s group)
Return embedding
of ISI; colored
points UPOs
Find periodic
orbit (period one)
Phase portraits
Of ISI; stable and
unstable manifolds
of unstable fixed
points defining
unstable periodic
orbits
An example of phase space reconstruction and measures made on
phase portrait of a physiological time series.
Phase Portrait of Temperature Time
Series
Before a Catch
X (t – 1)
X(t)
Effect of “catch and return” on fresh water
pike
fish temperature dynamics
Phase Portrait of
Temperature Time
SeriesAfter a Catch
X (t-1)
X(t)
Spontaneous, bursting, intermittent patterns of brain stem neuronal
discharge drive subcortical and neocortical membrane fluctuations: both at
several time scales with power spectral power law scaling.
Thanks to Carlson, Foote,
Guillemin,
“…equating entropy increase as the spontaneous
dispersal of energy, namely how much energy is
spread out in a process, or how widely dispersed
it becomes...” Leff, 1996,2007
AG
YM
Phase portraits of IMF3,4,5 of
C16-ssds(i) in AG, control, YM,
intermediate state, and SP, typical
medicated schizophrenic proband.
SP
Modal Descriptions
Frequency (power) Spectrum in Log-Log
plot slope → α
Broomhead-King autocovariance
eigenfunctions, Ψ
Morlet mother wavelet, wavelet
transformation of Ψ, W(Ψ)
Emperical Mode Decomposition →Intrinsic
Mode Functions, EMD →IMF
The continuous power spectrum with complex singularities of a chaotic
attractor (from “hydrodynamic” equation)
Farmer, Crutchfield, Froehling, Packard and Shaw, 1978-89
“The original UCSC chaos kids”
The “funnel”
Positive Lyapounov exponent
Continuous power spectrum
Complex singularities
Unstable periodic orbits
Recursive homoclinic behavior
near unstable fixed points
Chaotic dynamical systems
manifest a variety of power
spectral exponents.
Universality classes??
log(amplitude2)
(i.e., power)
“Universal” scaling laws show up in a wide range of contexts:
Log(frequency)-->
A common manifestation of hierarchical, multiscale, selfsimilar, fractal statistical dynamics is “1/f α noise.” The system
manifests correlations at many scales. This “signature” is
common to many systems with strong (cooperative)
interactions and many degrees of freedom (e.g. the brain’s
electromagnetic systems). It may also accompany distribution
functions with infinite second moments as in a Levy Process
Log-log Power Spectra of Three Minutes (108,000 points) of MEG Central C16 ssds(i) in Ten
Normal Controls
The Spectral Power law, α, approximates the Kolmogorov scaling of 5/3
(1)Use standard Fourier transform of
time series; (2) log transform the
frequency and power axes. (3)
Compute the slope of the middle
third of log-log plot; (4) –slope = α
f--α , Mean α = -1.67± 0.43; Median = -1.59; Var = 0.18; fs = 600/150 ComputHz
To Study the Inverse of the Time-Dependent Wavelengths,
Broomhead-king Decomposition Uses the Leading Eigenvectors of a Lagged
Autocovariance Matrix, Composed with the Original Series to Generate Two or
Three Leading B/K Eigenfunctions
To Observe the Dynamics in Time of Scaling System(s) We Apply the
Wavelet Transformation in which a “Mother” Wavelet is Convolved with the
Data as it is Translated Down the
Series, “b”, at Various Dilations, “a”.
Morlet mother wavelet, w, =
w = sine wave x Gaussian
Intermittent, hierarchical scaling vortices,
we call strudels, the German word for “whirlpools” or “eddies,”
in the 3-5 to 20+ second time scales. Here, two strudels:
Intermittent Vortices in the 3-5 to 20+ second BK eigenfunction time scales.
Four intermittent, hierarchical, scaling strudels, i(S), i = 4, are seen, defined by their
near continuity beginning below the middle scales and, over shorter or longer times,
reaching or exceeding the upper scaling bound of the wavelet graph. Both the incidence
and durations are within the range reported for TUTIs (task unrelated thoughts and
images): 5 to 20+ seconds.
Note how much detail and texture of the time series would be
lost reporting only their means, variance and higher moments
Local field potentials from neocortical pyramidal cell network in taskless,
resting monkey statistically resemble MEG ssds
(Shew and Plenz, 2009)
EEG, multielectrode
Local field
potentials
from
Pyramidal
cell layers
II and III
(self-referential
network)
MEG,
symmetric
sensor
difference
series, ssds
Morlet wavelet decomposition of 14 second,C16 sensor difference sequence
During brief “petite absence” seizure demonstrate spike, dome and
Vertically coherent strudels. Fast spiking drives an expanding flow.
~ 14 seconds
~0.8 Hz
~2.0 Hz
~3-4Hz
>100 Hz
Intermittent 3 to 8 Second Strudel “Absences” Persist Over the
2.3 min of Eyes-Closed Resting Record in Proband, YM
Λ = 0.451; DC =1.61; hT =0.348
memv = 0.2526 (0.572)
α = 2.79; X4=0.441;
To empirically “unpeel” the hierarchical scales revealed in
the log-log power spectra use Huang’s Empirical Mode
Decomposition yielding an array of Intrinsic Mode Functions.
• Identify successive pairs of zero crossings, identify local extrema
• Connect max, min with cubic splines (upper and lower
“envelopes”)
• Compute first mean, m1, of the envelopes
• [ssds]-m1 = h1; h2 = h1 – m2…. ….+ …residue(“sifting”)
• Inter-maxima distance is the local time scale
• Allows real time snap shots of nonstationary, “instantaneous”
fluctuations growing in scale (wavelength) from left to right.
•
hi
I = 0..n
AG
hn
YM
SP
Hilbert-Huang Intrinsic Mode Functions,
IMF1,2,3,4,5 MEG, C16(ssds),16.66 sec
Note Loss of modular amplitudes in proband SP’s intermediate time scales
Invariant Measure Theory
Using “Blind Boys and the Elephant” Metrics
Battery of measures being applied to resting, eyes closed,
asymmetric MEG sensor difference sequences: ssds:
symmetric sensor difference sequences
Note that axiosymmetric B fields would cancel
Probability
distribution
Symbolic
dynamics;
Information
Phase
space
geometry
Modal
analysis
Algorithmic Themes in Quantifying Global Fine
Structure in Expanding & Mixing Dynamical Systems
Partition (“generating”?) , transition matrix, with or without symbol
substitution, and symbolic dynamics
1.
2.
3.
4.
5.
6.
The quantities sought are fractional exponents (logarithms)
These quantities may indicate hierarchical scaling relations, “self
similarity” capacity, correlation, Hausdorff dimensions.Dc
Logarithmic relation between the measure (abscissa) and the
measurement (ordinate), capacity, correlation, Hausdorff dimensions
Dc.
From the growth rate of the trace of the exponentiated transition
incidence matrix, Rate of appearance of new recursive orbits,
topological entropy, hT
From the transition matrix, distribution of weights in normalized,
exponentiated Markoff matrix,metric entropyhM
Using the phase space reconstruction, determination of the separation
rate of recursively renewed “nearby initial conditions.”. Leading
(positive) Lyapounov exponent, λ or Λ
Measures: Λ, Dc, hT
“box” capacity dimension
Complexity of manifold
Log
measurement
Slope
Sensitivity to initial
conditions, Lyapounov
Exponent (rate of
Expansion)
Log measure (scales)
Partition→transition
incidence matrix→growth
rate of trace while
exponentiating matrix
Growth rate of new recursive orbits…..topological entropy
Multi-parameter quantitative-qualitative
descriptions of dynamical systems.
Leading Lyapounov exponent; divergence, expansion, mixing
Power spectral scaling exponent:log-log slope, global, scaling
Topological entropy; rate of new “loop” formation
Metric entropy: distribution of weights on loops
Capacity dimension: complexity of manifold of support
Non-uniformity ||top-met| difference
Measureable entropy manifold volume top x Lya x dimension
Unwinding number lags to asymptotic capacity dimension
Skew distributional asymmetry
Kurtosis peakedness and heavy tail
Levy exponent rate of converrgence of tail of distribution
Hurst exponent persistence vs antipersistent
Measurable entropy manifold volume, MEMV = [hT l DC ]
Thereom: hT = lDC Pesin, Young, Manning,
topological entropy = product of the leading lyapounov exponent and the
capacity dimension;
Premise: conservation of brain entropy
Λ = log (rad B/rad Bi )
hT = log(#Bi)
d = lim (me)/log(e)
Intuition about the
relations between
entropy, Lyapounov
and dimension
Λ increases
dC increases
hT increases
Fixing entropy, hT, log (3)
dimension, d, goes down
as the Lyapounov increases
Fixing Λ, hT and dC go up.
REDUCED VOLUME OF FRONTAL LOBE (F14 SSDS) MEASUREABLE ENTROPY
MANIFOLD VOLUME, MEMV, IN PROBANDS
Implicit function representation of the Pesin-Young Ansatz: ld=hT
(means of 10 SSDS, each of which were computed on 32,000
points)
l Lyapounov exponent
Topological
Entropy hT
d capacity dimension
Ten control subjects
memv = 3.13 log units
Ten medicated
schizophrenic patients
memv = 2.37 log units
Third new findings: aggregate measure relations and memv decreased in
“abnormal” brain plasmas.
Phase Portraits and Recurrence Plots of the Three
Leading Autocovariance Matrix Eigenfunctions
New: MEG-ssds Measure Suite
Data: symmetric sensor
difference series, ssds,
Four minutes 600/150 Hz
B/K phase
Space
Recurrence
plot
Topological entropy, hT , capacity dimension, DC , and
leading Lypounov exponent, Λ, and their Cartesian product,
measurable entropy manifold volume, memv = Π[Λ DC hT ] on ssds(i)
discriminate controls from probands.
x ≡ L, y ≡ DC, Z ≡ hT
Aggregate of relative changes in measures
From Intermittency to Transitivity in Neuropsychobiological Flows
AJ M, Am. J. Physiol. 245: R484-R494, 1983
•
•
•
•
•
•
•
•
•
•
Intermittency
∂(Λ) > 0
∂(α) < 0
∂(hT) > 0
∂(hM) > 0
∂|hT – hM| < 0
∂(DC) > 0
∂(σ3) < 0
∂(σ4) < 0
∂( memv ) > 0
Transitivity
A few (personal) references
Mandell, AJ (1987) Dynamical complexity and pathological order in the
cardiac monitring problem. Physica D 27:235-242.
Mandell, AJ & Selz, KA(1993) Brain stem neuronal noise and neocortical
resonance. J. Stat. Phys. 70:355-373.
Mandell, AJ & Selz, KA(1997) Entropy conservation as hT = Λ*Dc.
(1997) Chaos 7:67-81.
Mandell, AJ & Shlesinger, MF(1990) Lost choices, parallelism and
topological entropy decrements in neurobiological aging. AAAS
Washington.
Selz, KA & Mandell, AJ. (1991) Bernoulli partition equivalence of
intermittent neuronal discharge patterns. Int. J. Bifurcation Chaos
1:717-722.
Mandell, AJ (2013) Intermittent turbulent eddies in brain magnetic fields.
Chaos, Solitons & Fractals 55:95-101.
Robinson, S, Mandell, AJ & Coppola, R (2013) Spatiotemporal imaging of
complexity.Frontiers in Comp. Neurosci. 6:1-14 (#101).
New: MEG-ssds Measure Suite
Data: symmetric sensor
difference series, ssds,
Four minutes 600/150 Hz
B/K phase
Space
Recurrence
plot
Intermittent 3 to 8 Second Strudel “Absences” Persist Over the
2.3 min of Eyes-Closed Resting Record in Proband, YM
Λ = 0.451; DC =1.61; hT =0.348
memv = 0.2526 (0.572)
α = 2.79; X4=0.441;
Daydreaming, Thought Blocking and Strudels in the
Task-free, Resting Consciousness of the Brain Plasma’s
Magnetic Fields*
Arnold J. Mandella,b,d, Karen A. Selzb, John Avena,c, Tom Holroyda and Richard Coppolaa
a. NIMH Core MEG Facility, Building 10, NIMH, Bethesda, MD
b. Cielo Institute, 486 Sunset Dr., Asheville, NC 28804-3727
c. Fetzer-Franklin Fellow in Consciousness Studies at NIMH;
d. Corresponding Author www.cieloinstitute.org
* Supported by the Fetzer-Franklin Trust, DARPA ( Microelectronics), and the Space and
Naval Warfare Systems Center. “A plasma is lawfully and intrinsically multidisciplinary”
Consciousness :One of the Properties of the Body Temperature Brain Plasma
(1) “Particle” lengths overlap;
The plasma of consciousness
includes observable and subjective
elements
each particle effects many. Motion
is intrinsically cooperative;
(2) Important interactions in the
bulk, not like dipole magnets
(Stokes Theorem) at the surface.
(3) Elemental oscillations mucn
faster than collisions:→ EM
instantaneous forces dominate gas
and chemical kinetics.
(4) Responds strongly to
electromagnetic fields which can
generate transient structures in the
plasma.
(5)Need not have specific shape or
size
(6) Overlapping fields: chemical,
electromagnetic, psychological
(6) Composed of ionized and
neutral particles “balanced”
(8) Small space charge
(7) High density of charge carriers
(ions, electrons, neutrals,
hydrophobic charges).
(8) Spontaneous currents and
return currents., moving charges.
(9) Persistent magnetic fields
slower and cooler
We study the mean field approximation of the global magnetic field
component of the conscious brain plasma.
Traveling charges in apical dendrites of the neocortical pyramidal cell networks (MU) are associated with
extracellular return currents with varying impedance, capacitance, inductances which constitute
electromagnetic “ephaptic” fields (LFP). These, in turn, “feed back” to modulate the thresholds and
dynamics of the networks (Frolich, 2009).
apical
dedrites
Cahal
103
Electrical currents flowing within the apical dendrites of
pyramidal cells generate the surrounding magnetic field
Temporally covariant large volume
recorded from single central pair of
sensors. Ctx layers II and III
Some psychoanalytic characteristics of the
conscious brain plasma:
1.
2.
3.
4.
5.
6.
7.
All properties of the conscious brain plasma are deterministic; not random.
The plasma of consciousness is ceaselessly driven by energizing “drives”.
Psychic energy (entropy) is conserved.
The plasma of consciousness has levels of topographic scaling: unconscious,
preconscious and conscious, Ucs, Pcs, Cs.
The plasma of consciousness has finite set of thematic dynamical quasispecialized components: id, ego, superego (flavored energetics). These
components are not necessarily aware of each other.
Inhibitory “defenses” modulate access between levels of plasma
consciousness by primarily repression, more primitively denial,
dissociation, conversion, and undoing, and if persistent and stereotyped,
character formation, for example obsessive compulsive (doing and
undoing), and hysterical personality (dissociation and display).
Normal failures of defenses create leaks between levels of the plasma of
consciousness into the preconscious and conscious: dreaming, parapraxes,
“”I was thinking about one thing and I said my mother” “free
associations”, and day dreaming.
Psychological transients, eddies in the flow of consciousness in the brain’s
plasma
Task Unrelated Thoughts and Images, TUTIs.
“…thoughts, images (and sounds) that intrude into a person’s Cs unintentionally
(involuntarily)…and are unrelated to their activity, Giambra, 1995
1. Using probe windows of five to twenty-five seconds (“beep”) TUTI buttons are
pushed after daydreams one or more times in 60-70% of the windows during
repetitious tasks and/or taskless resting conditions
2. TUTIs are increased with psychologically perturbing preconditions .
3. Interrupting TUTIs makes the next one occur more quickly (“pressure”)
4. TUTI deficiency has been reported with aging, Alzheimer's, mTBI, interictal
epilepsies and schizophrenia.
5. Brain damage to some areas of the default network leads to “mental emptiness” ,
TUTI deficiency and reduction in spontaneous speech and thoughts.
6. Anterior cingulotomy (for OCD)---prominent in Columbia-Greystone Project
(1948-1956) leads to very vivid daydreams that are often confused with reality.
7. Increased demand via speed of signal processing or task difficulty decreases the
frequency and extent of TUTIs
Klinger, Antrobus, Singer, Giambra, Binder, Smallwood and others (1960-2000).
The products of ceaseless Ucs activity intrude into Pcs and Cs as
Task Unrelated Thoughts and Images, TUTIs, when internal
entropies increase (alertness, arousal, fearfulness) or when ongoing
attention demanding tasks are minimized
Implicit model of Antrobus, Singer, Giambra, Binder and others, 1960 to 1999
Psychic entropies
not conserved.
Conservation of
psychic entropies
Changes in total
brain entropies
“psychic energies”
Reciprocal
partitions of available
entropies (“psychic
energies” )
Purposeful thoughts, plans and
actions governed by The
Reality Principle, Freud, 1911
Involuntary thoughts and images, daydreaming:
↑ frequency/duration of TUTIs when quiet
governed by the Pleasure/Displeasure Principle.
Marcus Raichle’s 2001 default activation anatomy of a component of the
conscious plasma of the brain
fMRI regions light up when task-free and resting, Pleasure/Displeasure Principle;
but are dark when purposefully thinking or doing, the Reality Principle.
Averages of nine subjects: medial prefrontal, medial parietal, anterior and
posterior cingulate, habenula, etc.; generally the medial brain.
To study the magnetic fields
components of the plasma
of consciousness
Cooper electron
pairs tunnel
across JJs at
critical current;
this process is
perturbed by a
change in
magnetic flux
density.
0.00005 Tesla =
The CTF gradiometer records the magnetic fields of the
concious brain plasma
103
The ssds primary data range in
amplitude from 50-4000 fT
Mean Field Approximation of the Brain Plasma’s Magnetic Field/t
SYMMETRIC SENSOR DIFFERENCE SEQUENCES, ssds
L-R: C16, F14, T44, P57
C16
16.6 seconds of ssds
275-channel, superconducting quantum interference device
(SQUID),radial gradiometer system from VSM MedTech Ltd.,
Why ssds
1. Minimizes physical artifacts such as coughing/
blinking;
2. Reduce-covariances-including d,t,a,b,g modes; examining
“similarity regime”(Novikov, 1991)
3. Global, scalar fields makes location less relevant.
350 seconds of fMRI fluctuations
Reichle, 2009
4. Imposes a local gauge, [(0-ssds max) fT/Hz].
5. Relative motion regularizes locally (in time) the
globally nonstationary MEG signals
6. Difference metric (like velocity increment) is a common variable
in turbulence dynamics and statistics.
7. Evokes intuitions and techniques of magnetic hydrodynamic,
MHD, plasmas and fields
8. ssds minimizes central values and emphasizes outliers.
A single central ssds pair “sees” or “ is responsive with” a large volume of the
always conscious neocortical plasma
left
Central pair
difference reduces
sway
and other artifacts.
right
ssds =left-right
16.7 seconds
Lighter area
indicates neocortical
volume to
which the red C16
sensor pair’s ssds is
similarly
directionally
changing with
arbitrary threshold. ~
80%
First new observation: Log-log Power Spectra of 3 Minutes of Central C16 ssds(i)
in 10 NIMH Controls
f--α,
Mean α = -1.67± 0.43
Median = -1.59
Var = 0.18
fs = 600/150 Hz
Kolmogorov 5/3/s
To Study the Inverse of the Time-Dependent Wavelengths,
Broomhead-king Decomposition Uses the Leading Eigenvectors of a Lagged
Autocovariance Matrix, Composed with the Original Series to Generate Two or Three
Leading B/K Eigenfunctions
To Observe the Dynamics in Time of Scaling System(s) We Apply the Wavelet
Transformation in which a “Mother” Wavelet is Convolved with the Data as it is
Translated Down the
Series, “b”, at Various Dilations, “a”.
Morlet mother wavelet, w, =
w = sine wave x Gaussian
Morlet wavelet transformation of the leading B/K eigenfunctions
of 66.6’’ symmetric, ssds (top), and asymmetric, asds, sensor
difference sequences yielding intermittent, scaling strudels
C16-C16
C16-T44
C16-P57
T44 further from C16 then P57 yet more similar to C16
.
C16 left
C16 right
15 seconds;
600 Hz with
150 cut off
L – R ssds
Symmetric Sensor Difference Sequences: ssds
1.
2.
3.
4.
5.
6.
7.
Minimizes physical artifacts (e.g. coughs/blinks)
Reduces modal covariance including d, t, a, b, g
ssds as global, scalar fields reduce role of location and emphasize time.
Imposes a travelling local gauge, [0-ssds max) fT/Hz],
Locally normalizes the nonstationary signal.
Difference metric like velocity derivative in turbulence dynamics and statistics
Power spectra, Morlet wavelets of Broomhead/King leading eigenfunctions, and
multiple measures of ssds-MEG resemble closely those of ssds-local field
potentials of neocortical pyramidal cell layers II and III (monkey/Shew-Plenz).
The value off the ssds of a single pair, C16 (red), changes
in > 0.30 correlation with large regions of the neocortex.