First-Passage Statistics of Extreme Values
Eli Ben-Naim
Los Alamos National Laboratory
with:
Paul Krapivsky (Boston University)
Nathan Lemons (Los Alamos)
Pearson Miller (Yale, MIT)
Talk, publications available @ http://cnls.lanl.gov/~ebn
Random Graph Processes, Austin TX, May 10, 2016
Plan
I. Motivation: records & their first-passage statistics as a
data analysis tool
II. Incremental records: uncorrelated random variables III. Ordered records: uncorrelated random variables
IV. Ordered records: correlated random variables
I. Motivation:
records & their first-passage
statistics as a data analysis tool
Record & running record
• Record = largest variable in a series
!
XN = max(x1 , x2 , . . . , xN )
• Running record = largest variable to date
!
X1  X2  · · ·  XN
• Independent andZ identically distributed variables
1
dx (x) = 1
0
Statistics of extreme values
Feller 68
Gumble 04
Ellis 05
Average number of running records
• Probability that Nth variable sets a record
!
PN
1
=
N
= harmonic number
• Average number of records
1 1
1
!
MN = 1 +
2
+
3
+ ··· +
N
• Grows logarithmically with number of variables
MN ' ln N +
= 0.577215
Behavior is independent of distribution function
Number of records is quite small
Time series of massive earthquakes
10
M=8.5
9.5
9
8.5
1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010
year
10
M=8.0
9.5
9
8.5
8
1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010
year
10
9.5
M=7.5
9
8.5
8
7.5
1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010
1770 M>7 events
1900-2013
Magnitude
Annual #
9-9.9
8-8.9
7-7.9
6-6.9
1/20
1
15
134
5-5.9
1300
4-4.9
~13,000
3-3.9 ~130,000
2-2.9 ~1,300,000
year
Are massive earthquakes correlated?
Records in inter-event time statistics
t1
t2
t3
t4
Count number of running records in N consecutive events
9
8
7
6
5
MN
4
3
2
1
0 0
10
M>5 (37,000 events)
M>7 (1,770 events)
1+1/2+1/3+...+1/N
10
attribute deviation to
aftershocks
2
1
N
10
records indicate inter-event times uncorrelated
massive earthquakes are random EB & Krapivsky PRE 13
First-passage processes
•
Process by which a fluctuating quantity reaches a threshold for
the first time
•
First-passage probability: for the random variable to reach the
threshold as a function of time.
•
Total probability: that threshold is ever reached. May or may
not equal 1
•
First-passage time: the mean duration of the first-passage
process. Can be finite or infinite
S. Redner, A Guide to First-Passage Processes, 2001
II. Incremental records:
uncorrelated random variables
Marathon world record
Year
Athlete
Country
Record
Improvement
2002
Khalid Khannuchi
USA
2:05:38
2003
Paul Tergat
Kenya
2:04:55
0:43
2007
Haile Gebrsellasie Ethiopia
2:04:26
0:29
2008
Haile Gebrsellasie Ethiopia
2:03:59
0:27
2011
Patrick Mackau
Kenya
2:03:38
0:21
2013
Wilson Kipsang
Kenya
2:03:23
0:15
Incremental sequence of records
every record improves upon previous record
by yet smaller amount
Are incremental sequences of records common?
source: wikipedia
Incremental records
y4
y3
y2
y1
Incremental sequence of records
every record improves upon previous record
by yet smaller amount
random variable = {0.4, 0.4, 0.6, 0.7, 0.5, 0.1}
latest record = {0.4, 0.4, 0.6, 0.7, 0.7, 0.7}
latest increment = {0.4, 0.4, 0.2, 0.1, 0.1, 0.1} ⇥
What is the probability all records are incremental?
Probability all records are incremental
0
10
10
simulation
-0.317621
N
0
-0.3
N
Earthquake data
-1
10
S
S
-2
10
10
-1
10
0
10
1
N
10
2
10
SN ⇠ N
3
-3
10
0
10
1
2
10 10
3
10
4
10
N
5
10
6
10
7
10
8
10
= 0.31762101
Power law decay with nontrivial exponent
Problem is parameter-free Miller & EB JSTAT 13
Uniform distribution
/ 1/N
1/(N + 1)
• The variable x is randomly distributed in [0:1]
(x) = 1
!
for
0x1
• Probability record is smaller than x
•
RN (x) = x
Average record
AN
N
=
N +1
=)
N
1
AN ' N
1
Distribution of records is purely exponential
Distribution of records
• Probability a sequence is incremental and record < x
GN (x)
!
=)
• One variable
G (x) = x
• Two variablesG
(x) = 3 x
!
x2 = 2x1
=)
1
!
x2
x 1 > x1
2
4
x2 = x1
SN = GN (1)
2
S1 = 1
x/2
x
3
S2 =
4
=)
x
• In general, conditions are scale invariant x ! a x
• Distribution of records for incremental sequences
!
•
GN (x) = SN x
N
S1 = 1
S2 = 3/4
S3 = 47/72
N
x
Distribution of records for all sequences equals
Statistics of records are standard
Fisher-Tippett 28
Gumbel 35
Scaling behavior
1
N=1
N=2
N=3
N=4
x
2
x
3
x
4
x
GN(x)/GN(1)
0.8
0.6
0.4
0.2
0
0
• Distribution of
!
0.2
0.4
1
records for incremental sequences
GN (x)/SN = x
• Scaling variable
x
0.8
0.6
N
= [1
s = (1
(1
x)]
N
!e
x)N
Exponential scaling function
s
Distribution of increment+records
• Probability density S (x,y)dxdy that:
N
1. Sequence is incremental
2. Current record is in range (x,x+dx)
3. Latest increment is in range (y,y+dy) with 0<y<x
is
incremental
• Gives the probabilityZ a sequence
Z
1
SN =
!
x
dx
dy SN (x, y)
memory
• Recursion equation incorporates
Z
0
0
x y
!
SN +1 (x, y) = x SN (x, y) +
!
old record holds
0
dy SN (x
y
0
y, y )
a new record is set
has memory
• Evolution equation includes integral,
Z
SN (x, y)
=
N
x y
(1
x)SN (x, y) +
0
dy SN (x
y
0
y, y )
Scaling transformation
• Assume record and increment scale similarly
•
Introduce a scaling variable for the increment
y⇠1
s = (1
!
1
x⇠N
x)N
and
z = yN
• Seek a scaling solution
SN (x, y) = N 2 SN
!
(s, z)
time out of the master equation
• Eliminate
✓
◆
Z
2
⇥
⇥
+s+s
+z
⇥s
⇥z
1
(s, z) =
dz 0 (s + z, z 0 )
z
Reduce problem from three variables to two
Factorizing solution
• Assume record and increment decouple
(s, z) = e f (z)
Substitute into equation for similarity solution
s
•
!
✓
2
⇥
⇥
+s+s
+z
⇥s
⇥z
◆
Z
(s, z) =
1
dz 0 (s + z, z 0 )
z
• First order integro-differential equation
Z
!
•
•
0
zf (z) + (2
)f (z) = e
1
z
0
f (z )dz
0
z
Z
Cumulative distribution of scaled increment
g(z) = f (z )dz
Convert into a second order differential equation
00
zg (z) + (2
0
)g (z) + e
z
g(z) = 0
1
0
0
z
g(0) = 1
g 0 (0) =
1/(2
Reduce problem from two variable to one
)
Distribution of increment
• Assume record and increment decouple
•
zg (z) + (2
)g (z) + e
Two independent solutions
00
!g(z)
=z
⌫ 1
0
and
z
g(z) = 0
g(z) = const.
as
g(0) = 1
g 0 (0) =
1/(2
z!1
• The exponent is determined by the tail behavior !
= 0.317621014462...
• The distribution of increment has a broad tail
PN (y) ⇠ N
1
y
2
Increments can be relatively large
problem reduced to second order ODE
)
Numerical confirmation
Monte Carlo simulation versus integration of ODE
1
g(0) = 1
0.8
g 0 (0) =
1/(2
2
4
theory
simulation
)
0.6
g
0.4
0.2
0
0
z
6
hszi
!1
hsihzi
8
10
Increment and record become uncorrelated
Recap II
• Probability a sequence of records is incremental
• Linear evolution equations (but with memory)
• Dynamic formulation: treat sequence length as time
• Similarity solutions for distribution of records
• Probability a sequence of records is incremental
decays as power-law with sequence length
• Power-law exponent is nontrivial, obtained analytically
• Distribution of record increments is broad
First-passage properties of extreme values are interesting
III. Ordered records:
uncorrelated random variables
Ordered records
• Motivation: temperature records:
Record high increasing each year
xn
yn
Xn
Yn
• Two sequences of random variables
{X1 , X2 , . . . , XN }
and
xn =! max{X1 , X2 , . . . , Xn }
and
!
n
{Y1 , Y2 , . . . , YN }
• Independent and identically distributed variables
• Two corresponding sequences of records
• Probability S
N
x 1 > y1
yn = max{Y1 , Y2 , . . . , Yn }
records maintain perfect order
and
x 2 > y2
···
and
x N > yN
Two sequences
• Survival probability obeys✓ closed ◆recursion equation
!
SN = SN
1
1
• Solution is immediate
✓2N ◆
!
SN =
1
2N
N
2
2N
identical to Sparre Andersen 53!
• Large-N: Power-law decay with rational exponent
SN ' ⇡
1/2
N
1/2
Universal behavior: independent of parent distribution!
Ordered random variables
• Probability P
N
! X1
> Y1
and
variables are always ordered
X 2 > Y2
• Exponential decay
!
···
PN = 2
and
X N > YN
N
• Ordered records far more likely than ordered variables!
• Variables are uncorrelated
• Records are strongly correlated: each record
“remembers” entire preceding sequence
Ordered records better suited for data analysis
Three sequences
0
10-1
1.32
10-2
1.30
10-3
σ
1.28
10-4
10-5
1.26 1
2
3
4
5
10 10
10 10
10
SN 10-6
N
10-7
10-8
10-9
simulation
10-10
theory
10 -11
10 0
7
1
2
3
4
8
5
6
10 10 10 10 10 10 10 10 10
xn
yn
zn
Xn
Yn
Zn
n
• Third sequence of random variables
!
xn > yn > zn
N
n = 1, 2, . . . , N
• Probability S records maintain perfect order
• Power-law decay with nontrivial exponent?
N
SN ⇠ N
with
= 1.3029
leader
median
laggard
Rank of median record
| {z }
j
• Closed equations for survival probability not feasible
• Focus on rank of the median record
• Rank of the trailing record irrelevant
• Joint probability P that (i) records are ordered and
N,j
(ii) rank of the median record equals j survival probability
• Joint probability gives the
X
N
SN =
PN,j
j=1
Closed recursion equations
| {z }
j
•Closed recursion equations for joint probability feasible
3N + 2 j 3N + 1 j 3N j
PN,j
3N + 3
3N + 2 3N + 1
3N + 2 j 3N + 1 j
j
+
PN,j
3N + 3
3N + 2 3N + 1
N
+1
X
3N + 2 j
+
(3N
(3N + 3)(3N + 2)(3N + 1)
!
!
1
k)PN,k
k=j
!
N
+1
X
3N + 2 j
+
k PN,k
(3N + 3)(3N + 2)(3N + 1)
k=j
1
•The survival probability for small N
N
1
2
3
4
5
SN
1
6
29
360
4 597
90 720
5 393
149 688
179 828 183
6 538 371 840
no new records
O(N 0 )
PN +1,j =
O(N
1
) new leading records
O(N
1
) new median records
O(N
2
)
(3N )! SN
1
58
18 388
17 257 600
35 965 636 600
two new records
Key observation
0.5
2
N=10
3
N=10
4
N=10
0.4
Pj,N
SN
0.3
0.2
0.1
0
0
1
2
3
4
5
j
6
7
8
9
10
Rank of median record j and N become uncorrelated!
Asymptotic analysis
• Rank of median record j and N become uncorrelated! !
PN,j ' SN pj
N !1
as
• Assume power law decay for the survival probability
!
SN ⇠ N
• The asymptotic rankXdistribution is normalized 1
!
pj = 1
recursion
• Rank distribution obeys a much simpler
X
j=1
pj = (j + 1) pj
Scaling exponent
j
pj
3
1
1
3
1
k=j
pk
is an eigenvalue
The rank distribution
EB & Krapivsky PRE 15
• First-order differential✓ equation◆for generating function dP (z)
z)
+ P (z)
dz
(3
!
• Solution
r
!
P (z) =
1
3
z
z
✓
z
3
z
1
1
◆ Z
z
z
0
3
z
=
z
1
P (z) =
z
X
pj z j+1
j 1
du
(3 u)
u 1
(1 u)3/2
1/2
• Behavior near z=3 gives tail of the distribution
pj ⇠ j
!
1/2
3
j
• Behavior near z=1 gives the scaling exponent
2 F1
1 1
2, 2
;
3
2
;
1
2
=0
=)
= 1.302931 . . .
Three sequences: scaling exponent is nontrivial
Multiple sequences
• Probability S that m records maintain perfect order
• Expect power-law decay with m-dependent exponent
N
6
SN ⇠ N
!
5
m
4
bounds
• Lower and upper
1 1
1
0 ! m
m
1+
2
+
3
+ ··· +
m
σ3
2
1
0
1
2
3
4
5
6
7
grows linearly with number of sequences
• Exponent
(up to a possible logarithmic correction)
m
'm
m
In general, scaling exponent is nontrivial
8
Family of ordering exponents
•One sequence always in the lead: 1>rest
1
AN ⇠ N
!
↵m
↵m = 1
m
•Two sequences always in the✓ lead: 1>2>rest
BN ⇠ N
!
m
2 F1
1
m
m
,
1 m
2
1
2m 3
;
m 1
;
1
m
1
◆
=0
•Three sequences always in the lead: 1>2>3>rest
CN ⇠ N
5
m
δ
γ
β
α
σ
4
3
m
1
2
3
4
5
6
2
1
0
0
1
2
3
m
4
5
6
↵
0
1/2
2/3
3/4
4/5
5/6
1/2
1.302931
1.56479
1.69144
1.76164
1.302931
2.255
2.547
2.680
2.255
3.24
3.53
Recap III
•
•
•
•
•
•
Probability multiple sequences of records are ordered
Uncorrelated random variables
Survival probability independent of parent distribution
Power-law decay with nontrivial exponent
Exact solution for three sequences
Scaling exponent grows linearly with number of sequences
•
Key to solution: statistics of median record becomes
independent of the sequence length (large N limit)
•
Scaling methods allow us to tackle combinatorics
IV. Ordered records:
correlated random variables
Brownian positions
x2
t
Brownian records
x1 x2
x1
t
m2
m1
First-passage kinetics: brownian positions
Probability two Brownian particle do not meet
x2
x1
•
Sparre
Universal probability
Andersen
53
!
t
•
✓ ◆
2t
St =
2
t
t
Asymptotic behavior
S⇠t
Feller 68
1/2
Behavior holds for Levy flights, different mobilities, etc
Universal first-passage exponent
S. Redner, A guide to First-Passage Processes 2001
First-passage kinetics: brownian records
Probability running records remain ordered
x1 x2
m2
m1
10
0
simulation
-1/4
t
P
t
10
10
-1
-2
10
S⇠t
0
10
1
10
2
10
3
10
t
4
10
5
= 0.2503 ± 0.0005
Is ¼ exact? Is exponent universal?
10
6
10
7
10
8
t
m1 > m2
if and only if
x1 x2
m2
m1 > x 2
m1
From four variables to three
• Four variables: two positions, two records
!
m1 > x 1
and
m2 > x2
• The two records must always be ordered
m1 > m2
!
• Key observation: trailing record is irrelevant!
!
m1 > m2
if and only if
m1 > x 2
• Three variables: two positions, one record
m1 > x 1
and
m1 > x2
From three variables to two
• Introduce two distances from the record
!
u = m1
x1
and
v = m1
x2
• Both distances undergo Brownian motion !
⇥ (u, v, t)
2
= Dr (u, v, t)
⇥t
(ii) advection
• Boundary conditions:✓(i) absorption
◆
!
⇥
⇥u
=
0
and
v=0
⇥
⇥v
u=0
remain
ordered
• Probability records
Z Z
1
P (t) =
0
1
0
=0
du dv (u, v, t)
Diffusion in corner geometry
v
u
“Backward” evolution
• Study evolution as function of initial conditions
!
P ⌘ P (u0 , v0 , t)
• Obeys diffusion equation P (u0 , v0 , t)
2
= Dr P (u0 , v0 , t)
t
!
(ii) advection
• Boundary conditions:✓(i) absorption
◆
!P
v0
=
0
=0
and
P
P
+
u0
v0
=0
u0 =0
• Advection boundary condition is conjugate!
Solution
• Use polar coordinates
q
r=
!
u20
+
v02
• Laplace operator
⇥
and
v0
= arctan
u0
2
1
⇥
1
⇥
r2 = 2 +
+ 2 2
⇥r
r ⇥r r ⇥
!
2
(ii) advection
• Boundary conditions:✓(i) absorption
◆
!
P
=0
= 0 and
⇥P
r
⇥r
⇥P
⇥
=⇥/2
=0
law + separable form
• Dimensional analysis +✓power
◆
P (r, , t) ⇠
r2
Dt
f( )
Selection of exponent
•
Exponent related to eigenvalue of angular part of Laplacian
!
00
2
f (⇥) + (2 ) f (⇥) = 0
• Absorbing boundary condition selects solution
!
f (⇥) = sin (2 ⇥)
• Advection boundary condition selects exponent
!
tan ( ⇡) = 1
• First-passage probability
P ⇠t
1/4
EB & Krapivsky PRL 14
General diffusivities
• Particles have diffusion constants D
!
!
(x1 , x2 ) ! (b
x1 , x
b2 )
with
1
and D2
(b
x1 , x
b2 ) =
✓
ben Avraham Leyvraz 88
x1
x2
p ,p
D1
D2
• Condition on recordsr involves ratio of mobilities !
D1
m
b1 > m
b2
D2
!
D1
tan ( ⇥) = 1
D2
!
to repeat • Analysis straightforward
r
◆
mobility-dependent
• First-passage exponent:1 nonuniversal,
r
D
=
⇥
arctan
2
D1
Numerical verification
0.5
simulation
theory
0.4
0.3
β
0.2
0.1
0
0
0.5
1
1.5
2
D1/D2
perfect agreement
2.5
3
Properties
•
Depends on ratio of diffusion constants
!
•
(D1 , D2 ) ⌘
✓
D1
D2
◆
Bounds: involve one immobile particle
!
1
(0) =
2
(1) = 0
• Rational for special values of diffusion constants
!
(1/3) = 1/3
(1) = 1/4
(3) = 1/6
chasing slow” and “slow chasing fast”
• Duality: between “fast
✓
◆
✓
◆
D1
D2
+
D2
D1
1
=
2
Alternating kinetics: slow-fast-slow-fast
Multiple particles
!
•
Probability n Brownian positions are perfectly ordered !
•
↵n =
Records perfectly ordered
!
•
Pn ⇠ t
↵n
m1 > m2 > m3 > · · · > mn
In general, power-law decay
Sn ⇠ t
⌫n
n(n
1)
Fisher & Huse 88
4
n
2
3
4
5
6
⌫n
1/4
0.653
1.13
1.60
2.01
n /2
1/4
0.651465
1.128
1.62
2.10
Uncorrelated variables provide an excellent approximation
Suggests some record statistics can be robust
Recap IV
•
•
First-passage kinetics of extremes in Brownian motion
•
•
•
•
•
First-passage exponent obtained analytically
Problem reduces to diffusion in a two-dimensional
corner with mixed boundary conditions
Exponent is continuously varying function of mobilities
Relaxation is generally slower compared with positions
Open questions: multiple particles, higher dimensions
Why do uncorrelated variables represent an excellent
approximation?
First-passage statistics of
extreme values
•
•
•
•
Survival probabilities decay as power law
•
•
Many, many open questions
First-passage exponents are nontrivial
Theoretical approach: differs from question to question
Concepts of nonequilibrium statistical mechanics are
powerful: scaling, correlations, large system-size limit
Ordered records as a data analysis tool
Publications
1. Scaling Exponents for Ordered Maxima,
E. Ben-Naim and P.L. Krapivsky,
Phys. Rev. E 92, 062139 (2015)
2. Slow Kinetics of Brownian Maxima,
E. Ben-Naim and P.L. Krapivsky,
Phys. Rev. Lett. 113, 030604 (2014)
3. Persistence of Random Walk Records,
E. Ben-Naim and P.L. Krapivsky,
J. Phys. A 47, 255002 (2014)
4. Scaling Exponent for Incremental Records,
P.W. Miller and E. Ben-Naim,
J. Stat. Mech. P10025 (2013)
5. Statistics of Superior Records,
E. Ben-Naim and P.L. Krapivsky,
Phys. Rev. E 88, 022145 (2013)