iiiiiS
lv:':(fi;
!.
•!!'
imm-s)
^
I
ALFRED
P.
SLOAN SCHOOL OF MANAGEMENT
LIMIT LAWS FOR EXTREME ORDER STATISTICS
FROM STRONG-MIXING PROCESSES"''
by
Roy E. Welsch
Working Paper 533-71
May 1971
50
E OF TECHNOLOGY
MEMORIAL DRIVE
iMBRIDGE. MASSACHUSETTS
021'
MA.SS. INST. TECH.
MAY 24
1971
LIMIT LAWS FOR EXTREME ORDER STATISTICS
FROM STRONG-MIXING PROCESSES
by
£.Cn
1
Roy E. Welsch
Working Paper 533-71
May 1971
Science Foundation through
^This research was supported by the National
N0O14-67-A-0112-0015 at Stanford
its Graduate Fellowship program, contract
Institute of
University and DA-31-124-ARO-D-209 at the Massachusetts
Technology.
DevweV
RECEIVED
JUN 23
I
M.
I.
1971
T. Ubr<Mrtife#
2.
ABSTRACT
This paper considers the possible limit laws for a sequence of
normalized extreme order statistics (maximum, second maximum, etc.)
from a stationary strong-mixing sequence of random variables.
It
extends the work of Loynes who treated only the maximum process.
The maximum process leads to limit laws that are the same three
types that occur when the underlying process is a sequence of inde-
pendent random variables.
The results presented here show that the
possible limit laws for the k-th maximum process (k>l) from a strong-
mixing sequence form a larger class than can occur in the independent
case.
537448
3aCi-iY
OS
I
MOS Classification Numbers (1970):
Key Words:
Primary
62E20
Secondary
62G30
order statistics, mixing processes, asymptotic distributions,
reliability.
1.
Introduction
The limiting distributions of the extreme order
.
statistics from a sequence of independent, identically distributed
random variables have been exhaustively analyzed by Gnedenko [2] and
Smlrnov
[8]
Many authors have generalized these results for the
.
maximum term by relaxing the independence assumption in various ways,
e.g. Loynes [5] showed that the only possible limit laws for the
maximum term in a stationary strong-mixing seauence of random variables
are the same three types that occur in the independent case.
This paper extends the work of Loynes by considering the possible
limit laws of order statistics of fixed rank other than the maximum.
It is shown that these limit laws form a larger class than can occur
in the independent case.
These results were motivated in part by a specific model from
Consider a system of n identical components in
reliability theory.
parallel such that the lifetime of a component is dependent in a
certain way (e.g. a mixing condition) on the lifetimes of its nearest
In effect we expect that if a particular component
neighbors.
fails
(say because of excess heat) its nearest neighbors are highly likely
to be the next components
to fail.
We also assumed the system would
continue to operate if only one component failed but the sustem itself
would fail if two or more component failures occurred.
The lifetime of
the system is then represented by the (n-l)9t order statistic of the
sequence X, ,X^,...,X
1
/
n
of component lifetimes where the X,
1
are identically
5.
distributed and satisfy a specified dependence relation.
the n-th order statistic is the minimum.
In our notation
Since most of the literature
discusses maximia rather than minima we will deal with the maximum and
second maximum.
for minima.
A simple transformation converts our results to ones
6.
Notation and Preliminary Results
2.
.
If <X :n
>_
1^ is a strictly
stationary sequence of random variables with common distribution function
F(x) = P{X
<X
n
x}, the associated independent process of the process
<_
will be any sequence of mutually independent identically
n > l"^
'
—
:
distributed random variables <X
^
n
:
—>
Define the order statistics Y
X.
i
i ,n
•^
«=
Y-
{(X^
i1
(2.1)
x} = F(x) for all
by
largest among (X ,X
,
...,X
,...,X.
1
[4])
1^
^
)
n
= Y,
l,n
and
denote the o-field generated by events of the form
)7\
)
e
E}, where 1
m
m-dimensional Borel set.
(cf.
—<
It will be technically convenient to consider the joint law
.
/ ,n
Let
n
We shall limit our discussion to i=l,2 and set M
process.
n
with P{X
1^
'
denote the order statistics of the associated independent
and let Y,
S
n
Iz
a < i,
—<
—
Then \X
,
A
e
n
< b and E is an
m —
<
i„ <.
>^
1^ will be called strong-mixing
.
.<
i
if
sup{lP(AB)-P(A)P(B)|
:
>»™,
Loynes [5] referred to (2.1) as uniform
B e y\^^)
mixing.
1
a(k)
+
(k--
The following lemma is a direct consequence of the work of
Gnedenko
Lemma
[
2
]
If there exists a sequence of constants
_1.
n
n
nn—<ay+b}
n
n—<ax+6,S
so that P{M
/a
:n
y
>^
has a limiting distribution
the limiting distribution of P{M
H(x,y), with G(x),
0, b
>
a x + b
<_
}
non-degene ra te , then
G(y){l + log [G(x)/G(y)]}
y
<
x
G(x)
y
>^
X
H(x,y)
Since P{M
n
Proof.
4
of [2] that
For X
>
—<
+ b
a x
n
n
= F"(a x
}
n(l - F(a x + b
n
n
n
))
+ b
n
)
- log G(x)
-^
^-
G(x) we have by Lemma
when G(x) ^
0.
y
P{M
n
nn—<£y+b}=F(ay+b)
n
n
n
n
n—<ax+b,S
+
nF"~''^(£ y
and the result follows.
n
+ b
n
)
[F(a x + b ) - F(a y
n
n
n
+
b
n
)
]
Gnedenko also proved that G(x) has only three
possible forms (except for scale and location parameters)
X)
=
]
x)
=
X
<
X
>
0,
X
<
0, a
>
X
>_
X
<
/
r exp[-(-x)"]
J
-00
The symbol G(x)
Lemma
1
<
a >
°°.
will be used to denote one of these types.
shows that for an independent process there are only three
possible types for the joint law H(x,y).
It seems
conjecture that in view of Loynes' result for M
reasonable to
from a strong-mixing
process (only three possible limit laws) there would only be three
possible types for the joint limit law of M
simple example demonstrates.
and
S
when the under-
This is not true as the following
lying process is strong-mixing.
Let
^Z
:n
>^
1^ be a sequence of independent
identically distributed random variables with distribution function
T(.) and assume that T(') is in the domain of attraction of one of the
three limit laws in (2.2), i.e.
T"(a X + b
n
Example
1
.
n
)
there exist
constants a
n
b
,
n
such that
^ G(x).
Let X^ = max (Z^, Z
)
,
n=l,2, ...
is a stationary strong-mixing sequence, P{M
<
.
Then ^X
a x + b
}
:
->
n
>_
l')
G(x) and
n
n—<ax
P{M
Proof
b,S
n
n—<ay
n
Clearly
.
case M
+
=
:
n
(X.
X
b}-*
n
H(x,y) where
)
=
max
(Z.
1
,
.
.
.
,Z
.
.
n+1
)
(Z
P{M
n
" S
n
}
—>
P{M
y
<
X
y
>^
X
Now in this
1^ is a strong-mixing sequence.
>^
In
max
n
^
+
If M
.
n
,Z
n
,J)
n+1
= Z,
i-2,
i
= M
n
.
Therefore
(n-l)/n+l and the example follows immediately since
n
—<
a X + b } = T"'''^(a x + b ) * G(x).
n
n
n
n
This method of constructing a strong-mixing sequence is due to
Newell [6].
The limit law (2.3) is not of the same form as H(x,y) and we
conclude that by weakening the independence assumption a larger class
of limit laws is possible.
In the next section we prove a result which
limits the size of this class.
10.
3.
Possible Limit Laws
Theorem
Let <X :n
1.
>^
.
1> be a stationary strong-mixing sequence.
If there exists a sequence of constants /a
^^"n
-
^n^
^n' ^n
-
^n^
"^
0, b
>
^ n
"*"
n
'
— iN
:n >
so that
^^^ ^ limiting distribution, H(x,y),
^n^
with G(x), the limiting distribution of P{M
n
—<
a x
n
+ b
n
}
non-deaenerate,
*
then
rG(y){l -p[(log Q(x))/log
r(y)]
G(
log G(y)}
y < X
H(x,y)
\g(x)
where pCs),
£
»
1,
<_
y
l8 a concave, monotone
^
X
non-increasing function
which satisfies
p(0)(l-
s)
1
P(9)
1
1
- s.
G(-) is one of the three types
(2.2) and we interpret
(«>/°°)
= 1,
(0/0) - 1, and (0/«) - 0.
Proof.
The essential idea (cf. Loynes [5]) is t* brekk up
sets of km random variables (k fixed, m
m-q separated by blocks of length q.
>^
1)
into k blocks of length
The fact that the blocks of
length m-q are "nearly independent" because of the strong-mixing
property is then used to obtain a functional equation for H(x,y).
For X
<^
result.
y the assumptions of the theorem are the same as for Loynes 's
Therefore G(') is one of the three types (2.2)-
Given an
(k+l)(k-l)o(q)
>
e
<
and a fixed positive Integer
£/2 and define
k>_2,
choose q so that
11.
"
M
max
X
m
>
1
—<
q
l<J<n]
M
max
X., -._^.
(i-l)m+i
j^,
q+1 <J<m ^
=
.
m,i
.
k
1 <
—
-^
M*
max
=
""'
X,,
l<j<q
"n "
"'^'^
^
,v
,,
(i-l)"^J
Vl' V2'
•
'\*^q+m+-l'\4mf2'
•
S
m
second maximum.
shovn in Lemma
.,
m,i
S'
m,i
j«
i
<^
N, kP{E
S
,
n
<
,
.
2 at
'
'^2m'
n = km
.
are defined similarly for the
•^
M'
m,i
It is
}.
the end of this section tha<
Furthermore P{E
»'
for 2
>
S
Denote by E
the event {S
m,i
m,i
lim P{E^ ,} = 0.
nt+<x>
m
,
'
\m^
"•V(k-l)n^l'V(k-l)m+2
The quantities
^
•
.}
"''•
= P{E
,}
"-'•
Since q is now fixed we may choose an N so that
k.
-}
m, 1
<
f or
e/2(k+3).
The first step is to show that
(3.1)
<ax+b,S
n
nn—<ay+b}n
n
llmlP{M
^'
n—
P^{M
m—<
a y
n
+ b
n
}
n=km
m—<ax+b,S
nm—<ay+b}n
n
n
k[P{M
P{M
m—<
a y
n
+ b
n
}]
k-1
P{M —<
m
a y + b
n-'
n
}j
^
= 0,
12.
For X
y and m
>
P{M
(3.2)
N
>
n
nn—<ay+b}n
n—<ax+b,S
n-^
P{M
n—<
+
a X
n
b
nn—
S
,
a y + b } <
n
n —
<
E
P{E
^^
J —<
e/2 (k+3)
m,i
Now
n—<ax+b,S
n
n
n—<ay+b
n
P{M
- P{M
n
r
a y + b }
n
n
—<
.<ay+b;
ZP{ay+b n <Mm,i,<ax+b,S
— n
m,i — n
n
n
+
._,
n-^
M
^<ay
- n^
m,j
b,l<j<k,
- n'
+
J
»
jf^i]
J
and the strong-mixing property implies that
(3.3)
n
n—<ax+b,S
nn—<ay+b}n
P^{M
|P{M
m,l—<
n-'
'
- kP{a
n
,
a y + b
n"'
n
y+b n <Mm,l,<ax+b,S
,<ay+b}
— n
m,l — n
n
n
^*'~'^^\,1
-
V
"^
^n^l
-
(^-1>«<'5) + k(k-l)a(q)
£
e/2.
Finally we have that
(3.4)
}
-<ax+b,S
,<ay+bl
m,l—
m,l—
n
n
n
n
P{M
- P{M^ < a^x
+ b^, S^
<
a^y + b^}
<
P{E^^^}
<
c/2k(k+3)
13.
Combining (3.2), (3.3), and (3.4) gives
+ b,S <ay + b}n
nn—
'n—<ax
n
|P{M
p'^{M
n^
m—<
a y
n
+ b
n
}
m—<ay+b}]
n^
nm—<ay+b}-P{M
n
m—<ax+b,S
n
n
n
- k[P{M
.P^~-'-{M
< a
+ b )!
m - n-'y
n
<
E
'
In particular, when x = y
from which (3.1) follows.
'n—
lim|p{M
(3.5)
Set a
a X + b
<
n
mnmin
= a /a
,
We have remarked that F
mm
F (a x + b
that G
m
)
1/k
(y)
->
=
b
m
(b
(x)
G"'"'^(k).
P^{M
-
}
n
m—<
b )/a
ninni
-
-»
G(:t)
,
a x + b
n
}|
n'
=0.
and F (t) = P{M
m
m
—<
a t + b
m
m
}.
and from (3.5) it follows that
A theorem of Khintchine ([3], p. 40) states
and G(y) must be of the same type, i.e., there exist
real-valued constants a
such that G
>
and
m—<
a y + b
S,
(a,
y +
B, )
= G(y)
and
Let
Q (x,y) = kP^""^{M
m
m
m
m—<ax
m
}P{M
+
b,S
mm—<aym
+
b},
m
Q(x,y) = kG''"-'-(y)H(x,y),
and assume that (x,y) and
for H.
(ax
+
6,
,
a.y +
Equation (3.1) is equivalent to
6,
)
are points of continuity
14.
lim Q (a X + b
m
m m
,
a y + b
m
m
)
= H(x,y) + (k-l)G(y).
But a standard argument ([3], p. 41) shows that
lim Q (a X + b , a y + b ) = Q(o, x +
^
m'
m-'
m
^m m
k
6,
k
,
a,
y +
6, )
k"^
k
G(aj^y
+ H^)]
and therefore
H(x,y) = G(y) + k[H(a^x + 6^,
(3.6)
.
when X
^
+ &^) -
a^^y
G(^-^>/Ny)
y.
It is apparent from (2.1)
increasing for
t
such that
<
that G(t) is continuous and strictly
G(t)
G(y)
(3.8)
H(x2,y) - H(x^,y)
<.
y
<
G(X2) -
From (3.7) we have that H(x,y) =
and H(x,y) =
Furthermore
1.
H(x,y) and
(3.7)
if G(y) = 0.
<
1
G(xp
<_
x
Yl^il^Z'
if G(y) = 1 and from (3.6) H(x,y) =
Finally (3.8) implies that H(x,y) = lim H(t,y) if G(x) =
if G(x) = 0.
Therefore if x
>^
y it is possible to
express H(x,y) in the form
H(x,y) - R(-log G(x), - log G(y))
Moreover H(x,y) has the same form if x
<
=
y,
R(u,v)
so that we may take
1,
15.
1
With
u < V
<
(3.9)
f
<
<»
equation (3.6) takes the simple form
k
(u.v) = kf (u/k, v/k)
^
1
function so that
H(x,y) = H(x',y')
lim
x+x' ,y+y'
or 1 we have
and since G(') is monotone increasing except when G(') =
R(u/r, v/r) = R(u/r', v/r')
lim
(3.10)
<
u
<
v
r4-r'>0
Using (3.9) it is easy to show that f(u,v) = rf(u/r, v/r) for all
rational r
>
0.
Then- (3.10) implies that f(u,v) = zf(u/z, v/z) for
all real
>
0.
Now let
z
z =
f(u,v) - vf(u/v, 1)
v
^
so that
>
vp(u/v)
and from (3.8) we obtain
It is clear that p(l) *
-s +1
-s +1
(3.12)
<
p(s^) - p(8,)
<
e
^
" e
"
16,
where
s.
- -log G(x^), s„ - -log G(x
)
and log G(y) - -1.
Therefore
p(«) is continuous and monotone decreasing on [0,1] and p(')
In order to show that
p(0
>_
0.
is concave we will use the following
lemma whose proof may be found in [7].
Lemma
.
Let r, s, Ar, As be any such numbers as
and As - es (e
,
.
^
-^
is
>
0)
ii)(s+As)
As
,
or r = s =
- 4»(s)
^
-
and
i|/(r+Ar)
<
Ar
<
<
As.
r < s,
Ar = er
Then
- 4'(r)
Ar
necessary and sufficient for the continuous bounded function
i|/(')
to be concave in [O,").
17.
For convenience in applying this letnma we extend the domain of
=0
definition of p(') by letting p(s)
for 1
_^
s
<
=».
<
G(x^)
Since H(x,y) is a distribution function
with X2
^
x^, y^
2.
When x^
y^-
>
x^, y^
G(y 2) rp(s^) - p(s2)"|
?[
^1-^2
i. Vj^.
<
1,
P(r^) - p(r2)
^1-
J
^2
with r^ = (log G(x^))/log G(y^) and s^ = (log G(x^))/log G(y2),
i
- 1.2.
If we set
= (s -s )/s
e
=
^'^l~^2^''^2
^^^^ ^2
^
^'
^^^^ (3.15)
becomes
pS2+As2)
G(y 2)
(3.16)
G(y
- p(s2)1
?l
J-
where As. = es^ and Ar^ = er„.
arbitrarily close to
1,
^
p(r2+Ar2) - p(r2)
'~2
"^2
Since (3.15) must hold for G(y2)/G(yp
(3.13) is satisfied.
When ^2 = 52== 0,(3.15)
again implies that (3.13) holds and therefore p(.) is concave in [0,1],
Finally from (3.12)
-S+1
-^
S-1—>^ 8-1^
/
N
,
—<
s
<
1
18.
so that 11m inf
s+1
that p(s)
_<
[p
(s)/(s-l)
1 - s.
>^
]
-1.
Since pCs) in concave we conclude
Substituting for (u,v) in (3.11) completes the
proof.
Berman
and Sibuya [7] have used similar arguments to obtain
[1]
limiting forms for bivariate extreme value distributions.
The
techniques used above generalize to the third maximum etc. but with
increasing complexity.
Lemma
2
.
If
then lim P{E
Proof.
^X
:n
is a strictly stationary ergodic sequence
1^
>_
,} = 0.
m, 1
For ease of notation let E
show that P{
^
= E
m
and M' = M , and assume
m
m,l
,
m,l
If <A represents the rational numbers r
E
}
=»
0.
r}
<
1
then
m=q+l
such that P{X
<^
PCOE m } —<
m
?{({) E
Z
-D
m
reCA
(M'
m)
—<
m
r)}
and
P{(nE J n
I
i=q+l
P{X
(M*
<
r)}
,,
-<
r,
q+1
+ P{X
1
'
r;
X
q+2
-<
r,...,X,
j
q
+
_^„
>.
'
'
2}.
i
-<
r,
'
X,_^,
i+2
19.
By the strong law of large numbers for strictly stationary ergodic
sequences
lim
n^
almost surely.
for
i
^
(
1,
^^,)/(n-l)
^^j-''^
Z
j=i+l
= P{X,
This implies that P^X
and ?{() E
}
=
^
<_
£
r}
r,
follows immediately.
X
<
1
_<
r,...} =
20.
Conclusions
A.
Theorem
.
1
clearly includes Lemma 1 and hence
for an independent process p(s) =
Let^Z
n
,
1^ be as in Example
>^
X
iM.n
(n-l)k+l'
max (Z,
=
n
where k and
£
1
1
- s.
In Example l>p(s)
=
0.
and set
Z.
,.,,_,...,Z- ^^,.„) n=l,2,...
'
(n-l)k+2'
(n-l)k+£
are fixed positive integers.
The sequence
^X
n
:
>^
1^
is strong-mixing and it is possible to show that there exist constants
a
n
and b
n
so that P{M
n
<ay+b}
—<ax+b,S
n
n
n — n-'
n
converges
and H(x,y)
•j'-'
n
*
is of the form given in Theorem 1 with p(s) = c(l - s) where c is a
rational number,
<^
c
£
which is a function of k and
1,
I.
The proof
is not difficult but rather tedious and the details will be omitted.
Thus far we have only succeeded in constructing examples where p(-)
is
linear.
The problem of finding a strong-mixing sequence leading
to a strictly concave
p(0
or sharpening Theorem
1
to exclude this
case is still open.
In the reliability model mentioned earlier, we note that Theorem
implies that P{S
<
—
n
a x
n
+
b
n
}
- G(x)[l
- p(0)
log G(x)] and therefore
the strong-mixing assumption can have a considerable effect on the
asymptotic distribution of the second maximum (p(0) =
independent case)
being explored.
.
1
in the
The consequences of this result are currently
1
Acknowledgiment
.
The author wishes to express his appreciation
to Professor Samuel Karlin of Stanford University for his guidance
and encouragement.
in [9].
A weaker version of Theorem
1
originally appeared
22.
REFERENCES
[1]
Berman, S.M. (1961).
Convergence to bivarlate extreme value
distributions, Annals of the Institute of Statistical
Mathematics 13 217-223.
.
[2]
Gnedenko, B.V. (1943).
Sur la distribution du terme maximum
d'une serie aleatoire, Ann. Math 44 423-453.
[3]
Gnedenko, B.V. and A.N. Kolmogorov (1968).
Limit
Distributions
for Sums of Independent Random Variables Addison-Wesley
Reading. Mass.
.
,
[4]
Ibragimov, I. A. (1962).
Some limit theorems for stationary
processes, Theor. Probability Appl 7 349-382.
.
[5]
Loynes, R.M. (1965).
Extreme values in uniformly mixing
stationary stochastic processes, Ann. Math. Statist
36 993-999.
.
/*»
—————^——
[6]
Newell, G.F. (1964). Asymptotic extremes for m-dependent random
variables, Ann. Math. Statist 35 1322-1325.
[7]
Sibuya, M. (1960).
Bivariate extreme statistics, Annals of the
Institute of Statistical Mathematic s. 11 195-210.
^^
[8]
Smirnov, N.V. (1952).
Limit distributions for the terms of a
variational series, Amer. Math. Soc. Transl No. 67.
[9]
Welsch, R.E. (19 69). Weak convergence of extreme order
statistics from (ji-mixing orocesses. Ph.D. thesis.
Department of Mathematics, Stanford University.
.
K^
.