Sample Paths, Convergence,
and Averages
Convergence
• Definition: {an : n = 1, 2, …} converges to b as
n  ∞ means that
 > 0, n0(), such that n > n0(), |an – b| < 
• Example: an = 1/n converges to 0 as n  ∞
since  > 0, choosing n0() = 1/ ensures
that n > n0(), an = 1/n < 1/n0() < 
2
Convergence Almost Surely
(with Probability 1)
• {Yn : n = 1, 2, …} converges almost surely to μ as n  ∞, if
ε > 0, P{limn∞ |Yn – μ| > ε} = 0
• In other words, the probability mass of the set of sample paths that
misbehave (Yn deviates from μ by more than ε) is 0 if we take n to be
large enough
– Note: This does not mean that no sample path misbehaves, and only that
the total probability mass of these “bad” paths is (of measure) zero
– A sample path misbehaves if no matter what value of n we choose, it is
still possible to have |Yn – μ|
Consider the average of n coin flips. Most sample paths will converge to
a value of ½. However, some sample paths wont, e.g., the sample path
1,1,1,1,1,1,… We have convergence almost surely, because even though
there are an uncountable number of “bad” paths, that number becomes
vanishingly small (of measure 0) as n goes to infinity
3
Convergence in Probability
• Yn : n = 1, 2, …} converges in probability to μ as n  ∞, if
 ε > 0, limn∞P{|Yn – μ| > ε} = 0
(the limit applies to the probability, and not the random
variables)
• In other words, the odds that an individual sample path
behaves badly for Yn go to 0 as n  ∞
• However, this does not preclude the possibility that all (or a
non-negligible number) sample paths occasionally behave
badly as n  ∞
• Convergence almost surely implies convergence in
probability, but the converse is not true
– Example: See review problem S4.4
4
Strong & Weak Laws of Large Numbers
• Weak Law: Let X1, X2, X3,… be i.i.d. random variables
with mean E[X], Sn = Σ{i=1,…,n}Xi, and Yn = Sn/n
Then Yn converges in probability to E[X]
• Strong Law: Let X1, X2, X3,… be i.i.d. random variables
with mean E[X], Sn = Σ{i=1,…,n}Xi, and Yn = Sn/n
Then Yn converges almost surely to E[X]
• Implications: Almost every sample paths of successive
trials of Xn converges to a mean value of E[X]
– There can still be bad sample paths, but the odds of picking
one tend to zero as n  ∞
5
Time & Ensemble Averages
• Basic concept: Run a single experiment for 10 hours
versus run 100 experiments of 6 minutes each
– Record average value of experiment variable over the 10
hours of the first experiment and compare it to the average
of the experiment variable at the end of each of the 100 6
t
minutes experiments
N v dv

time avg
0
 lim
• Time average N
t 
t
• Ensemble average N
ensemble
 lim E N t   i 0 ipi

t 
where pi = limt∞P{N(t) = i}
6
Ergodicity
• An ergodic system is positive recurrent, aperiodic, and
irreducible
– Irreducible: We can get from any state to any other state
– Positive recurrent: Every system state is visited infinitely
often, and the mean time between successive visits is finite
(and the visit of each state is a renewal point – the system
probabilistically restart itself)
– Aperiodic: The system state is not deterministically coupled
to a particular time period
• For an ergodic system, the ensemble average exists and
with probability 1 is equal to the ensemble average
7
Non-Ergodic Systems
• Systems where the state evolution depends on some
initial conditions
– Flip a coin and with probability p the system will receive 1
job/sec, and with probability 1 – p it will receive 2 jobs/sec
– System state at time t will be very different depending on
the starting condition
• Periodic systems
– At the start of every 5mins interval, the system receives 2
jobs each taking 1min to process
– The ensemble average does not exist in this case (different
results depending on when the systems are sampled)
8
System Averages
• Average number of jobs in system: N
– Time average: In every time interval during which
the number of jobs is constant, record interval
duration, ti, and number of jobs, ni
• Time average of number of jobs is Ntime = (ΣiNiti)/(Σiti)
– Ensemble average: Run M experiments of duration t,
where Mt = Σiti, and record nk(t), k = 1, 2, …, M
• Ensemble average of number of jobs is Nensemble = Σknk(t)/M
• Average job time in system
9