SET THEORY
If A and B are events, S the universal set, and the null set:
P[] 0 ; P[ S ] = 1 ; P = 0 ; P Ac = 1 − p A ; P A B = P A + P B − P A B
A and B are mutually exclusive if A B = ; A and B are collectively exhaustive if A B = S
De Morgan's law: ( A B ) = AC B C
C
Conditional probability of the event A given the occurrence of the event B : P A | B =
P A B
P B
m
For any event space {B1 , B2 , . . . , Bm }, with P[ Bi ] 0 for all i : P A = P A | Bi P Bi
Bayes' theorem: P B | A =
i =1
P A | B P B
P A
Independence: Events A and B are independent if and only if: P A B = P A P B
Counting Methods
Number of k-permutations of n distinguishable objects: ( n )k =
n!
( n − k )!
n
n!
Number of ways to choose k objects out of n distinguishable objects: =
k k !( n − k )!
For n repetitions of a sub experiment with sample space S={s0,...,sm−1}, the number of length
n
n!
n=n0 +· · ·+nm−1 observation sequences with si appearing ni times is:
=
n0 , n1 ,..., nm−1 n0 !n1 !...nm−1 !
For a subexperiment with a sample space S sub = s0 , , sm −1 and P si = pi : for n = n0 + + nm −1 independent
n
n0
nm−1
trials, the probability of ni occurences of si , i = 0,1, , m − 1 is : P En0 ,...,nm−1 =
p0 pm −1
n
,...,
n
m −1
0
DISCRETE RANDOM VARIABLES (d.r.v.)
X: d.r.v.
| PX(x): probability mass function (PMF)
|
FX (x) : cumulative distribution function (CDF)
For any x, PX ( x) 0 ; xS PX ( x) = 1 ; FX ( x ) = P X x ; FX ( − ) = 0 and FX ( ) = 1
X
CDF properties: for constants a and b , and a<b
(1) P a X b = FX ( b ) − FX ( a )
(4) P a X b = P X = a + FX ( b ) − FX ( a )
(2) P X a = 1 − FX ( a )
(5) P a X b = FX ( b ) − FX ( a ) − P X = b
(3) P X b = FX b − , b − = lim →0 ( b − )
(6) P a X b = P X = a + FX ( b ) − FX ( a ) − P X = b
( )
0
Expected value (E[X]), variance (Var[X]), and standard deviation (X)
E X = X = x PX ( x)
xS X
Given a random variable X with PMF PX(x) and the derived random
variable Y=g(X), the expected value of Y is:
E Y = g ( x ) PX ( X )
xS X
Var X = E ( X − X ) E X 2 − ( E X )
2
X = Var X
2
If a and b are constants and X and Y are random variables:
E a = a
; E bX = bE X
E a + bX = a + bE X
E aX + bY = aE X + bE Y
E X Y = E X E Y , if X and Y are independent
Selected special function distributions:
Poisson
Discrete
Uniform
x e −
, x = 0,1, 2,
PX ( x) = x !
0
, otherwise
where 0
1
l − k + 1 , x = k , k + 1, k + 2, l
)
PX ( x) = (
0
, otherwise
EX =
E x =
k +l
2
Var X =
Var X =
( l − k )( l − k + 2 )
12
where k and l are intergers and k l
Pascal
x − 1 k
x −k
PX ( x) =
p (1 − p )
k − 1
with, 0 p 1 and k is an interger k 1
E x =
Binomial
n
n− x
PX ( x) = p x (1 − p )
x
with, 0 p 1 and n is an interger n 1
E x = n p
Geometric
p (1 − p ) x −1 , x = 1, 2,
PX ( x) =
, otherwise
0
with, 0 p 1
E x =
Bernoulli
1 − p , x = 0
PX ( x) = p ,
x =1
0 ,
otherwise
with, 0 p 1
E x = p
k
p
1
p
Var X =
k (1 − p )
p2
Var X = np (1 − p )
Var X =
(1 − p )
p2
Var X = p (1 − p )
CONTINUOUS RANDOM VARIABLES (c.r.v.)
X: c.r.v. : for each x ∈ SX, PX [X=x]=0
Cumulative distribution function (CDF) FX (x): for constants x1 and x2 and x1<x2:
FX ( x ) = P X x ; FX ( − ) = 0 ; FX ( + ) = 1 ; PX x1 X x2 = FX ( x2 ) − FX ( x1 )
Probability density function (PDF): f X ( X ) =
dFX ( x)
dx
x
x2
−
−
x1
f X ( x) 0, for all x ; FX ( x ) = f X ( x ) dx
f X ( x ) dx = 1 ; PX x1 X x2 = f X ( x ) dx
Expected value (E[X] or X), variance (Var[X]), and standard deviation (X) of a c.r.v. X
The expected value of a function, g(X), of X is:
E X = x f X ( x ) dx
E g ( X ) = g ( x ) f X ( x ) dx
−
−
E X − X = 0 ; E a X + b = a E X + b
Var X = E X 2 − X2 ; Var a X + b = a 2 Var X
2
2
Var X = E ( X − X ) E X 2 − ( E X )
X = Var X
If a and b are constants and X and Y are c.r.v.:
E a = a
; E bX = bE X
E aX + bY = aE X + bE Y
E X Y = E X E Y , if X and Y are independent
E a + bX = a + bE X
Selected special function distributions:
Distribution
fX (x)
FX (x)
Uniform
Random
Variable
1
,a xb
; with b a
b − a
0 , otherwise
0, x a
x−a
, a xb
b − a
1 , x b
Exponential
Random
Variable
e − x , x 0
; with 0
, otherwise
0
1 − e− x , x 0
, otherwise
0
Erlang
Random
Variable
n x n −1 e − x
,x0
( n − 1) !
0 , otherwise
0 ; n is an integer and n 1
Gaussian
(or normal)
Random
variable
−( x − )
1
2
2
e 2
2
2
k
− x
n −1 ( x ) e
1 − k =0
, x0
k!
0, otherwise
E[X]
Var(X)
a+b
2
(b − a )
1
1
2
n
n
2
2
12
Z standard normal r.v.
−u
1
( z) =
e
2 du
2 −
z
2
Gaussian distribution properties: Gaussian r.v. X to equivalent standard normal r.v. Z
x−
x−
b−
a−
z=
; FX ( x ) = ( z ) =
; ( − z ) = 1 − ( z ) ; P a x b =
−
2
MULTIPLE RANDOM VARIABLES (X AND Y)
(a) 0 FX ,Y ( x, y ) 1 ; (b) FX ,Y (, ) = 0 ; (c) FX ,Y ( x, −) = 0 ; (d) FX ,Y ( −, y ) = 0
(e) FX ( x) = FX ,Y ( x, ) ; (f) FY ( y ) = FX ,Y (, y ) ; (g) If x x1 and y y1 then FX ,Y ( x, y ) FX ,Y ( x1 , y1 )
(h) P x1 X x2 , y1 Y y2 = FXY ( x2 , y2 ) − FXY ( x2 , y1 ) − FXY ( x1 , y2 ) + FXY ( x1 , y1 )
Joint PMF : PX ,Y ( x, y ) = P X = x , Y = y
Marginal PMF : PX ( x ) = PX ,Y ( x, y ) ; PY ( y ) = PX ,Y ( x, y )
ySY
x
xS X
y
Joint CDF : FX ,Y ( x, y ) = f X ,Y (u , v) dvdu
;
Joint PDF : f X ,Y ( x, y ) =
2 FX ,Y ( x, y )
− −
Marginal PDF : f X ( x ) = f X ,Y ( x, y ) dy
x y
;
−
fY ( y ) = f X ,Y ( x, y ) dx
−
Random variables X and Y are independent if and if only:
Discrete :
PX ,Y ( x, y ) = PX ( x) PY ( y )
Continuous : f X ,Y ( x, y ) = f X ( x ) fY ( y )
•For r.v. X and Y, the expected value of W=g(X,Y) is:
Discrete : E W = g ( x, y ) PX ,Y ( x, y )
Continuous : E W = g ( x, y ) f X ,Y ( x, y ) dxdy
;
xS x yS y
− −
E X + Y = E X + E Y ; Covariance: Cov X , Y = E XY − E X E Y
CONDITIONAL PROBABILITY
Given the event B with P[ B ] 0 then the CDF of X is: FX |B ( x) = P X x | B
Conditional PMF : PX |B ( x) = P X = x | B ; Conditional PDF : f X |B ( x) =
dFX |B ( x)
dx
Conditional PMF / PDF of X given B:
PX ( x )
fX ( x)
, xB
, xB
Discrete : PX |B ( x ) = P B
; continuous : f X |B ( x ) = P B
0 , otherwise
0 , otherwise
Discrete r.v. X : For any x B , PX |B ( x ) 0 ; PX |B ( x ) = 1
xB
The conditional probability that X is in the set C is: P C | B = PX |B ( x )
xC
Continuous r.v. X : For any x B , f X |B ( x ) 0 ; f X |B ( x ) dx = 1
B
The conditional probability that X is in the set C is: P C | B = f X |B ( x ) dx
C
Conditional expected value of r.v. X given condition B:
discrete : E X | B = x PX |B ( x ) ; continuous : E X | B = x f X |B ( x ) dx
xB
−
CONDITIONAL PROBABILITY (cont.)
Conditional expected value of Y = g ( X ) given condition B :
discrete : E Y | B = E g ( X ) | B = g ( x ) PX |B ( x )
xB
continuous : E Y | B = E g ( X ) | B = g ( x ) f X |B ( x ) dx
−
Var X | B == E X | B − ( E X | B ) ; X |B = Var X | B
Conditional joint PMF of X and Y given B : PX ,Y |B ( x, y ) = P X = x, Y = y | B
2
2
PX ,Y ( x, y )
, ( x, y ) B
Conditional joint PMF of X and Y given B : PX ,Y |B ( x, y ) = P B
0 , otherwise
f X ,Y ( x, y )
, ( x, y ) B
Conditional joint PDF of X and Y :f X ,Y |B ( x, y ) = P B
0 , otherwise
Conditional expected value of W = g ( X , Y ) given B:
discrete : E W | B = g ( x, y ) PX ,Y |B ( x, y )
;
xS X ySY
continuous : E W | B = g ( x, y ) f X ,Y |B ( x, y ) dxdy
− −
Discrete r.v. variables X and Y with joint PMF PX ,Y ( x, y ), and x and y such that PX ( x) 0 and PY ( y ) 0 :
PX |Y ( x | y ) =
PX ,Y ( x, y )
PY ( y )
; PY | X ( y | x ) =
PX ,Y ( x, y )
PX ( x )
For x and y such that f X ( x) 0 and fY ( y ) 0, the conditional PDFs of Y given { X = x} and X given {Y = y} are:
fY | X ( y | x ) =
f X ,Y ( x, y )
fX ( x)
;
f X |Y ( x | y ) =
f X ,Y ( x, y )
fY ( y )
Discrete random variables X and Y with joint PMF PX ,Y ( x, y ), and x and y such that PX ( x) 0 and PY ( y ) 0:
PX ,Y ( x, y ) = PY | X ( y | x ) PX ( x ) = PX |Y ( x | y ) PY ( y )
Continuous random variables X and Y with joint PDF f X ,Y ( x, y), and x and y such that f X ( x) 0 and fY ( y ) 0:
f X ,Y ( x, y ) = fY | X ( y | x ) f X ( x ) = f X |Y ( x | y ) fY ( y )
If variables X and Y are independent:
discrete : PX |Y ( x | y ) = PX ( x ) ;
PY | X ( y | x ) = PY ( y )
continuous : f X |Y ( x | y ) = f X ( x )
fY | X ( y | x ) = fY ( y )
Conditional expected value of g ( X , Y ) given Y = y :
discrete : E g ( X , Y ) | Y = y = g ( x, y ) PX |Y ( x, y ) ;
xS X
continuous : E g ( X , Y ) | Y = y = g ( x, y ) f X |Y ( x, y ) dx
−
Mathematical identities
a 2 x 2 − 2ax + 2 ) e ax
ax − 1) e ax
(
(
e ax
ax
2
ax
; x e dx =
;
e dx = a ; x e dx = a 2
a3
d n
x n +1
1
n −1
n
x
=
n
x
;
x
dx
=
, n −1 ; dx = ln ( x )
(
)
dx
n +1
x
ax
Quadratic Equation: ax 2 + bx + c = 0
→ x=
Integration by parts : u dv = u v − v du
−b b 2 − 4ac
2a