12.215 Modern Navigation
Thomas Herring
Review of last class
• Map Projections:
– Why projections are needed
– Types of map projections
• Classification by type of projection
• Classification by characteristics of projection
– Mathematics of map projections
10/30/2006
12.215 Modern Naviation L12
2
Today’s Class
• Basic Statistics
– Statistical description and parameters
•
•
•
•
Probability distributions
Descriptions: expectations, variances, moments
Covariances
Estimates of statistical parameters
• Propagation of variances
– Methods for determining the statistical parameters
of quantities derived other statistical variables
10/30/2006
12.215 Modern Naviation L12
3
Basic Statistics
• Concept behind statistical description: Some processes involve
so many small physical effects that deterministic model is not
possible.
• For Example: Given a die, and knowing its original orientation
before throwing; the lateral and rotational forces during throwing
and the characteristics of the surface it falls on, it should be
possible to calculate its orientation at the end of the throw. In
practice any small deviations in forces and interactions makes
the result unpredictable and so we find that each face comes up
1/6th of the time. A probabilistic description.
• In this case, any small imperfections in the die can make one
face come up more often so that each probability is no longer
1/6th (but all outcomes must still add to a probability of 1)
10/30/2006
12.215 Modern Naviation L12
4
Probability descriptions
• For discrete processes we can assign probabilities of specific
events occurring but for continuous random variables, this is not
possible
• For continuous random variables, the most common description
is a probability density function.
• A probability density function gives the probability that a random
variable x will have a value between x and x+dx
• To find the probability of a random variable taking on a value
between x1 and x2, the density function is integrated between
these two values.
• Probability density functions can derived analytically for variables
that are derived from other random variables with know
probability density functions, or if the number of samples is large,
then a histogram can be used to determine the density function
(normally by fitting to a know class of density functions).
10/30/2006
12.215 Modern Naviation L12
5
Example of random variables
4.0
3.0
Random variable
2.0
1.0
0.0
-1.0
-2.0
-3.0
-4.0
0.00
10/30/2006
Uniform
Gaussian
200.00
400.00
Sample
600.00
12.215 Modern Naviation L12
800.00
6
Histograms of random variables
Gaussian
Uniform
490/sqrt(2pi)*exp(-x^2/2)
Number of samples
200.0
150.0
100.0
50.0
0.0
-3.75
10/30/2006
-2.75
-1.75
-0.75
0.25
1.25
Random Variable x
12.215 Modern Naviation L12
2.25
3.25
7
Characterization Random Variables
• When the probability distribution is known, the
following statistical descriptions are used for random
variable x with density function f(x):
Expected Value
Expectation
Variance
Moments
< h(x) >
<x>
< (x − μ) 2 >
∫ h(x) f (x)dx
∫ xf (x)dx = μ
∫ (x − μ) 2 f (x)dx
< (x − μ) n > ∫ (x − μ) n f (x)dx
Square root of variance is called standard deviation
10/30/2006
12.215 Modern Naviation L12
8
Theorems for expectations
• For linear operations, the following theorems are
used:
– For a constant <c> = c
– Linear operator <cH(x)> = c<H(x)>
– Summation <g+h> = <g>+<h>
• Covariance: The relationship between random
variables fxy(x,y) is joint probability distribution:
σ xy =< (x − μx )(y − μy ) >=
∫ (x − μ )(y − μ ) f
x
y
xy
(x, y)dxdy
Correlation : ρ xy = σ xy /σ xσ y
10/30/2006
12.215 Modern Naviation L12
9
Estimation on moments
• Expectation and variance are the first and second moments of a
probability distribution
N
1
μˆ x ≈ ∑ x n /N ≈
T
n=1
∫ x(t)dt
N
N
n=1
n=1
σˆ x2 ≈ ∑ (x − μx ) 2 /N ≈ ∑ (x − μˆ x ) 2 /(N −1)
• As N goes to infinity these expressions approach their
expectations. (Note the N-1 in form which uses mean)
10/30/2006
12.215 Modern Naviation L12
10
Probability distributions
• While there are many probability distributions there
are only a couple that are common used:
1
−(x− μ )2 /(2σ 2 )
Gaussian f (x) =
e
σ 2π
1
− (x− μ )T V −1 (x− μ )
1
2
Multivariant f ( x ) =
e
(2π ) n V
r / 2−1 −x / 2
x
e
2
Chi − squared χ r (x) =
Γ(r /2)2 r / 2
10/30/2006
12.215 Modern Naviation L12
11
Probability distributions
• The chi-squared distribution is the sum of the squares
of r Gaussian random variables with expectation 0
and variance 1.
• With the probability density function known, the
probability of events occurring can be determined.
For Gaussian distribution in 1-D; P(|x|<1σ) = 0.68;
P(|x|<2σ) = 0.955; P(|x|<3σ) = 0.9974.
• Conceptually, people think of standard deviations in
terms of probability of events occurring (ie. 68% of
values should be within 1-sigma).
10/30/2006
12.215 Modern Naviation L12
12
Central Limit Theorem
• Why is Gaussian distribution so common?
• “The distribution of the sum of a large number of
independent, identically distributed random variables
is approximately Gaussian”
• When the random errors in measurements are made
up of many small contributing random errors, their
sum will be Gaussian.
• Any linear operation on Gaussian distribution will
generate another Gaussian. Not the case for other
distributions which are derived by convolving the two
density functions.
10/30/2006
12.215 Modern Naviation L12
13
Covariance matrices
• For large systems of random variables (such as GPS range
measurements, position estimates etc.), the variances and
covariances are arranged in a matrix called the variancecovariance matrix (or simply covariance matrix).
• This is the matrix used in multivariate Gaussian probability
density function.
⎡σ 2 σ
12
⎢ 1
2
σ
σ
2
C = ⎢ 12
⎢ M
M
⎢
⎣σ 1n σ 2n
10/30/2006
L σ 1n ⎤
⎥
L σ 2n ⎥
Notice that matrix is symmetric
⎥
O M
2⎥
L σn ⎦
12.215 Modern Naviation L12
14
Properties of covariance matrices
• Covariance matrices are symmetric
• All of the diagonal elements are positive and usually
non-zero since these are variances
• The off diagonal elements need to satisfy at least that
-1<=σij/(σiσj)<=1 where σi and σj are the standard
deviations (square root of diagonal elements)
• The matrix has all positive (or zero) eigenvalues
• Matrix is positive definite (i.e., for all vectors x, xTVx is
positive)
10/30/2006
12.215 Modern Naviation L12
15
Propagation of Variance-Covariance
matrices
• Given the covariance matrix of a set of random
variables, the characteristics of expected values can
be used to determine the covariance matrix of any
linear combination of the measurements.
Given linear operation : y = Ax with Vxx as
covariance matrix of x
Vyy =< yy T >=< Axx T A T >= A < xx T > A T
Vyy = AVxx A T
10/30/2006
12.215 Modern Naviation L12
16
Applications
• Propagation of variances is used extensively to
“predict” the statistical character of quantities derived
by linear operations on random variables
• Because of the Central Limit theorem we know that if
the random variables used in the linear relationships
are Gaussianly distributed, then the resultant random
variables will also be Gaussianly distributed.
• Example:
⎡σ 2 σ ⎤
⎡ x1 ⎤
1
12
for random variable ⎢ ⎥
C =⎢
2⎥
⎣x 2 ⎦
⎣σ 12 σ 2 ⎦
y = x1 ± x 2 then σ y2 = σ 12 + σ 22 ± 2σ 12
10/30/2006
12.215 Modern Naviation L12
17
Application
• Notice here that depending on σ12 (sign and
magnitude), the sum or difference of two random
variables can either be very well determined (small
variance) or poorly determined (large variance)
• If σ12 is zero (uncorrelated in general and independent
for Gaussian random variables (all moments are
zero), then the sum and difference have the same
variance.
• Even if the covariance is not known, then an upper
limit can be placed on the variance of the sum or
difference (since the value of σ12 is bounded).
• We return to covariance matrices after covering
estimation.
10/30/2006
12.215 Modern Naviation L12
18
Summary
• Basic Statistics
– Statistical description and parameters
•
•
•
•
Probability distributions
Descriptions: expectations, variances, moments
Covariances
Estimates of statistical parameters
• Propagation of variances
– Methods for determining the statistical parameters
of quantities derived other statistical variables
10/30/2006
12.215 Modern Naviation L12
19