System Identification
6.435
SET 7
– Parameter Estimation Methods
– Minimum Prediction Error Paradigm
– Maximum Likelihood
Munther A. Dahleh
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
1
Parameters Estimation Methods
Paradigm: Pick the parameters that minimize a scalar function of
the prediction error. “Min Prediction Error Paradigm”.
Set-Up:
where m
is a model structure.
Prediction error
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
2
Data
Problem: Minimize the prediction error.
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
3
Points to consider:
1) The choice of
is arbitrary. Typical choices
Typical choices
2)
may or may not correspond to a linear regression:
ARX
ARMAX
pseudo linear
3) Practical considerations
– Choice of l ; Robustness
– Filtering the data (equivalently
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
).
4
Quadratic Criterion
Recall:
Let
Lecture 7
to be the N-point DFT of
6.435, System Identification
Prof. Munther A. Dahleh
5
Then:
However
Then
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
6
where
Strong relationship to spectral estimation: What is the best
approximation of
that lies in m.
We will do more with this formula.
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
7
MIMO Systems
Define
Examples of h:
for some weighting matrix Λ .
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
8
Maximum Likelihood
Assumption:
are independent and have the PDF
Comment:
will have the distribution of the noise when
actual value. The above is an approximation.
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
9
Define
Then
min prediction error
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
10
Example:
is Gaussian
known
Quadratic objective
unknown
Parameterized norm criterion;
Information matrix:
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
11
Suppose l depends on
only. Then.
where
Evaluate the information matrix
are indep.
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
12
If
is Gaussian with variance
, it follows that
The Cramer-Rao Bound: for any unbiased estimator
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
13
Exact Likelihood Function
Example:
is WN with PDF
Need to find
Lecture 7
?
6.435, System Identification
Prof. Munther A. Dahleh
14
Approximate Likelihood
Lecture 7
6.435, System Identification
Prof. Munther A. Dahleh
15