Page 80 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

DATA FITTING 69

The least squares fit,or least squares estimate (LS) is the parameter
vector which minimizes this norm:
n o
T
^ x x LS ðzÞ¼ argmin ðz hðxÞÞ ðz hðxÞÞ ð3:49Þ
x
If v is random with a normal distribution, zero mean and covariance
2
matrix C v ¼ I, the LS estimate is identical to the ML estimate:
v
^ x x LS ^ x ML . To see this, it suffices to note that in the Gaussian case the
x
likelihood takes the form
T
1 ðz hðxÞÞ ðz hðxÞÞ !
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi exp ð3:50Þ
pðzjxÞ¼ q 2 2
N 2N
ð2 Þ v v
The minimization of (3.48) is equivalent to the minimization of (3.50).
If the measurement function is linear, that is z ¼ Hx þ v, and H is an
N M matrix having a rank M with M < N, then according to (3.25):

1
T
T
x
^ x x LS ðzÞ¼ ^ x ML ðzÞ¼ ðH HÞ H z ð3:51Þ
Example 3.7 Repeated measurements
Suppose that a scalar parameter x is N times repeatedly measured
using a calibrated measurement device: z n ¼ x þ v n . These repeated
T
measurements can be represented by a vector z ¼ [z 1 ... z N ] . The
T
corresponding measurement matrix is H ¼ [1 .. . 1] . Since
T
(H H) 1 ¼ 1/N, the resulting least squares fit is:
N
1 T 1 X
^ x x LS ¼ H z ¼ z n
N N
n¼1
In other words, the best fit is found by averaging the measurements.

Nonlinear sensors

If h(:) is nonlinear, an analytic solution of (3.49) is often difficult. One is
compelled to use a numerical solution. For that, several algorithms exist,
such as ‘Gauss–Newton’, ‘Newton–Raphson’, ‘steepest descent’ and
many others. Many of these algorithms are implemented within
MATLAB’s optimization toolbox. The ‘Gauss–Newton’ method will be
explained shortly.