Page 68 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

BAYESIAN ESTIMATION 57

. On the other hand, if the prior knowledge is weak, i.e. if the
variances of the parameters are very large, the inverse covariance
matrix C 1 tends to zero. In the limit, the estimate becomes:
x

T 1 1 T 1
^ x x MMSE ðzÞ¼ H C H H C z ð3:21Þ
v v
In this solution, the prior knowledge, i.e. m , is completely ruled out.
x
Note that the mode of a normal distribution coincides with the expec-
tation. Therefore, in the linear-Gaussian case, MAP estimation and
MMSE estimation coincide: ^ x MMSE (z) ¼ ^ x MAP (z).
x
x

3.1.4 Maximum likelihood estimation

In many practical situations the prior knowledge needed in MAP estima-
tion is not available. In these cases, an estimator which does not depend
on prior knowledge is desirable. One attempt in that direction is the
method referred to as maximum likelihood estimation (ML estimation).
The method is based on the observation that in MAP estimation, (3.17),
the peak of the first factor p(zjx) is often in an area of x in which the
second factor p(x) is almost constant. This holds true especially if
little prior knowledge is available. In these cases, the prior density p(x)
does not affect the position of the maximum very much. Discarding
the factor, and maximizing the function p(zjx) solely, gives the ML
estimate:

^ x x ML ðzÞ¼ argmaxfpðzjxÞg ð3:22Þ
x
Regarded as a function of x the conditional probability density is called the
likelihood function. Hence the name ‘maximum likelihood estimation’.
Another motivation for the ML estimator is when we change our
viewpoint with respect to the nature of the parameter vector x. In the
Bayesian approach x is a random vector, statistically defined by means
of probability densities. In contrast, we may also regard x as a non-
random vector whose value is simply unknown. This is the so-called
Fisher approach. In this view, there are no probability densities asso-
ciated with x. The only density in which x appears is p(zjx), but here x
must be regarded as a parameter of the density of z. From all estimators
discussed so far, the only estimator that can handle this deterministic
point of view on x is the ML estimator.