Page 71 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

60 PARAMETER ESTIMATION

estimator, i.e. a quadratic cost function. The constraint results in an
estimator that is not as good as the (unconstrained) MMSE estimator,
but it requires only knowledge of moments up to the order two, i.e.
expectation vectors and covariance matrices.
The starting point is the overall risk expressed in (3.11). Together with
the quadratic cost function we have:

Z Z
T
R ¼ ðKz þ a xÞ ðKz þ a xÞpðxjzÞpðzÞdxdz ð3:27Þ
z x
The optimal unbiased linear MMSE estimator is found by minimizing
R with respect to K and a. Hence we differentiate R with respect to a and
equate the result to zero:

dR Z Z
¼ ð2a þ 2Kz 2xÞpðxjzÞpðzÞdxdz
da z x

¼ 2a þ 2Km 2m ¼ 0
z
x
yielding:

a ¼ m K m z ð3:28Þ
x
with m and m the expectations of x and z.
z
x
Substitution of a back into (3.27), differentiation with respect to K,
and equating the result to zero (see also Appendix B.4):

Z Z
T
R ¼ ðKðz m Þ ðx m ÞÞ ðKðz m Þ ðx m ÞÞpðxjzÞpðzÞdxdz
x
z
z
x
z x
T

¼ trace KC z K þ C x 2KC zx
dR
¼ 2KC z 2C xz ¼ 0
dK
yields:
K ¼ C xz C 1 ð3:29Þ
z
T
C z is the covariance matrix of z, and C xz ¼ E[(x m )(z m ) ] the
z
x
cross-covariance matrix between x and z.