Page 154 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

PARAMETRIC LEARNING 143

5.2.1 Gaussian distribution, mean unknown

Let us assume that under class ! k the measurement vector z is a Gaussian
random vector with known covariance matrix C k and unknown expect-
ation vector m . No prior knowledge is available concerning this
k
unknown vector. The purpose is to find an estimator for m .
k
Since no prior knowledge about m is assumed, a maximum likelihood
k
estimator seems appropriate (Section 3.1.4). Substitution of (5.5) in (3.22)
gives the following general expression of a maximum likelihood estimator:

( )
N k
Y
^ m m ¼ argmax pðz n j! k ; mÞ
k
m n¼1
ð5:6Þ
( )
N k
X
¼ argmax lnðpðz n j! k ; mÞÞ
m n¼1
The logarithms introduced in the last line transform the product into
a summation. This is only a technical matter which facilitates the
maximization.
Knowing that z is Gaussian, the likelihood of m from a single observ-
k
ation z n is:

1 1 T 1
k
k
k
k
pðz n j! k ; m Þ¼ q ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi exp ðz n m Þ C ðz n m Þ ð5:7Þ
N 2
ð2 Þ jC k j
Upon substitution of (5.7) in (5.6), rearrangement of terms and elimin-
ation of irrelevant terms, we have:
( )
N k
X T
1
^ m m ¼ argmin ðz n mÞ C ðz n mÞ
k k
m n¼1
ð5:8Þ
( )
N k N k N k
1
1
1
T
X X X
T
t
¼ argmin z C z n þ m C m 2 z C m
k
n
k
k
n
m n¼1 n¼1 n¼1
Differentiating the expression between braces with respect to m (Appen-
dix B.4) and equating the result to zero yields the average or sample
mean calculated over the training set:
1 X
N k
^ m m ¼ z n ð5:9Þ
k
N k
n¼1