Page 247 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB
P. 247
236 UNSUPERVISED LEARNING
the K Gaussians generated the corresponding object z n . The x n are called
indicator variables. They use position coding to indicate the Gaussian
associated with z n . In other words, if z n is generated by the k-th Gaus-
sian, then x n,k ¼ 1 and all other elements in x n are zero. With that, the
prior probability that x n,k ¼ 1is k .
In this case, the complete log-likelihood of C can be written as:
N S N S
Y Y
LðYjCÞ¼ ln pðz n ; x n jCÞ¼ ln pðz n jx n ; CÞpðx n jCÞ
n¼1 n¼1
K
N S
Y Y
¼ ln ½Nðz n jm ; C k Þ k x n;k ð7:16Þ
k
n¼1 k¼1
K N S
X X
¼ x n;k ln Nðz n jm ; C k Þþ x n;k ln k
k
k¼1 n¼1
Under the condition of a given Z, the probability density p(YjZ, C) ¼
p(X, ZjZ, C) can be replaced with the marginal probability P(XjZ, C).
Therefore in (7.15), we have:
Z
ðiÞ
E½LðYjCÞjZ¼ ð ln pðYjCÞÞpðYjZ; C ÞdY
ð7:17Þ
X
ðiÞ
¼ LðYjCÞPðXjZ; C Þ
X
But since in (7.16) L(YjC) is linear in X, we conclude that
E½LðYjCÞjZ¼ LðX; ZjCÞ ð7:18Þ
where X is the expectation of the missing data under the condition of Z
(i)
and C :
ðiÞ ðiÞ
x x n;k ¼ E½x n;k jz n ; C ¼ Pðx n;k jz n ; C Þ
ðiÞ
ðiÞ ðiÞ N z n jm ; C ðiÞ ðiÞ
pðz n jx n;k ; C ÞPðx n;k jC Þ k k k ð7:19Þ
¼ ¼
ðiÞ K
pðz n jC Þ P ðiÞ ðiÞ ðiÞ
N z n jm ; C j j
j
j¼1
The variable x n,k is called the ownership because it indicates to what
x
degree sample z n is attributed to the k-th component.
