Page 322 -

Appendix C - Orthonormal Transformation

Suppose that we are presented with feature vectors y that have correlated features
and covariance matrix C. The aim is to determine a linear transformation that will
yield a new space with uncorrelated features. Following the explanation in section
2.3, assume that we knew the linear transformation matrix A that generated the
vectors, y = Ax, with x representing feature vectors that have unit covariance
matrix I as illustrated in Figure 2.9.
Given A, it is a simple matter to determine the matrix Z of its unit length
eigenvectors:

with z;z, = 0 and Z'Z = I (i.e., Z is an orthonormal matrix). (B- 1 a)

Let us now apply to the feature vectors y a linear transformation with the
transpose of Z, obtaining new feature vectors u:

u = Z'y. (B-2)

Notice that from the definition of eigenvectors, Azi = Aizi, one has:

AZ = AZ,

where A is the diagonal matrix of the eigenvalues:

Using (B-3) and well-known matrix properties, one can compute the new
covariance matrix K of the feature vectors u, as:
(2-18b) (A is symmetric)
K = Z'CZ=Z'AIA'Z=Z'AA'Z = Z'AIAZ=