Page 461 - Advanced Linear Algebra

Singular Values and the Moore–Penrose Inverse 445

<
Theorem 17.1 Let and be finite-dimensional inner product spaces over d
=
s
B
or and let ²<Á = ³ have rank . Then there are ordered orthonormal
bases and for and , respectively, for which
=
9
<
8
’•“•” ’•••“•••”

8 ~ ²" Á ÃÁ" Á " b ÁÃÁ" ³
ONB for im ²³ ONB for ker ²³
i

and
’•“•” ’•••“•••”

9 ~ ² # Á ÃÁ# Á # b ÁÃÁ# ³

i
ONB for im ²³ ONB for ker ²³

Moreover, for   ,

"~ #
i

#~ "

where € are called the singular values of , defined by

i
"~ " Á €

for  . The vectors " ÁÃÁ" are called the right singular vectors for and

are called the left singular vectors
the vectors #Á Ã Á # for .
The matrix version of the previous discussion leads to the well-known singular-
value decomposition of a matrix. Let (  C ²-³ and let 8 Á ~ ²" Á à Á " ³

=
<
and 9 ~²# Á Ã Á # ³ be the orthonormal bases from and , respectively, in

. Then
Theorem 17.1, for the operator (

´ µ 89Á ~ ' ~ diag ² Á ÁÃÁ Á ÁÃÁ ³

A change of orthonormal bases from the standard bases to and gives
:
9
( ~ ´ µ ~ Á 4 µ ( 8 9 ;8 ~ Á 7 8 ' ( ;; i
´ Á 4 Á9 ;
are unitary/orthogonal. This is the singular-
where 7~ 4 9;Á and 8 ~ 4 8 ;Á
value decomposition of .
(
'
As to uniqueness, if (~ 7 8 i , where and are unitary and is diagonal,
8
'
7
with diagonal entries , then

i
i
i
i
(( ~ ²7 8 ³ 7 8 ~ 8 ' i ' 8 i
'
'
and since '' ~ i ² diag Á Ã Á ³ , it follows that the 's are eigenvalues of
i
((, that is, they are the squares of the singular values along with a sufficient
number of 's. Hence, is uniquely determined by , up to the order of the
(
'

diagonal elements.