Page 146 - Matrices theory and applications

7.7. Singular Value Decomposition
129
of the singular values of an arbitrary matrix, not necessarily invertible.
∗
Recalling that (see Exercise 17 in Chapter 2) when M is n × m, M M and
∗
MM have the same nonzero eigenvalues, counting with multiplicities, one
may even speak of the singular values of a rectangular matrix, up to an
ambiguity concerning the multiplicity of the eigenvalue 0.
The main result of the section is the following.
Theorem 7.7.1 Let M ∈ M n×m (CC) be given. Then there exist two
unitary matrices U ∈ U n , V ∈ U m and a quasi-diagonal matrix
 
s 1
.
 . 
 . 
 
D =  s r  ,
 
0
 
 
. . .
with s 1 ,... ,s r ∈ (0, +∞), such that M = UDV .The numbers s 1 ,... ,s r
are uniquely defined up to permutation; they are the nonzero singular values
of M.In particular, r is therankof M.
If M ∈ M n×m (IR), then one may choose U, V to be real orthogonal.
Remark: The factorization given in the theorem is far from being unique,
even for invertible square matrices. In fact, the number of real degrees of
2
2
freedom in that factorization is n +m +min(n, m), which is always greater
than the dimension 2nm of M n×m (CC)as an IR-vector space.
Proof
Since MM ∗ is positive semidefinite, we may write its eigenvalues as
2
2
s ,... ,s , 0,... ,where the s j ’s, the singular values of M,are positive
r
1
∗
real numbers. The spectrum of M M has the same form, except for the
multiplicity of 0. Indeed, the multiplicities of 0 as an eigenvalue of MM ∗
and MM , respectively, differ by n − m, while the multiplicities of other
∗
eigenvalues are the same for both matrices. We set S = diag(s 1 ,... ,s r ).
Since M and MM have the same rank, and since R(MM ) ⊂ R(M), we
∗
∗
have R(MM )= R(M). Since MM is Hermitian, its kernel is R(M) ,
∗
∗
⊥
where orthogonality is relative to the canonical scalar product; with the
duality formula, we conclude that ker MM ∗ =ker M .Now we are in
∗
position to state that
n ∗ ⊥ ∗
CC = R(MM ) ⊕ ker M .
Therefore, there exists an orthonormal basis {u 1 ,... , u n } of CC n consist-
2
ing of eigenvectors of MM , associated to the s ’s, followed by vectors of
∗
j
∗
ker M . Let us form the unitary matrix
U =(u 1 ,... , u n ).
Written blockwise, we have U =(U R ,U K ), where
2
MM U R = U R S , M U K =0.
∗
∗