Page 465 - Advanced Linear Algebra
P. 465
Singular Values and the Moore–Penrose Inverse 449
Theorem 17.3 Let ( 4 Á ²-³ . Among the least squares solutions to the
system
(% ~ # V
b
there is a unique solution of minimum norm, given by (# , where ( b is the MP
inverse of .
(
Proof. A vector is a least squares solution if and only if $ ( ~ V . Using the
$
#
characterization of the best approximation , we see that is a solution to
#
$ V
($ ~ # V if and only if
($ c # im ²(³
Since im²(³ ~ ker ²( ³ this is equivalent to
i
i
(²($ c #³ ~
or
i
i
(($ ~ (#
This system of equations is called the normal equations for (% ~ # . Its
solutions are precisely the least squares solutions to the system (% ~ # .
b
To see that $~( # is a least squares solution, recall that, in the notation of
Figure 17.1,
#
b
(( # ~ F
and so
i
i b (# i
((²( # ³ ~ F ~ (#
b
and since 9 ~²# Á Ã Á # ³ is a basis for , we conclude that ( # satisfies the
=
normal equations. Finally, since (# ker ²(³ , we deduce by the preceding
b
b
remarks that (# is the unique least squares solution of minimum norm.
Exercises
i
1. Let ²<³ . Show that the singular values of are the same as those of
B
.
2. Find the singular values and the singular value decomposition of the matrix
(~ > ?
Find ( b .
3. Find the singular values and the singular value decomposition of the matrix
