Page 219 - Matrix Analysis & Applied Linear Algebra

214 Chapter 4 Vector Spaces

This follows because a unique solution (to either system) exists if and only if
T T
0 = N (A)= N A A , and this insures (A A) n×n must be nonsingular (by
(4.2.11)), so (4.5.7) is the unique solution to both systems. Caution! When
A is not square, A −1 does not exist, and the reverse order law for inversion
−1
T
doesn’t apply to A A , so (4.5.7) cannot be further simplified.
There is one outstanding question—what do the solutions of the normal
T
T
equations A Ax = A b represent when the original system Ax = b is not
consistent? The answer, which is of fundamental importance, will have to wait
until §4.6, but let’s summarize what has been said so far.
Normal Equations

• For an m × n system Ax = b, the associated system of normal
T
T
equations is defined to be the n × n system A Ax = A b.
T
T
• A Ax = A b is always consistent, even when Ax = b is not
consistent.
• When Ax = b is consistent, its solution set agrees with that of
T
T
A Ax = A b. As discussed in §4.6, the normal equations provide
least squares solutions to Ax = b when Ax = b is inconsistent.
T
T
• A Ax = A b has a unique solution if and only if rank (A)= n,
T −1 T
in which case the unique solution is x = A A A b.
• When Ax = b is consistent and has a unique solution, then the
T
T
same is true for A Ax = A b, and the unique solution to both
−1
T T
systems is given by x = A A A b.
Example 4.5.1
T
Caution! Use of the product A A or the normal equations is not recom-
mended for numerical computation. Any sensitivity to small perturbations that
is present in the underlying matrix A is magnified by forming the product
T
A A. In other words, if Ax = b is somewhat ill-conditioned, then the asso-
T
T
ciated system of normal equations A Ax = A b will be ill-conditioned to an
T
even greater extent, and the theoretical properties surrounding A A and the
normal equations may be lost in practical applications. For example, consider
the nonsingular system Ax = b, where

36 9
A = and b = .
12.01 3.01
If Gaussian elimination with 3-digit floating-point arithmetic is used to solve
Ax = b, then the 3-digit solution is (1, 1), and this agrees with the exact