Page 262 - A Course in Linear Algebra with Applications

246 Chapter Seven: Orthogonality in Vector Spaces

the same rank. By 5.2.5 this is just the condition for the
normal system to be consistent.
The next point to establish is that every solution of the
normal system is a least squares solution of AX = B. Suppose
that X\ and X 2 are two solutions of the normal system. Then
T T T
A A{X X - X 2) = A B - A B = 0, so that Y = X x - X 2
T
belongs to the null space of A A. By 7.4.2 the latter equals
the null space of A. Thus AY = 0. Since X x - Y + X 2, we
have
AXi -B = A(Y + X 2)-B = AX 2 - B.

This means that E = \\AX — B\\ 2 has the same value for
X = Xi and X = X 2. Thus all solutions of the normal
system give the same value of E. Since by 7.4.1 every least
squares solution is a solution of the normal system, it follows
that the solutions of the normal system constitute the set of
all least squares solutions, as claimed.
Finally, suppose that A has rank n. Then the matrix
T
T
A A also has rank n since by 7.4.2 the column space of A A
T
equals the column space of A , which has dimension n. Since
T
A A is n x n, it is invertible by 5.2.4 and 2.3.5. Hence the
T
equation A AX = A T B leads to the unique solution
T
1 T
X = (A A)- A B,
which completes the proof On the other hand, if the rank of
A is less than n, there will be infinitely many least squares
solutions. We shall see later how to select one that is in some
sense optimal.
Example 7.4.1

Find the least squares solution of the following linear system:
xi + x 2 + x 3 = 4
-x\ + x 2 + x 3 — 0
- x 2 + x 3 =1
xi + x 3 = 2