Page 268 - A Course in Linear Algebra with Applications

252 Chapter Seven: Orthogonality in Vector Spaces

Now, if X is an optimal solution, then ||X|| = \\X\\\, so that
1
\\X 0\\ = 0 and hence X Q = 0. Thus X = Xi belongs to U . It
follows that each optimal least squares solution must belong
T
1
to t/- , the column space of A .
Finally, we show that there is a unique least squares so-
±
lution in U . Suppose that X and X are two least squares
solutions in U^. Then from 7.4.4 we see that AX and AX are
both equal to the projection of B on the column space of A.
Thus A(X — X) = 0 and X — X belongs to U, the null space
of A. But X and X also belong to U^~, whence so does X — X.
Since U D U 1 = 0, it follows that X - X = 0 and X = X.
Hence X is the unique optimal least squares solution and it
s
belongs to V -. Combining these conclusions with 7.4.4, we
obtain:

Theorem 7.4.6
A linear system AX = B has a unique optimal least squares
solution, namely the unique vector X in the column space of
A T such that AX is the projection of B on the column space
T
ofA .
The proof of 7.4.6 has the useful feature that it tells us
how to find the optimal least squares solution of a linear sys-
tem AX = B. First find any least squares solution, and then
T
compute its projection on the column space of A .
Example 7.4.4
Find the optimal least squares solution of the linear system

Xl - X2 + X3 = 1
xi + x 2 - 2x 3 = 2
2xi - x 3 = 4