Page 266 - A Course in Linear Algebra with Applications

250 Chapter Seven: Orthogonality in Vector Spaces

Hence the least squares solution is

1
T
X = R~ Q B = 1 ) ,

t h a t is, X\ = 1, X2 = 0, X3 = 0.

Geometry of the least squares process
There is a suggestive geometric interpretation of the least
squares process in terms of projections. Consider the least
squares problem for the linear system AX = B where A has
m rows. Let S denote the column space of the coefficient
matrix A. The least squares solutions are the solutions of the
T
T
normal system A AX = A B, or equivalently
T
A {B - AX) = 0.
The last equation asserts that B — AX belongs to the null
1
T
space of A , which by 7.2.6 is equal to S . Our condition
can therefore be reformulated as follows: X is a least squares
1
solution of AX = B if and only if B — AX belongs to S .
Now B = AX + (B - AX) and AX belongs to S. Recall
from 7.2.4 that B is uniquely expressible as the sum of its
1
projections on the subspaces S and S- ; we conclude that
B — AX belongs to S 1 precisely when AX is the projection of
B on S. In short we have a discovered a geometric description
of the least squares solutions.

Theorem 7.4.4
Let AX = B be an arbitrary linear system and let S denote
the column space of A. Then a column vector X is a least
squares solution of the linear system if and only if AX is the
projection of B on S.

Notice that the projection AX is uniquely determined by
the linear system AX = B. However there is a unique least