Page 267 - A Course in Linear Algebra with Applications

7.4: The Method of Least Squares 251

squares solution X if and only if X is uniquely determined by
AX, that is, if AX = AX implies that X = X. Hence X is
unique if and only if the null space of A is zero, that is, if the
rank of A is n. Therefore we can state

Corollary 7.4.5
There is a unique least squares solution of the linear system
AX = B if and only if the rank of A equals the number of
columns of A.

Optimal least squares solutions
Returning to the general least squares problem for the
linear system AX = B with n unknowns, we would like to be
able to say something about the least squares solutions in the
case where the rank of A is less than n. In this case there
will be many least square solutions; what we have in mind is
to find a sensible way of picking one of them. Now a natural
way to do this would be to select a least squares solution
with minimal length. Accordingly we define an optimal least
squares solution of AX = B to be a least squares solution X
whose length \\X\\ is as small as possible.
There is a simple method of finding an optimal least
squares solution. Let U denote the null space of A; then U
T ±
equals (column space of A ) , by 7.2.6. Suppose X is a least
squares solution of the system AX = B. Now there is a unique
expression X = XQ + X\ where XQ belongs to U and X\ be-
1
longs to U ; this is by 7.2.4. Then AX = AX 0 + AX X = AX ±;
for AXQ = 0 since XQ belongs to the null space of A. Thus
AX — B = AXi — B, so that X\ is also a least squares solution
of AX = B. Now we compute
T
||X|| 2 = \\XQ+X X\\ 2 = (X 0+X 1) (X 0+X 1) = XZXQ+XTXL

For XQXI = 0 = XJ'XQ since X 0 and X x belong to U and
1
U - respectively. Therefore
2
2
2
||x|| HI*ol| 2 + ||Xi|| >||Xi|| .