Page 261 - A Course in Linear Algebra with Applications

7.4: The Method of Least Squares 245

1
is the same thing, to S- . But AX also belongs to S; for it is
1
a linear combination of the columns of A. Now S fl S - is the
zero space by 7.2.3. Hence AX = 0 and X belongs to the null
space of A. On the other hand, it is obvious that if X belongs
to the null space of A, then it must belong to the null space
T
T
of A A. Hence the null space of A A equals the null space of
A.
Finally, by 7.2.6 and the last paragraph we can assert
that the column space of A T A equals

±
T
(null space of A A) ± = (null space ofA) .
T
This equals the column space of A , as claimed.
We come now to the fundamental theorem on the Method
of Least Squares.
Theorem 7.4.3
Let AX = B be a linear system ofm equations in n unknowns.
T
T
(a) The normal system (A A)X = A B is always con-
sistent and its solutions are exactly the least squares solutions
of the linear system AX = B;
T
(b) if A has rank n, then A A is invertible and there is
a unique least squares solution of the normal system, namely
l T
T
X = (A A)- A B.
Proof
T
By 7.4.2 the column space of A A equals the column space of
T
A . Therefore the column space of the matrix

T
T
[A A | A B\
T
T
equals the column space of A A; for the extra column A B
is a linear combination of the columns of A T and thus belongs
T
to the column space of A A. It follows that the coefficient
matrix and the augmented matrix of the normal system have