Page 260 - A Course in Linear Algebra with Applications

244 Chapter Seven: Orthogonality in Vector Spaces

for k = 1,2,... ,n. This is a new linear system of equations
in x\,..., x n whose matrix form is

T
T
(A A)X = A B.
It is called the normal system of the linear system AX = B.
The solutions of the normal system are the critical points of
E.
Now E surely has an absolute minimum - after all it is a
continuous function with non-negative values. Since the func-
tion E is unbounded when \XJ\ is large, its absolute minima
must occur at critical points. Therefore we can state:

Theorem 7.4.1
Every least squares solution of the linear system AX — B is
T
T
a solution of the normal system (A A)X = A B.
At this point potential difficulties appear: what if the
normal system is inconsistent? If this were to happen, we
would have made no progress whatsoever. And even if the
normal system is consistent, need all its solutions be least
squares solutions?
To help answer these questions, we establish a simple
result about matrices.
Lemma 7.4.2
7
Let A be a real mxn matrix. Then A A is a symmetric nxn
matrix whose null space equals the null space of A and whose
T
column space equals the column space of A .
Proof
T
T
T T
T
T
In the first place {A A) T = A {A ) = A A, so A A is
certainly symmetric. Let S be the column space of A. Then
L
by 7.2.6 the null space of A T equals S .
Let X be any n-column vector. Then X belongs to the
T
T
null space of A A if and only if A (AX) = 0; this amounts
to saying that AX belongs to the null space of A T or, what