Page 269 - A Course in Linear Algebra with Applications

7.4: The Method of Least Squares 253

T
The first step is to identify the normal system (A A)X =
T
A B;
6Xx — 3x 3 = 11
- = 1
2x 2 3x 3
—3xi — 3x2 + 6x3 = —7
Any solution of this will do; for example, we can take the
solution vector

'-(?)•

To obtain an optimal least squares solution, find the projec-
T
tion of X on the column space of A ; the first two columns
of A T form a basis of this space. Proceeding as in Example
7.2.12, we find the optimal solution to be

so that Xi = 67/42, x 2 = —3/14, X3 = —10/21 is the optimal
least squares solution of the linear system.
Least squares in inner product spaces
In 7.4.4 we obtained a geometrical interpretation of the
least squares process in R n in terms of projections on sub-
spaces. This raises the question of least squares processes in
an arbitrary finite-dimensional real inner product space V.
First we must formulate the least squares problem in V.
This consists in approximating a vector v in V by a vector in
a subspace S of V. A natural way to do this is to choose x in
2
S so that ||x — v|| is as small as possible. This is a direct
n
generalization of the least squares problem in R . For, if we
are given the linear system AX = B and we take S to be the
column space of A, v to be B and x to be the vector AX of
2
S, then the least squares problem is to minimize || AX" — -E?|| .