Page 258 - A Course in Linear Algebra with Applications

242 Chapter Seven: Orthogonality in Vector Spaces

It remains to explain what is meant by the best-fitting
straight line. It is here that the "least squares" arise.
Consider the linear relation y = cx+d; this is the equation
of a straight line in the xy-plane. The conditions for the line
to pass through the m data points are

{ mi + d = b\

d
ca 2+
=
b 2
ca m + d — b m
Now in all probability these equations will be inconsistent.
However, we can ask for real numbers c and d which come
as close to satisfying the equations of the linear system as
possible, in the sense that they minimize the "total error". A
good measure of this total error is the expression

2 2
+ d- bi) H + d- b m) •
(ca x h (ca m
This is the sum of the squares of the vertical deviations of
the line from the data points in the diagram above. Here the
squares are inserted to take care of any negative signs that
might appear.
It should be clear the line-fitting problem is just a par-
ticular instance of a general problem about inconsistent linear
systems. Suppose that we have a linear system of m equations
in n unknowns x\,..., x n

AX = B.

Since the system may be inconsistent, the problem of interest
is to find a vector X which minimizes the length of the vector
AX — B, or what is equivalent and also a good deal more
convenient, its square,

E= WAX -Bf.