Page 33 - Compact Numerical Methods For Computers

Formal problems in linear algebra 23
T
so that A A is non-negative definite. In fact, if the columns of A are linearly
independent, it is not possible for y to equal the order-m null vector 0, so that in
T
this case A A is positive definite. This is also called the full-rank case.
Secondly, many computer programs for solving the linear least-squares problem
ignore the existence of special algorithms for the solution of linear equations
having a symmetric, positive definite coefficient matrix. Above it has already been
T
established that A A is positive definite and symmetry is proved trivially. The
special algorithms have advantages in efficiency and reliability over the methods
for the general linear-equation problem.
T
Thirdly, in chapter 5 it will be shown that the formation of A A can lead to loss
of information. Techniques exist for the solution of the least-squares problem
without recourse to the normal equations. When there is any question as to the
true linear independence of the columns of A, these have the advantage that they
permit the minimum-length least-squares solution to be computed.
It is worth noting that the linear-equation problem of equation (2.2) can be
solved by treating it as a least-squares problem. Then for singular matrices A
there is still a least-squares solution x which, if the system of equations is
T
consistent, has a zero sum of squares r r. For small-computer users who do not
regularly need solutions to linear equations or whose equations have coefficient
matrices which are near-singular (ill conditioned is another way to say this), it is
my opinion that a least-squares solution method which avoids the formation of
T
A A is useful as a general approach to the problems in both equations (2.2) and
(2.14).
As for linear equations, linear least-squares problems are categorised by
whether or not they can be stored in the main memory of the computing device at
hand. Once again, the traditional terms dense and sparse will be used, though
some problems having m large and n reasonably small will have very few zero
entries in the matrix A.

Example 2.3. Least squares
It is believed that in the United States there exists a linear relationship between
farm money income and the agricultural use of nitrogen, phosphate, potash and
petroleum. A model is therefore formulated using, for simplicity, a linear form

(money income) = x + x (nitrogen) + x (phosphate) + x (potash) + x (petroleum).
3
2
5
4
1
(2.28)
For this problem the data are supplied as index numbers (1940 = 100) to avoid
difficulties associated with the units in which the variables are measured. By
collecting the values for the dependent variable (money income) as a vector b and
the values for the other variables as the columns of a matrix A including the
constant unity which multiplies x , a problem
1
A x b (2.14)
is obtained. The data and solutions for this problem are given as table 3.1 and
example 3.2.