Page 242 - Matrix Analysis & Applied Linear Algebra
P. 242
4.6 Classical Least Squares 237
m×n m
4.6.9. For A ∈ and b ∈ , prove that x 2 is a least squares solution
for Ax = b if and only if x 2 is part of a solution to the larger system
I m×m A x 1 b
= . (4.6.5)
A T 0 n×n x 2 0
Note: It is not uncommon to encounter least squares problems in which
A is extremely large but very sparse (mostly zero entries). For these
situations, the system (4.6.5) will usually contain significantly fewer
nonzero entries than the system of normal equations, thereby helping to
overcome the memory requirements that plague these problems. Using
(4.6.5) also eliminates the undesirable need to explicitly form the prod-
T
T
uct A A —recall from Example 4.5.1 that forming A A can cause
loss of significant information.
4.6.10. In many least squares applications, the underlying data matrix A m×n
does not have independent columns—i.e., rank (A) <n —so the corre-
T
T
sponding system of normal equations A Ax = A b will fail to have
a unique solution. This means that in an associated linear estimation
problem of the form
y = α 1 t 1 + α 2 t 2 + ··· + α n t n + ε
there will be infinitely many least squares estimates for the parameters
α i , and hence there will be infinitely many estimates for the mean value
of y at any given point (t 1 ,t 2 ,...,t n ) —which is clearly an undesirable
situation. In order to remedy this problem, we restrict ourselves to mak-
ing estimates only at those points (t 1 ,t 2 ,...,t n ) that are in the row
space of A. If
t 1 ˆ α 1
t 2 ˆα 2
T
t = . ∈ R A , and if x = .
. .
. .
t n ˆ α n
T
T
is any least squares solution (i.e., A Ax = A b ), prove that the esti-
mate defined by
n
T
ˆ y = t x = t i ˆα i
i=1
is unique in the sense that ˆ is independent of which least squares
y
solution x is used.
