Page 253 - Advanced engineering mathematics

7.8 Least Squares Vectors and Data Fitting 233

is a matrix of numbers, then
AX = x 1 C 1 + x 2 C 2 + ··· + x m C m = B

exactly when B is a linear combination of the columns of A, hence is in S.
The following lemma reveals a connection between the least squares vectors for AX=B and
orthogonal projections, as suggested by the inequality (7.2).

LEMMA 7.1

Let B be an n-vector. Then an m-vector X is a least squares vector for AX = B if and only if
∗
∗
AX = B S ,
where B S is the orthogonal projection of B onto S.

∗
Proof Suppose first that AX = B S . Then
∗
B − B S = B − AX
≤ B − C
for all vectors C in S, because B S is the vector in S closest to B. But the vectors C in S are exactly
m
the vectors AX for X in R ,so
∗
B − AX ≤ B − AX
for every m-vector X, and this proves that X is a least squares vector for AX = B.
∗
Conversely, suppose X is a lease squares vector for AX = B. Then
∗
∗
AX − B ≤ AX − B
∗
for all X in S. But then AX is the vector in S closest to B. Because B S is the unique vector with
this property, then AX = B S . This completes the proof.
∗
We are now able to completely characterize the least squares vectors of AX = B as the
solutions of a system of linear equations obtained using A.

THEOREM 7.17 Least Squares Vectors for AX = B

An m-vector X is a least squares vector of AX = B if and only if X is a solution of the system
A AX = A B.
t
t
Proof Suppose first that X is a least squares vector of AX = B. By the lemma,
∗
AX = B S .
∗
⊥
∗
⊥
We know that B − B S is in S ,so B − AX is in S . This means that the columns of A are
∗
∗
orthogonal to B − AX . Writing the dot product of column j of A with B − AX as a matrix
product, this orthogonality means that
t
∗
(C j ) (B − AX ) = 0.
t
t
Now (C j ) is row j of A ,so
t
∗
A (B − AX ) = O
in which O is the m × 1 zero matrix. But this equation can be written
t
t
A AX = A (B)
∗
Copyright 2010 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s).
Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
October 14, 2010 14:23 THM/NEIL Page-233 27410_07_ch07_p187-246