Page 270 - A Course in Linear Algebra with Applications

2 5 4 Chapter Seven: Orthogonality in Vector Spaces

It turns out that the solution of this general least squares
problem is the projection of v on S, just as in the special case
n
ofR .

Theorem 7.4.7
Let V be a finite-dimensional, real inner product space, and let
v be an element and S a subspace of V. Denote the projection
p
of v on S by . Then, if x is any vector in S other than p,
the inequality ||x — v|| > ||p — v|| holds.

Thus p is the vector in S which most closely approximates
v in the sense that it makes ||p — v|| as small as possible.

Proof
Since x and p both belong to S, so does x — p. Also p — v
1
belongs to S - since p is the projection of v on S. Hence
< p — v, x — p > = 0. It follows that

||x - v|| 2 = < (x - p) + (p - v), (x - p) + (p - v) >
= < x - p , x - p > + < p - v , p - v >
2
= ||x - P|| 2 + ||P - v|| > ||p - v f

since x — p ^ 0. Hence ||x — v|| > ||p — v||.

In applying 7.4.7 it is advantageous to have at hand an
orthonormal basis {vi,..., v m } of S. For the task of comput-
ing p, the projection of v on S, is then much easier since the
formula of 7.3.3 is available:

m
P = ^2 < V, Si > Sj .
i = l

Example 7.4.5
Use least squares to find a quadratic polynomial that approx-
imates the function e x in the interval [—1, 1].