Page 437 - Matrix Analysis & Applied Linear Algebra

5.13 Orthogonal Projection 433

Connections with Gram–Schmidt and QR. Recall from (5.5.4) on p. 309
that the vectors in the Gram–Schmidt sequence generated from a linearly inde-
m
pendent set {x 1 , x 2 ,..., x n }⊂ are u 1 = x 1 / x 1 and
2
T
I − U k U
k x k
u k =
T
, where U k = u 1 | u 2 |···| u k−1 for k> 1.

I − U k U
k x k 2
Since {u 1 , u 2 ,..., u k−1 } is an orthonormal basis for span {x 1 , x 2 ,..., x k−1 } ,
it follows from (5.13.4) that U k U T must be the orthogonal projector onto
k
T
T
span {x 1 , x 2 ,..., x k−1 } . Hence U k U = P k and (I−P k )x k =(I−U k U )x k ,
k
k

so
I − U k U T
= ν k is the k th projected height in (5.13.7). This means
k x k 2
that when the Gram–Schmidt equations are written in the form of a QR fac-
torization as explained on p. 311, the diagonal elements of the upper-triangular
matrix R are the ν k ’s. Consequently, the product of the diagonal entries in R
is the volume of the parallelepiped generated by the x k ’s. But the QR factor-

ization of A = x 1 | x 2 |···| x n is unique (Exercise 5.5.8), so it doesn’t matter
whether Gram–Schmidt or another method is used to determine the QR factors.
Therefore, we arrive at the following conclusion.
• If A m×n = Q m×n R n×n is the (rectangular) QR factorization of a matrix
with linearly independent columns, then the volume of the n-dimensional
parallelepiped generated by the columns of A is V n = ν 1 ν 2 ··· ν n , where
the ν k ’s are the diagonal elements of R. We will see on p. 468 what this
means in terms of determinants.
Of course, not all projectors are orthogonal projectors, so a natural question
to ask is, “What characteristic features distinguish orthogonal projectors from
more general oblique projectors?” Some answers are given below.

Orthogonal Projectors

n×n 2
Suppose that P ∈ is a projector—i.e., P = P. The following
statements are equivalent to saying that P is an orthogonal projector.
• R (P) ⊥ N (P). (5.13.8)
2
T
T
• P = P (i.e., orthogonal projector ⇐⇒ P = P = P ). (5.13.9)
• P =1 for the matrix 2-norm (p. 281). (5.13.10)
2
Proof. Every projector projects vectors onto its range along (parallel to) its
nullspace, so statement (5.13.8) is essentially a restatement of the definition of
an orthogonal projector. To prove (5.13.9), note that if P is an orthogonal
projector, then (5.13.3) insures that P is symmetric. Conversely, if a projector