Page 229 - Matrix Analysis & Applied Linear Algebra

224 Chapter 4 Vector Spaces

minimal. The distance from (t i ,b i ) to a line f(t)= α + βt is
ε i = |f(t i ) − b i | = |α + βt i − b i |,
so that the objective is to find values for α and β such that
m m
2 2
ε = (α + βt i − b i ) is minimal.
i
i=1 i=1
Minimization techniques from calculus tell us that the minimum value must
occur at a solution to the two equations

m 2
∂ i=1 (α + βt i − b i )
m
0= =2 (α + βt i − b i ) ,
∂α
i=1

m 2

∂ (α + βt i − b i ) m
i=1
0= =2 (α + βt i − b i ) t i .
∂β
i=1
Rearranging terms produces two equations in the two unknowns α and β

m m m

1 α + t i β = b i ,
i=1 i=1 i=1
(4.6.1)
m m m

2
t i α + t i β = t i b i .
i=1 i=1 i=1
By setting
1 t 1 b 1
   
 1 t 2   b 2 
α
A =  . .  , b =  .  , and x = ,
 . β
.
. . .   . 
1 t m b m
T
T
we see that the two equations (4.6.1) have the matrix form A Ax = A b.
In other words, (4.6.1) is the system of normal equations associated with the
system Ax = b (see p. 213). The t i ’s are assumed to be distinct numbers,
so rank (A)=2, and (4.5.7) insures that the normal equations have a unique
solution given by
−1
T T
x = A A A b
2
1 t i − t i b i
= 2
2
m t − ( t i ) − t i m t i b i
i
1 t 2 i b i − t i t i b i
α
= 2 = .
2
m t − ( t i ) m t i b i − t i b i β
i
Finally, notice that the total sum of squares of the errors is given by
m m
2 2 T
ε = (α + βt i − b i ) =(Ax − b) (Ax − b).
i
i=1 i=1