Page 412 -

Section 12.3 Registering Deformable Objects 380

Reference points Relaxed points Relaxed intensity

FIGURE 12.8: A set of reference points placed over a face, on the left.At the center,
these points in a relaxed configuration. Now assume we have a reasonable triangulation
of the original set of points. By placing those points in correspondence with the relaxed
configuration, we can map the intensities of the reference face to a relaxed configuration
(right). This figure was originally published as Figure 1 of “Active Appearance Models,”
by T. Cootes, G. Edwards, and C. Taylor, IEEE Transactions on Pattern Analysis and
Machine Intelligence, 2001, c IEEE, 2001.

(i.e., that the (s, t) values naturally interpolate between the vertices of the triangle).
We can then produce a neutral image of the face simply by moving the vertices to
their neutral position (Figure 12.8).
There is nothing special about the neutral locations of the mesh vertices; we
can generate an intensity field for any configuration of these vertices where triangles
don’t overlap. This means we can search for the location of a deformed triangle
in a new image I d by sampling (s, t) space at a set of points (s j ,t j ), and then
minimizing
2
g(||I d (p(s j ,t j ; w)) −I n (p(s j ,t j ; v))|| )
j
as a function of the vertices w i . Here, as before, if we do not expect outliers, then
g is the identity, and if we do, it could be some M-estimator. If we expect that the
illumination might change, then it makes sense to minimize
2
g(||aI d (p(s j ,t j ; w)) + b −I n (p(s j ,t j ; v))|| )
j

as a function of the vertices w i and of a, b.
When there is more than one triangle, the notation gets slightly more com-
plicated. We write v (k) and w (k) for the vertices of the kth neutral and deformed
triangles respectively. We do not expect the vertices to move independently. A vari-
ety of models are possible, but it is natural to try and make the model linear in some
set of parameters. One reasonable model is obtained by writing V =[v 1 ,... , v n ]
(resp. W =[w 1 ,..., w n ]) for the 2 × n matrices whose columns are the vertices of
the neutral (resp. deformed) points. Now we have a set of r 2×n basis matrices B l ,