Page 298 -

6.1 2D and 3D feature-based alignment 277

The minimum can be found by solving the symmetric positive definite (SPD) system of nor-
mal equations 2
Ap = b, (6.8)

where
T
A = J (x i )J(x i ) (6.9)
i
T
is called the Hessian and b = J (x i )Δx i . For the case of pure translation, the result-
i
ing equations have a particularly simple form, i.e., the translation is the average translation
between corresponding points or, equivalently, the translation of the point centroids.
Uncertainty weighting. The above least squares formulation assumes that all feature
points are matched with the same accuracy. This is often not the case, since certain points
may fall into more textured regions than others. If we associate a scalar variance estimate σ i 2
with each correspondence, we can minimize the weighted least squares problem instead, 3

−2 2
E WLS = σ r i . (6.10)
i
i
As shown in Section 8.1.3, a covariance estimate for patch-based matching can be obtained
by multiplying the inverse of the patch Hessian A i (8.55) with the per-pixel noise covariance
2
σ (8.44). Weighting each squared residual by its inverse covariance Σ −1 = σ −2 A i (which
n i n
is called the information matrix), we obtain
2 T −1 −2 T
E CWLS = r i −1 = r Σ r i = σ n r A i r i . (6.11)
i
i
Σ i
i
i i i
6.1.2 Application: Panography
One of the simplest (and most fun) applications of image alignment is a special form of image
stitching called panography. In a panograph, images are translated and optionally rotated and
scaled before being blended with simple averaging (Figure 6.3). This process mimics the
photographic collages created by artist David Hockney, although his compositions use an
opaque overlay model, being created out of regular photographs.
In most of the examples seen on the Web, the images are aligned by hand for best artistic
effect. 4 However, it is also possible to use feature matching and alignment techniques to
perform the registration automatically (Nomura, Zhang, and Nayar 2007; Zelnik-Manor and
Perona 2007).
Consider a simple translational model. We want all the corresponding features in different
images to line up as best as possible. Let t j be the location of the jth image coordinate frame
in the global composite frame and x ij be the location of the ith matched feature in the jth
image. In order to align the images, we wish to minimize the least squares error
2
E PLS = (t j + x ij ) − x i , (6.12)
ij
2 For poorly conditioned problems, it is better to use QR decomposition on the set of linear equations J(x i )p =
Δx i instead of the normal equations (Bj¨ orck 1996; Golub and Van Loan 1996). However, such conditions rarely
arise in image registration.
3 Problems where each measurement can have a different variance or certainty are called heteroscedastic models.
4 http://www.flickr.com/groups/panography/.