Page 43 -

22 1 Introduction

pensate for camera rotation and zooming, as well as affine or planar perspective motion (Sec-
tion 8.2). This is then generalized to spline-based motion models (Section 8.3) and finally
to general per-pixel optical flow (Section 8.4), including layered and learned motion models
(Section 8.5). Applications of these techniques include automated morphing, frame interpo-
lation (slow motion), and motion-based user interfaces.
Chapter 9 is devoted to image stitching, i.e., the construction of large panoramas and com-
posites. While stitching is just one example of computation photography (see Chapter 10),
there is enough depth here to warrant a separate chapter. We start by discussing various pos-
sible motion models (Section 9.1), including planar motion and pure camera rotation. We
then discuss global alignment (Section 9.2), which is a special (simplified) case of general
bundle adjustment, and then present panorama recognition, i.e., techniques for automatically
discovering which images actually form overlapping panoramas. Finally, we cover the topics
of image compositing and blending (Section 9.3), which involve both selecting which pixels
from which images to use and blending them together so as to disguise exposure differences.
Image stitching is a wonderful application that ties together most of the material covered
in earlier parts of this book. It also makes for a good mid-term course project that can build
on previously developed techniques such as image warping and feature detection and match-
ing. Chapter 9 also presents more specialized variants of stitching such as whiteboard and
document scanning, video summarization, panography, full 360 spherical panoramas, and
◦
interactive photomontage for blending repeated action shots together.
Chapter 10 presents additional examples of computational photography, which is the pro-
cess of creating new images from one or more input photographs, often based on the careful
modeling and calibration of the image formation process (Section 10.1). Computational pho-
tography techniques include merging multiple exposures to create high dynamic range images
(Section 10.2), increasing image resolution through blur removal and super-resolution (Sec-
tion 10.3), and image editing and compositing operations (Section 10.4). We also cover the
topics of texture analysis, synthesis and inpainting (hole filling) in Section 10.5, as well as
non-photorealistic rendering (Section 10.5.2).
In Chapter 11, we turn to the issue of stereo correspondence, which can be thought of
as a special case of motion estimation where the camera positions are already known (Sec-
tion 11.1). This additional knowledge enables stereo algorithms to search over a much smaller
space of correspondences and, in many cases, to produce dense depth estimates that can
be converted into visible surface models (Section 11.3). We also cover multi-view stereo
algorithms that build a true 3D surface representation instead of just a single depth map
(Section 11.6). Applications of stereo matching include head and gaze tracking, as well as
depth-based background replacement (Z-keying).
Chapter 12 covers additional 3D shape and appearance modeling techniques. These in-
clude classic shape-from-X techniques such as shape from shading, shape from texture, and
shape from focus (Section 12.1), as well as shape from smooth occluding contours (Sec-
tion 11.2.1) and silhouettes (Section 12.5). An alternative to all of these passive computer
vision techniques is to use active rangefinding (Section 12.2), i.e., to project patterned light
onto scenes and recover the 3D geometry through triangulation. Processing all of these 3D
representations often involves interpolating or simplifying the geometry (Section 12.3), or
using alternative representations such as surface point sets (Section 12.4).
The collection of techniques for going from one or more images to partial or full 3D