Page 230 -

Section 7.1 Binocular Camera Geometry and the Epipolar Constraint 198

P A D
B C

O O’ O O’

d a’
p p’ c b c’ b’
a d’
FIGURE 7.2: The binocular fusion problem: In the simple case of the diagram shown on
the left, there is no ambiguity, and stereo reconstruction is a simple matter. In the more
usual case shown on the right, any of the four points in the left picture may, apriori,
match any of the four points in the right one. Only four of these correspondences are
correct; the other ones yield the incorrect reconstructions shown as small gray discs.

a brief incursion into human stereopsis (Section 7.3), we switch with Section 7.4 to
the presentation of several algorithms for binocular fusion that rely on the compar-
ison of local brightness or edge patterns to establish correspondences. Section 7.5
shows that ordering and smoothness constraints among nearby pixels can be in-
corporated in the matching process. In this setting, stereo fusion is naturally cast
as a combinatorial optimization problem, which can be solved by several efficient
algorithms (Chapter 22). We conclude in Section 7.6 with a discussion of multi-
camera stereo fusion (see also Chapter 19 for applications of multi-view stereopsis
to image-based modeling and rendering).
Note: We assume throughout that all cameras have been carefully calibrated so
their intrinsic and extrinsic parameters are precisely known relative to some fixed
world coordinate system. The case of uncalibrated cameras is examined in the
context of structure from motion in Chapter 8.

7.1 BINOCULAR CAMERA GEOMETRY AND THE EPIPOLAR CONSTRAINT
As noted in the introduction, it appears apriori that, given a stereo image pair,
any pixel in the first (or left) image may match any pixel in the second (or right)
one. As shown in this section, matching pairs of pixels are in fact restricted to
lie on corresponding epipolar lines in the two pictures. This constraint plays a
fundamental role in the stereo fusion process because it reduces the quest for image
correspondences to a set of one-dimensional searches.

7.1.1 Epipolar Geometry

Consider the images p and p of a point P observed by two cameras with optical
centers O and O . These five points all belong to the epipolar plane defined by

the two intersecting rays OP and O P (Figure 7.3). In particular, the point p lies

on the line l where this plane and the retina Π of the second camera intersect.