Page 212 -

Section 6.3 Synthesizing Textures and Filling Holes in Images 180

FIGURE 6.13: If an image contains repeated structure, we have a good chance of finding
examples to fill a hole by searching for patches that are compatible with its boundaries.
Top left: An image with a hole in it (black pixels in a rough pedestrian shape). The
pixels on the region outside the hole, but inside the boundary marked on the image,
match pixels near the other curve, which represents a potentially good source of hole-
filling pixels. Top right: The hole filled by placing the patch over the hole, then using
a segmentation method (Chapter 9) to choose the right boundary between patch and
image. This procedure can work for apparently unpromising images, such as the one on
the bottom left, an image of the facade of a house, seen at a significant slant. This
slant means that distant parts of the facade are severely foreshortened. However, if we
rectify the facade using methods from Section 1.3, then there are matching patches. On
the bottom right, the hole has been filled in using a patch from the rectified image,
that is then slanted again. This figure was originally published as Figures 3 and 6 of
“Hole Filling through Photomontage,” by M. Wilczkowiak, G. Brostow, B. Tordoff, and
R. Cipolla, Proc. BMVC, 2005 and is reproduced by kind permission of the authors.

successful. Blending is typically achieved using methods also used for image seg-
mentation (Section 9.4.3 describes one method that can be used for blending).
As you would expect, matching methods work very well when a good match
is available, and poorly otherwise. If the hole is in a region of relatively regular
texture, then a good match should be easy to find. If the texture is less strongly
structured, it might be hard to find a good match. In cases like this, it makes
sense to try and synthesize the texture over the region of the hole, using the rest
of the image as an example. Making such texture synthesis methods work well
requires considerable care, because the order in which pixels are synthesized has
a strong effect on the results. Texture synthesis tends to work better for patches
when most of their neighbors are known, because the match is more constrained.
As a result, one wants to synthesize patches at the boundary of the hole. It is
also important to extend edges at the boundary of the hole into the interior (for