Page 213 -
P. 213

Section 6.3  Synthesizing Textures and Filling Holes in Images  181


                                                                    Onionskin order



                              Image        Hole
















                                                                   Boundary edges

                            FIGURE 6.14: Texture synthesis methods can fill in holes accurately, but the order in
                            which pixels are synthesized is important. In this figure, we wish to remove the sign,
                            while preserving the signpost. Generally, we want to fill in pixels where most of the
                            neighbors are known first. This yields better matching patches. One way to do so is to fill
                            in from the boundary. However, if we simply work our way inwards (onionskin filling), long
                            scale image structures tend to disappear. It is better to fill in patches close to edges first.
                            This figure was originally published as Figure 11 of “Region Filling and Object Removal
                            by Exemplar-Based Image Inpainting,” by A. Criminisi, P. Perez, and K. Toyama, IEEE
                            Transactions on Image Processing, 2004 c   IEEE, 2004.


                            example, see Figure 6.14); in practice, this means that it is important to synthesize
                            patches at edges on the boundary before one fills in other patches. It is possible
                            to capture both requirements in a priority function ((Criminisi et al. 2004)), which
                            specifies where to synthesize next.
                                 If we choose an image patch at (i, j) as an example to fill in location (u, v)
                            in the hole, then image patches near (i, j) are likely to be good for filling in points
                            near (u, v). This observation is the core of coherence methods, which apply this
                            constraint to texture synthesis. Finally, some holes in images are not really texture
                            holes; for example, we might have a hole in a smoothly shaded region. Texture
                            synthesis and matching methods tend to work poorly on such holes, because the
                            intensity structure on the boundary is not that distinctive. As a result, we may
                            find many matching patches, some of which have jarring interiors. Variational
                            methods apply in these cases. Typically, we try to extend the level curves of the
                            image into the hole in a smooth way. Modern hole-filling methods use a combi-
                            nation of these approaches, and can perform very well on quite demanding tasks
                            (Figure 6.15).
   208   209   210   211   212   213   214   215   216   217   218