Page 150 - Video Coding for Mobile Communications Efficiency, Complexity, and Resilience

Section 5.2. Warping-Based Methods: A Review 127

Perspective:

u = g x (x; y)= a 1 x + a 2 y + a 3 ;
a 7 x + a 8 y +1
(5.6)

v = g y (x; y)= a 4 x + a 5 y + a 6 :
a 7 x + a 8 y +1

Motion estimation and compensation using higher-order models is usually
performed using the following steps:
1. A 2-D mesh is used to divide the current frame into nonoverlapping poly-
gonal patches (or elements). The points shared by the vertices of the
patches are referred to as grid or node points.

2. The motion of each node is estimated. This will map each node in the
current frame to a corresponding node in the reference frame. In e>ect,
this will map each patch in the current frame to a corresponding patch in
the reference frame.
3. For each patch in the current frame, the coordinates of its vertices and
those of the matching patch in the reference frame are used to (nd the
motion parameters {a i } of the underlying motion model.
4. During motion compensation, the estimated motion parameters {a i } are
substituted in the appropriate spatial transformation, Equations (5.4)–(5.6),
to warp the patch in the reference frame to provide a prediction for the
corresponding patch in the current frame.
An example of this process is illustrated in Figure 5.1. In this (gure the
current frame is divided into square patches. This forms a uniform mesh.
During motion estimation, node points A, B, C, and D in the current frame
�
are mapped to node points A ,B ,C , and D in the reference frame. During
�
�
�
�
motion compensation, the deformed patch A B C D is warped to provide a
�
�
�
prediction for the square patch ABCD.
It should be pointed out that there is a lack of consistency in the lit-
erature when referring to this type of motion estimation and compensation
methods. Examples of the numerous names employed are control grid in-
terpolation [108, 109, 110], warping-based methods [111, 112, 113], spatial-
transformation-based methods [114, 115, 116, 117], geometric-transformation-
based methods [118], generalized motion estimation methods [119, 120], and
mesh-based methods [121, 122, 123, 124, 125, 126, 127].
When designing a warping-based technique, several aspects of the
method need to be considered and de(ned, as discussed in the following
subsections.