Page 73 - Dynamic Vision for Perception and Control of Motion

2.4 Spatiotemporal Embedding and First-order Approximations 57

nature, if a corresponding management system for gaze control, knowledge appli-
cation and interpretation of multiple, piecewise smooth image sequences is avail-
able.

2.4.2 Role of Jacobian Matrix in the 4-D Approach to Dynamic Vision
It is in connection with 4-D spatiotemporal motion models that the sensitivity ma-
trix of perspective feature mapping gains especial importance. The dynamic mod-
els for motion in 3-D space link feature positions from one time to the next. Con-
trary to perspective mapping in a single image (in which depth information is
completely lost), the partial first-order derivatives of each feature with respect to
all variables affecting its appearance in the image do contain spatial information.
Therefore, linking the temporal motion process in 4-D with this physically mean-
ingful Jacobian matrix has brought about a quantum leap in visual dynamic scene
understanding [Dickmanns, Meissner 1983, Wünsche 1987, Dickmanns 1987, Dickmanns,
Graefe 1988, Dickmanns, Wuensche 1999]. This approach is fundamentally different
from applying some (arbitrary) motion model to features or objects in the image
plane as has been tried many times before and after 1987. It was surprising to learn
from a literature review in the late 1990s that about 80 % of so-called Kalman–
filter applications in vision did not take advantage of the powerful information
available in the Jacobian matrices when these are determined, including egomotion
and the perspective mapping process.
The nonchalance of applying Kalman filtering in the image plane has led to the
rumor of brittleness of this approach. It tends to break down when some of the (un-
spoken) assumptions are not valid. Disappearance of features by self-occlusion has
been termed a catastrophic event. On the contrary, Wünsche [1986] was able to
show that not only temporal predictions in 3-D space were able to handle this situa-
tion easily, but also that it is possible to determine a limited set of features allowing
optimal estimation results. This can be achieved with relatively little additional ef-
fort exploiting information in the Jacobian matrix. It is surprising to notice that this
early achievement has been ignored in the vision literature since. His system for
visually perceiving its state relative to a polyhedral object (satellite model in the
laboratory) selected four visible corners fully autonomously out of a much larger
total number by maximizing a goal function formed by entries of the Jacobian ma-
trix (see Section 8.4.1.2).
Since the entries into a row of the Jacobian matrix contain the partial derivatives
of feature position with respect to all state variables of an object, the fact that all
the entries are close to zero also carries information. It can be interpreted as an in-
dication that this feature does not depend (locally) on the state of the object; there-
fore, this feature should be discarded for a state update.
If all elements of a column of the Jacobian matrix are close to zero, this is an in-
dication that all features modeled do not depend on the state variable correspond-
ing to this column. Therefore, it does not make sense to try to improve the esti-
mated value of this state component, and one should not wonder that the
mathematical routine denies delivering good data. Estimation of this variable is not
possible under these conditions (for whatever reason), and this component should