Page 274 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

SYSTEM IDENTIFICATION 263

The minimalization of J(^ x(0), a) using the MATLAB function
x
fminsearch from the Optimization Toolbox gives the final result.
The numerical optimization is initiated with the parameters obtained
from particle filtering.
Figures 8.4(b), (c) and (d) show the estimated levels ^ x(i) obtained
x
with minimal J() for the three considered models.

8.1.4 Evaluation and model selection

The last step is the evaluation of the candidate models and the final
selection. For that purpose, we select a quality measure and evaluate and
compare the various models. Popular quality measures are the log-like-
lihood and the sum of squares of the residuals (the difference between
measurements and model-predicted measurements).
Models with a low order are preferable because the risk of over-
fitting is minimized and the estimators are less sensitive to estima-
tion errors of the parameters. Therefore, it might be beneficial to
consider not only the model with the best quality measure, but also
the models which score slightly less, but with a lower order. In order
to evaluate these models, other tests are useful. Ideally, the cross
correlation between the residuals and the input signal is zero. If not,
there is still part of the behaviour of the system that is not explained
by the model.

Example 8.4 Model selection for the hydraulic system
Qualitative evaluation of the three models using the log-likelihood as
a quality measure (or equivalently, the sum of squared Mahalanobis
distances) yields the following results:

Torricelli’s model: J ¼ 4875
Second order linear model: J ¼ 281140
Fourth order linear model: J ¼ 62595
Here, there is wide gap between the nonlinear Torricelli’s model and
the two linear models. Yet, the quality measure J ¼ 4875 is still too
large to ascribe it fully to the measurement noise. Without the model-
ling errors, the mean value of the sum of squared Mahalanobis dis-
tances (I ¼ 300, N ¼ 2) is 600. Inspection of Figure 8.4(b) reveals
that the feet of the curves cause the discrepancy. Perhaps the Torricelli
model should be extended with a small linear friction term.