Page 196 -

184 5 Neural Networks

5.6 Performance of Neural Networks

5.6.1 Error Measures

There are several indexes used to evaluate and compare the performance of neural
net solutions in classification problems. Most of them are related to the squared
error measure defined in (5-2). Considering the individual errors ei = zi - t,, of n
patterns, the following performance indexes are quite popular:

Error mean:

Absolute error mean:

Relative error:

Average squared error:

Root mean square (RMS) error: Em=&

Error standard deviation:

The average squared error is the squared error divided by the number of
patterns. Note its relation with formula (5-2a). A problem when comparing neural
net classifiers using E and ERMs is the dependence of these indexes on the threshold
values of the output activation functions. In some cases it may be advisable to
evaluate these indexes only for the misclassified cases, using the threshold value as
target value.
The standard deviation of the errors is useful for ranking the solution obtained.
In regression problems a good solution should have, besides a high correlation
between predicted and true values, a standard deviation of the errors, s,, at least
about an order of magnitude lower than the standard deviation of the target values,
s,, otherwise the regression solution is completely erratic. For instance, for the