Page 211 - Becoming Metric Wise

202 Becoming Metric-Wise

often used, be it inappropriately, as a proxy for journal quality; similarly,
the number of downloads is taken as a proxy for readership.

7.1.3 General Terminology: Validity, Fairness,
Usefulness and Reliability
The term validity refers to whether or not the indicator measures what it
claims to measure. An indicator is said to be fair if it is free of any kind of
bias. It is useful if it provides essential information.
Finally, we consider reliability as a synonym of reproducibility.
Calculation of an indicator in two similar circumstances should not lead
to very different values, unless there is a clear explanation and then the
circumstances were actually not similar.
Although fairness is a natural requirement, it is in practice probably
impossible that an indicator is fair to everyone and in all circumstances.

7.1.4 Normalization
A normalized indicator is a ratio of an absolute indicator divided by a
normalization parameter. This normalization parameter is either the
maximum possible value, or an expected value. Normalization is usually
applied to make results comparable over different, but similarly distrib-
uted, data sets. Normalization can be done in many ways. If indicator
values are normally distributed, then z-values are used as normalized
values. Unfortunately, few data sets or indicators in informetrics are
normally distributed. If data are not normally distributed, but still more
or less symmetric with respect to the median (which in this case coincides
with the mean) one may divide by the mean or the median.
Of course, one may wonder why anyone would want to make the
kind of comparisons alluded to above. Certainly such comparisons should
not be made on an individual level i.e., one should not try to answer if
geographer X is a better scientist than philosopher Y. Yet, we think that
for a university leader it is a natural question to ask if the department of
computer science is, compared with all other computer science depart-
ments in the country or in the world, better than the department of
nanotechnology. By using an indicator and an appropriate normalization
one may obtain an acceptable answer. Yet, one would not get an answer
to the question which department performs the highest intellectual
achievements, or the most useful results for society.