Page 255 - Becoming Metric Wise

CHAPTER 8

Research Evaluation

In 1985 Garfield (1985) wrote:
By basing funding or even scholarly tenure and hiring decisions on quantita-
tive bibliometric data, there is always the potential for making two serious
mistakes: one, in believing that mere publication or citation counting is
equivalent to citation analysis; and, two, in believing that citation analysis,
even when carefully performed by experts, is sufficient by itself to ensure
objectivity.
This statement clearly determines the boundaries of this chapter, but
we recall from Subsection 7.1.3 that indicators must be valid, fair, useful,
and reliable.
Yet, since the time that Garfield wrote the above statement the world
has become an audit society. Science has not escaped this evolution,
maybe even on the contrary. The idea of a stable career in science has
become blurred: scientists are in perpetual transition, have to prove their
capability repeatedly, and, with the idea of tenure being questioned, are
all the time at risk to be eliminated from the system or suffer burn-out
(de Meis et al., 2003).
In this chapter on research evaluation, we discuss aspects related to
measuring research impact, focusing on universities, research groups,
and countries as main units. Less attention is given to individuals, while
journals and their evaluation procedures and indicators have been dis-
cussed in Chapter 6, Journal Citation Analysis. Put differently, we focus
on the macro (countries, regions, universities) and meso (departments,
research groups) level and pay less attention to the micro (individuals)
level. For completeness’ sake we mention that it is also possible to per-
form studies on the nano level, i.e., the level of articles or parts
thereof.
A notion that we will not try to define but that always hoovers around
when talking about evaluations is the notion of “quality.” We just recall
Subsection 3.1.7 and mention the obvious, namely that quality is a rela-
tive concept determined by context, such as the field and goals of
research.

Becoming Metric-Wise © 2018 Elsevier Ltd.
DOI: http://dx.doi.org/10.1016/B978-0-08-102474-4.00008-X All rights reserved. 247