Page 258 - Becoming Metric Wise

250 Becoming Metric-Wise

8.1.4 Use of Citations for Research Evaluation
Basic Assumptions for the use of Citations in Evaluations
(Smith, 1981)
Linda Smith stated the following four basic assumptions:
1. Citation of a document implies use of that document by the citing
author.
2. Citations are positive and reflect merit quality, significance, impact.
3. The best possible works are cited.
4. If a document is cited, it is related in content with the citing
document.
Some comments on these four assumptions.

Assumption 1: According to Smith (1981) assumption one has two
parts:
1. Authors refer to all, or at least to the most important, documents used
during the preparation of their work.
2. All documents listed are used, i.e., authors refer to a document only if
its contents have contributed to their work. Failure to meet this cri-
terion for good citation behavior leads to “sins of omission and commis-
sion.” It is, however, evident that what is cited is only a small percentage
of what is read or what has in some way influenced the author.
Moreover, a document is often cited because of the use of a small part of
it, for instance one particular observation, theorem, or result.

Assumption 2: The underlying assumption in the use of citation counts
as a kind of quality indicators is that there is a high positive correlation
between the number of received citations and the quality of a document.
Clearly, if most citations were made on the base of Thorne’s list (Thorne,
1977), see Subsection 5.3.2, conclusions stemming from citation analysis
would be invalid. Moreover, quality, impact and visibility are different
notions and in practical applications a careful distinction should be made.
Nevertheless, this second assumption has been tested and has found some
support, see e.g., De Bellis (2009).

Assumption 3: If one assumes that citations are made to the best possible
works, then one must imagine that authors sift through all the possible
documents that could be cited and carefully select those judged the best.
However, studies on referencing behavior suggested that, for instance