Page 259 - Becoming Metric Wise

251
Research Evaluation

accessibility is as important as quality as a factor in the selection of an infor-
mation source. Soper (1976) found that the largest proportion of docu-
ments cited in authors’ recent papers was located in personal collections, a
smaller proportion was located in libraries in departments and institu-
tions to which respondents belonged, and the smallest proportion was
located in other cities and countries. This investigation clearly predates
the Internet and the availability of journal collections on one’s com-
puter. Yet, we are convinced that availability, be it with a somewhat
different meaning, still plays a major role in the selection of reading
and citing material.

Assumption 4: The fact that the cited document is related in content to
the citing document is supported by the use of citation databases as infor-
mation retrieval tools. Successful applications of the notions of biblio-
graphic coupling and cocitation are further testimony that this fourth
assumption is often met in practice.

Overall, it seems likely that none of these four assumptions holds abso-
lutely true in every single case, yet, as already mentioned, there is evidence
that these assumptions are often met in practice. A typical case in which
the cited document is not directly related to the actual contents of the cit-
ing article is a reference to a statistical test used in a medical article.
These four assumptions form the base for all evaluation studies using
citation counts.
An older, but still interesting essay on the use of citations for the eval-
uation of scientists was written by Edward (1992). Edward, a Canadian
professor in organic chemistry, takes a look at the most-cited scientists
and most-cited papers in his field mostly during the 1960s and 1970s. He
gives comments on the obliteration effect, notes that most articles are not
or rarely cited and that often methods papers, such as the famous paper
by Lowry et al. (1951), are more cited than theoretical breakthroughs. He
concludes that citation counts give some indication of scientific excel-
lence if one considers a whole corpus of papers, rather than individual
papers. This idea can also be found in White (2001), where the author
writes: “Within its limits, citation analysis works.” Edward further
observes that, besides for evaluative purposes, the Science Citation Index
can be used to study the genealogy of ideas. By checking why papers are
cited one may learn how others have developed one’s ideas and by this,
learn to become more alert oneself.