Page 236 - Becoming Metric Wise

227
Indicators

propose the notion of merit, to be calculated as follows. We are given a
(homogeneous) group of N articles, for instance all publications by
research group G during the latest 5 years. Now, the evaluator must make
two choices: the first is the indicator to be used, e.g., average number of
received citations; total number of received citations, h-index, .. ., and
secondly a reference set. Next one determines the indicator value for
group G. In a following step one determines a random set of N articles
from the reference set and determines the indicator value for this random
set. This procedure is repeated 1000 or more times. The merit of G is its
percentile value among these 1000 or more indicator results. Clearly, a
good choice of reference sets makes it possible to compare the merit of
different research groups in a meaningful way.

7.11 TIME SERIES OF INDICATORS

In this Section, based on (Liu & Rousseau, 2008) we provide precise defi-
nitions of time series of indicators. Such definitions are necessary to avoid
possible confusion as different time series may lead to different conclu-
sions. We provide a general scheme for time series, using the terms
sequence and series interchangeably. We only discuss time series for which
terms—individual elements in the series—are obtained as sums (occasion-
ally sums consisting of one element). Time series for which terms are
averages can easily be derived from these.

7.11.1 The General Framework

We assume that the focus is on one set of articles. This set can be a scien-
tist’s research record, one specific journal, the set of all journals in one
particular field, or even all journals in a database. For this set we intend to
calculate a time series of indicators based on publication and citation data.
As a first example we consider how two simple time series can be
derived from a given one. Assume we know the numbers of citations
c 0 , c 1 , .. ., c 10 received by a set of articles published in the year Y 0 , where
c 0 is received in the year Y 0 , c 1 in the year Y 0 1 1 and so on. During
an investigation one may be interested in a subsequence such as
s 5 (c k ) k51,...,10 (we are not interested in the number of citations received
during the publication year). Similarly, we might be interested in the

P 1 P 2 P 10
cumulative sequence S 5 j50 j ; j50 j ; :::; j50 j .
c
c
c
Next we come to the main point of this section starting from a
general matrix of data, not a simple sequence. Consider a p-c matrix