Page 202 - Becoming Metric Wise
P. 202
192 Becoming Metric-Wise
A comprehensive list of the advantages and disadvantages of the jour-
nal ranking method based on citation impact measures is provided in
Table II of (Serenko & Bontis, 2013) which we republish here with some
adaptions.
Advantages of the citation indicator method for ranking journals:
a. The method is more objective than using peer review or experts’
opinions as it involves the actual use of a large group of scientists.
b. There exist many citation based indicators, which may be com-
bined in the most appropriate manner.
c. Over the years citation based indicators have obtained a wide
acceptance for journal rankings.
Disadvantages of the citation indicator method for ranking journals:
a. Citation behavior differs between fields and hence comparison
between fields is not valid. Moreover, multidisciplinary journals
cannot really be included in journal rankings.
b. Some indicators, such as the JIF (being an average), are highly
influenced by the skewness of citation data.
c. Citations may be manipulated by journal editors.
d. Citation impact is generally low for new or niche journals.
e. Retracted articles may be included in citation counts.
f. All databases contain errors.
g. If a journal is not included in the database used for the ranking
exercise it cannot be ranked.
h. Path dependency (Truex et al., 2008). This means that authors have
the tendency to use and re-use journals studied and ranked by pre-
decessors. This is even true for the WoS as a whole, a point
discussed in (Rousseau & Spinak, 1996).
i. Most citation indicators do not make a distinction between citing
sources (but eigenvector related methods do).
j. Different citation indicators or different weightings in composite
measures may lead to different rankings.
Most of these disadvantages apply to any use of citation measures, not
just for journal rankings.
6.18.3 Combination of the Two Approaches
When applying this approach, journal quality assessments and rankings are
generated through a combination of the two approaches listed above.
Often initial journal rankings are generated based on bibliometric