Page 201 - Becoming Metric Wise

191
Journal Citation Analysis

c. Identity concerns: if respondents fear that their identity would
become known they might follow the general opinion; similarly
they might favor journals of which they are an editorial board
member.
d. Rater fatigue if they have to rate long lists of journals.
e. Path dependency: lists to be ranked are built upon older lists so that
some journals stay on these lists for ever (this also holds for the cita-
tion based approach).
f. Underrepresentation of practitioners and journals aiming at this
group.
g. Order bias: the order in which journals are presented to raters may
have a confounding effect on their rankings.
Most of these disadvantages can be avoided by taking the necessary
precautions when soliciting expert opinions.

6.18.2 Citation Based Approach

The citation based approach is the most-used approach to produce journal
rankings. Moreover, a variety of citation based indicators have been applied
for journal evaluations. Several studies employed total citations (Liebowitz
& Palmer, 1984) or average citations (Doyle & Arthurs, 1995) in specified
evaluation windows. Moreover, one sometimes emphasizes selected cita-
tions. Tahai and Meyer (1999), e.g., produced the SMJ99 management
journal ranking based on citations that were received from the top 17 jour-
nals in the field using a 2-year window. Among these citation based indica-
tors, the JIF received by far the most attention. The JIF assigns identical
weights to all received citations, regardless of the prestige of citing sources
(papers, authors, journals or conferences). The concept of weighted cita-
tions (Kochen, 1974; Pinski & Narin, 1976) is, after its conversion to
Google’s PageRank, applied for journals as the Eigenfactor (Bergstrom,
2007), the SJR index (http://www.scimagojr.com/journalrank.php)or, in
a normalized form, as the AIS and used, as such, for journal rankings.
The well-known h-index (Hirsch, 2005) has also been used in the
assessment of academic journals as suggested by Braun et al. (2005, 2006).
These authors moreover claimed that the h-index is a more robust index
than the JIF. However, the h-index only counts the publications in the
h-core and ignores all the rest. Also the g-index (Egghe, 2006a,b,c) has
been applied for journal rankings (Bontis & Serenko, 2009; Serenko &
Bontis, 2009).