Page 202 - Becoming Metric Wise
P. 202

192   Becoming Metric-Wise


             A comprehensive list of the advantages and disadvantages of the jour-
          nal ranking method based on citation impact measures is provided in
          Table II of (Serenko & Bontis, 2013) which we republish here with some
          adaptions.
             Advantages of the citation indicator method for ranking journals:
             a. The method is more objective than using peer review or experts’
                opinions as it involves the actual use of a large group of scientists.
             b. There exist many citation based indicators, which may be com-
                bined in the most appropriate manner.
             c. Over the years citation based indicators have obtained a wide
                acceptance for journal rankings.
             Disadvantages of the citation indicator method for ranking journals:
             a. Citation behavior differs between fields and hence comparison
                between fields is not valid. Moreover, multidisciplinary journals
                cannot really be included in journal rankings.
             b. Some indicators, such as the JIF (being an average), are highly
                influenced by the skewness of citation data.
             c. Citations may be manipulated by journal editors.
             d. Citation impact is generally low for new or niche journals.
             e. Retracted articles may be included in citation counts.
             f. All databases contain errors.
             g. If a journal is not included in the database used for the ranking
                exercise it cannot be ranked.
             h. Path dependency (Truex et al., 2008). This means that authors have
                the tendency to use and re-use journals studied and ranked by pre-
                decessors. This is even true for the WoS as a whole, a point
                discussed in (Rousseau & Spinak, 1996).
             i. Most citation indicators do not make a distinction between citing
                sources (but eigenvector related methods do).
             j. Different citation indicators or different weightings in composite
                measures may lead to different rankings.
             Most of these disadvantages apply to any use of citation measures, not
          just for journal rankings.


          6.18.3 Combination of the Two Approaches
          When applying this approach, journal quality assessments and rankings are
          generated through a combination of the two approaches listed above.
          Often initial journal rankings are generated based on bibliometric
   197   198   199   200   201   202   203   204   205   206   207