Page 58 - Becoming Metric Wise

48 Becoming Metric-Wise

(and hence a N decision in the other 50%) when he or she receives YN
advice. The second one is the clear-cut case. It is further assumed that N
and Y have an equal chance of occurring. It is shown that a decision is
reversed in 37.5% of the cases in the 50-50 situation, and in 25% of the
cases when the clear-cut rule is applied. Such changes are not to be
ignored and illustrate the responsibility taken by an EIC. Yet, any choice
of referees entails other possible outcomes. This is an inherent property of
any decision scheme under variable conditions.
Luckily, Weller (2001) tells us that YY and NN each occur more than
YN (in whatever order), and that NN occurs more than YY. This fact is
confirmed in Bornmann and Daniel (2009b). This is good news for the edi-
torial process: reviewers easily recognize weak or erroneous submissions.
We note that more than 25 years ago, Hargens (1988) already devel-
oped a model for the manuscript-refereeing process. This model included
two variables: the journal’s decision structure and the correlation between
referees’ evaluations of manuscripts. The rejection rates (RRs) (see
Subsection 3.1.7 for a definition) obtained in his models corresponded to
existing ones and hence he suggested that these variations suffice to
explain observed variations in RRs. He concluded that space shortages
cannot explain disciplinary variations in journal RRs. Finally, he pro-
posed that actual RRs can be used as an indicator for consensus in a field.
Perakakis et al. (2010) propose a very experimental article evaluation
system, in which, for instance, journals may compete to get an article pub-
lished in their journal when it has received many good comments in an
author-guided peer review system i.e., authors invite colleagues to write
comments on their articles, placed in a repository. As authors may update
their articles as many times as they want before publishing in a journal this
would reduce the number of publications. Perceived importance of an arti-
cle would be determined by actual comments and not by external factors,
such as the impact factor of the journal in which an article is published.
Most scientists would agree with Mandavilli (2011) who wrote: “The
peer review process isn’t very good but there really isn’t anything that’s
better.” Yet, some like Macdonald (2015) clearly state that, especially for
top journals, the peer review system cannot cope anymore. The reason
for this statement is that these journals are flooded by submissions so that
the large majority of them are desk-rejected (not rejected by peers). This
behavior clearly favors established scholars working at top universities.
An overview of the peer review system can be found in Bornmann
(2011).