Page 31 - Becoming Metric Wise

20 Becoming Metric-Wise

performs the same investigation, using the same experiments, materials, and
tools as the original or one re-analyzes the original data. A validation study
has the same purpose as the original, but one may use different data and do
similar experiments with the purpose of coming to the same conclusions.
Replication research is not popular at all as it is not original and hence
rarely leads to recognition by peers. For an inquisitive mind, it might also
be less interesting. Yet, double-checking is an aspect of the scientific
method as described by Popper. Normal science as we know it produces
new findings. The more sensational, say unexpected or difficult to find,
the higher the response inside and even outside the author’s field. Yet,
many publications are never or rarely cited and hence their content is
never thoroughly checked after publication. As long as uncited is the
same as unused no harm is done, except in an indirect way as clearly a lot
of money may have been wasted. Unfortunately, unreplicable research,
especially in the life sciences, is not rare at all (Begley & Ellis, 2012).
For instance, scientists at Bayer reported that among 67 projects only 14
matched up with the published findings (Mullard, 2011).
A large-scale replication effort in psychology sought to replicate 100
studies published in reputable psychology journals in 2008 (Open Science
Collaboration, 2015). While 97 of the original 100 studies reported statisti-
cally significant results, only 36 of the replications yielded significant results.
Overall, many replications had far weaker results than the original studies.
Russell (2013) rightly states that reproducibility separates science from
anecdote. Consequently, he argues that funding agencies should tie grant
funding to replication attempts. He realizes that such a proposal may
encounter objections, but provides answers to them. We refer the reader
to the original article for more details.
We should mention also that scientometrics struggles with irreproduc-
ibility. Gla ¨nzel (1996) already warned that scientometric studies should be
reproducible, and stressed that this can only happen if all sources, proce-
dures, and techniques are properly documented. He further mentions that
validation studies should be designed for detecting systematic errors and
estimating random ones. Methodological validation studies must ensure
that scientometricians really measure what is intended to be measured.

2.2.3 Shneider’s Typology of Scientists
Shneider (2009) distinguishes four stages in scientific research and corre-
spondingly four types of scientists based on the type of work they prefer.