Page 271 - Becoming Metric Wise

263
Research Evaluation

The Performance Ranking of Scientific Papers for World Universities,
also known as the NTU Ranking uses the number of publications, the
number of citations, the average number of citations, the h-index, the
number of highly cited articles and the number of articles published in
high-impact journals. The main data sources are the SCIE and SSCI.
The SIR provides a composite indicator, which combines 12 indica-
tors from three domains:
• Research (50%) includes 8 indicators related to output, impact and
collaboration. One of the indicators is the number of publications
published in journals that are ranked in the first quartile of their cate-
gory according to the SCImago Journal Ranking. Data in this domain
comes from Scopus.
• Innovation (30%) accounts for the number and proportion of publica-
tion output that is cited in patents. This part is based on the
PATSTAT database.
• Societal impact (20%) is operationalized using two web-based indica-
tors: the number of web pages in Google and the number of inbound
links to the university as counted by Ahrefs (https://ahrefs.com/).
Finally, the Webometrics Ranking of World Universities uses:
• Total number of web pages including rich files such as PDF (10%).
• Number of external inlinks, as counted by Ahrefs and Majestic (the
top 20 linking domains are excluded) (50%).
• GS Citations institutional profile (10%).
• Top 10% most cited publications per discipline based on SCImago
data (30%).
It should be mentioned that none of the leading international lists
takes the quality of education into account as a direct measure. Most indi-
cators used for international ranking try to measure the quality of research
and the prestige of the university.

8.4.4 Is it Really Possible to Measure “The Quality of
Universities”? (Buela-Casal et al., 2007)

Once these lists became known and started attracting the attention of
scientists, research policy makers and even newspapers, discussions
emerged about the feasibility of such lists: do there really exist indicators
that lead to a meaningful ranking of universities?
Clearly, no single indicator can lead to a ranking that takes all aspects
of university education and research into account. Hence a better ques-
tion might be: which indicators may contribute to an accurate