Research Methodologies Extras

Impact: can be potential impact (valued by reviewers and journals and conferences) or real impact (which can be measured only after some time).

How to estimate impact?

Can we use one number metrics?

• Number of citations: (bibliometric) papers with high number of citations are considered to have high impact.

Is there a rationale behind how people cite? Is it uniform? No, it is very variable and depends on the field (different disciplines have different number of average citations). It is also prone to un-ethical practices (self-citations, citation rings, etc) for inflation. Also, the number of citations grows over time and then slows down, and citations may not always be positive (negative citations).

Number of citations is used to calculate the impact factor of journals (reputation of the publisher), which causes highly skewed distributions, with a few papers having a lot of citations and most papers having very few (average is often misleading).

• H-index: (bibliometric) maximum H such that a given author has published H papers that have each been cited at least H times (goodness of the author). Should not be used for ranking, especially inter-disciplinary, for the reasons above.

• Number of downloads per artifact: (altmetric) number of times a paper has been downloaded.

• Number of licences: (altmetric) number of times a paper has been licensed.

Moral: don't use one number metrics for determining the impact of papers or authors.

iCST: conference ranking (list of conferences and their ranking).