Every researcher is affected by how scientific performance is measured. How should it be measured? Do we have the right data to do it?
Data is gaining importance in scientific evaluation, but it can be both used and abused to guide hiring decisions as well as national research policy. Very sophisticated success measures have been developed. However, do they measure what they claim and if so, how much of the tedious research efforts do they really reflect?
Modern research is also a team sport which is shaped by various social forces. Collaborations affect the success of individual scientists. Additionally, scientific success and careers depend on mobility across countries and research institutions. Indeed, academic careers span different spatial scales, e.g., countries, cities and institutions. To better understand these phenomena, new models and methods have emerged from the collaboration of experts from different fields, taking a complex systems perspective.
This half day-long Satellite Workshop is a forum to discuss and present these developments critically.
Session 1: Scientific Collaborations and Success
Modern science is increasingly carried out in collaborations, which affect scientific output and success. Notably, these collaborations are not centrally managed, but are formed by individual scientists. When considering all these collaboration decisions, an evolving collaboration network emerges. To understand the interplay between this network and scientific success, we will have one invited speaker (Dr. Riccardo Gallotti) and two contributed talks.
Dr. Giacomo Livan (University College London, UK)
Early coauthorship with top scientists predicts success in academic careers (RECORDING)
Dr. Riccardo Gallotti (Fondazione Bruno Kessler, IT)
Effects of homophily and academic reputation in the nomination and selection of Nobel laureates (RECORDING)
Session 2: Measuring Success in Science
The main metrics used to quantify scientific success are citation counts, h-index, and the impact factor of journals. Even though scientometricians are aware of the shortcomings and faultiness of these metrics, the scientific community still relies on them.
One major shortcoming is that different scientific fields have different citation cultures. For example, Fields medal winners might have fewer citations, a smaller h-index than a less accomplished scientist working in molecular biology. Another problem is the notorious cumulative advantage of scientists that have older publications: older publications have more time to attract citations. If we naively use citation data to develop success metrics, we are unable to capture the impact of scientists.
In our satellite, one invited talk from Prof. Ludo Waltmann and two other speakers will present recent advances to address these problems and quantify scientific success.
Shuqi Xu (University of Electronic Science and Technology of China, CN)
Unbiased evaluation of ranking metrics reveals consistent performance in citation data (RECORDING)
Session 3: Understanding Scientific Careers
To understand scientists' careers many factors must be considered, e.g., social networks, geographic distance, institutional prestige, mentorship. For example, the social network of a scientists can positively affect his/her visibility. On the other hand geographical and institutional constraints can limit access to career opportunities. Prof. Alexander Petersen and two contributed talks will address this issue.
Prof. Alexander Petersen (University of California - Merced, US)
Quantifying the impact of weak, strong, and super ties in scientific careers - (RECORDING)
If you are a researcher working on the topics listed in the 3 sessions we would like to hear from you. We welcome unpublished works for inclusion in a special issue of the journal Advances in Complex Systems (ACS).
Relevant topics include, but are not limited to:
Biases and strategies in scientific publishing
Ranking of universities, journals and papers
Biases in citation-based measures
Scientists' migration/mobility and their impact
Paper submission deadline: 15. April, 2021
Responsible Editors: Prof. Alexander Petersen, Dr. Luca Verginer, Dr. Giacomo Vaccario