Scientometrics | EPFL Graph Search

Scientometrics is the field of study which concerns itself with measuring and analysing scholarly literature. Scientometrics is a sub-field of informetrics. Major research issues include the measurement of the impact of research papers and academic journals, the understanding of scientific citations, and the use of such measurements in policy and management contexts. In practice there is a significant overlap between scientometrics and other scientific fields such as information systems, information science, science of science policy, sociology of science, and metascience. Critics have argued that over-reliance on scientometrics has created a system of perverse incentives, producing a publish or perish environment that leads to low-quality research. Modern scientometrics is mostly based on the work of Derek J. de Solla Price and Eugene Garfield. The latter created the Science Citation Index and founded the Institute for Scientific Information which is heavily used for scientometric analysis. A dedicated academic journal, Scientometrics, was established in 1978. The industrialization of science increased the number of publications and research outcomes and the rise of the computers allowed effective analysis of this data. While the sociology of science focused on the behavior of scientists, scientometrics focused on the analysis of publications. Accordingly, scientometrics is also referred to as the scientific and empirical study of science and its outcomes. The International Society for Scientometrics and Informetrics founded in 1993 is an association of professionals in the field. Later, around the turn of the century, evaluation and ranking of scientists and institutions came more into the spotlights. Based on bibliometric analysis of scientific publications and citations, the Academic Ranking of World Universities ("Shanghai ranking") was first published in 2004 by the Shanghai Jiao Tong University. Impact factors became an important tool to choose between different journals.

The organisation of science: topics, incentives and funding.

Omar Ballester Gonzalez

Ever since the links between the development of new technologies and economic growth became evident, researchers have attempted to study how the creation of knowledge fosters progress. If pushing the frontier of knowledge has an impact on progress and well-being, it is essential to pursue some form of science policy. Policymakers rely on the scholarly work of researchers in order to understand the likely impact of new policies and investments, and evaluate the state of the art in science and innovation policy. Therefore, the work of social scientists, economists of science and information scientists, among others, is vital to the characterisation, understanding and management of science. In recent years, the availability and quality of science data (including bibliographic data and metadata, funding, relational databases, ontologies and classifications) has boosted the empirical work in the depiction of the organisational structure of science. In turn, policy analysis has been able to accurately identify many unsuspected effects of past investments and policy decisions both at the macro and micro level. Using topic models, we develop a novel method for evaluating the robustness of different text-to-text similarity models. Employing that procedure, we find that the neural-network-based paragraph embeddings approach seems capable of providing statistically robust estimates of document--document similarities. Finding methods to estimate the similarity between individual publications is an area of long-standing interest in the information science and scientometrics communities. These techniques enable researchers to build indicators and classification methods based on the analysis of large text corpora. We show that the most widely used techniques suffer from inconsistencies upon retraining, and provide a procedure to evaluate and compare the quality of different methods, regardless of the data. Next, we present a game-theoretic model of rewards to scientific contributions. Our model of science may help explain the resulting social organisation of science from a simple social dilemma model. We model a researcher's payoff as a common-pool resource game, intrinsically connecting the appropriability of scientific output to a scientist's optimal strategy. This simple model of reward allocation sheds new light on a variety of behaviours that have been observed amongst researchers. Finally, we propose an empirical analysis of the relationship between basic knowledge generation and spillovers to innovation. Using the United States' 2001 ban on federal funding of human embryonic stem cells (hESC), we disentangle the effect that policy had on downstream innovation. We employ recently developed data on patent-to-scientific-article citations to measure the spillovers, and we characterise the causal impact of the policy on subsequent innovation with a difference-in-differences estimator. Our estimates suggest that in the years following the policy, scholarly publications subject to the ban received 65 to 80 per-cent fewer patent citations than the control group. We then apply topic modelling techniques to examine changes in the direction of science. In particular we build a topic-variety metric. Our findings show that variety decreased in the aftermath of the policy. Our results suggest that even the most modest policy changes have a profound impact on downstream innovation and the advancement at the frontier.

EPFL2020

Evolution of Topics and Novelty in Science

Orion B Penner

Methods of estimating the similarity between individual publications is an area of long-standing interest in the scientometrics community. Traditional methods have generally relied on references and other metadata, while text mining approaches based on title and abstract text have appeared more frequently in recent years. In principle, Topic Models have great potential in this domain. But in practice, they are often difficult to successfully employ and, in particular, are notoriously inconsistent as latent space dimension grows. That is, running the same model, with the same parameters, on the same data, but with a different random seed produces radically different similarity estimates as the number of topics increase. In this manuscript we develop a simple, but novel, methodology for evaluating the robustness of topic models. Employing that methodology, we find that the neural network based Doc2Vec approach seems capable of providing (statistically) robust estimates of document-document similarities, even for topic spaces far larger than prudent for the most common topic model approach: Latent Dirichlet Allocation. As this is a work in progress, we do not venture deeply into the question of whether these estimates also reflect reality, but do provide some preliminary evidence and future directions for those efforts.

INT SOC SCIENTOMETRICS & INFORMETRICS-ISSI2019

The organisation of science: topics, incentives and funding.

Omar Ballester Gonzalez

EPFL2020