Combating Online Scientific Misinformation

The drastic shift towards digital communication in our mediasphere has caused a profound change in the production and consumption of information, which in turn has substantial implications on the social and political landscape. Misinformation, as a side effect of mass information diffusion, has become a fundamental problem for governments, platforms, and the general public in light of critical events such as elections, pandemics, and wars. In this thesis, we focus on the problem of online scientific misinformation. As a starting point, we survey the evolution of misinformation and present the main characteristics and approaches against it, framing the high-level positioning of this thesis with respect to related literature. Then, we discuss three major scientific contributions of this thesis: our methods for combating claim-based, article-based, and source-based scientific misinformation.For combating claim-based scientific misinformation, we introduce SciClops, a method for detecting and contextualizing scientific claims for assisting manual fact-checking. Our method involves three steps: (1) extracting scientific claims using a domain-specific, fine-tuned transformer model, (2) clustering similar claims together with related scientific literature using a method that exploits their content and the connections among them, and (3) highlighting check-worthy claims broadcasted by popular yet unreliable sources. Our experiments show that SciClops effectively assists non-expert fact-checkers in verifying complex scientific claims, facilitating them to outperform commercial fact-checking systems.For combating article-based scientific misinformation, we introduce SciLens, a method for evaluating the quality of scientific news articles. Our method involves a series of quality indicators for news articles that derive from: (1) their content, including the use of attributed quotes, (2) their scientific context, including their semantic similarity and web proximity to the scientific literature, and (3) their social context, including their social media reach and stance. Our experiments show that these indicators help non-experts evaluate the quality of articles more accurately compared to non-experts that do not have access to these indicators. Moreover, SciLens can also produce completely automated quality scores for articles, which agree more with expert evaluators than manual evaluations done by non-experts.For combating source-based scientific misinformation, we introduce SciLander, a method for learning representations of news sources reporting on scientific topics. Our method involves heterogeneous source indicators that capture: (1) the copying of news stories between sources, (2) the semantic shift of terms across sources, (3) the usage of jargon, and (4) the stance towards specific citations. SciLander uses these indicators as signals of source agreement to train unsupervised source embeddings. Our experiments show that the learned source representations outperform state-of-the-art baselines on the task of news veracity classification while encoding information about the reliability, political leaning, and partisanship bias of these sources.In the last part of this thesis, we introduce NewsTeller, a real-time news analytics platform that runs operationally, handling daily thousands of news articles, social media reactions, and references.

Evaluating and Interpreting Deep Convolutional Neural Networks via Non-negative Matrix Factorization

Edo Collins

With ever greater computational resources and more accessible software, deep neural networks have become ubiquitous across industry and academia. Their remarkable ability to generalize to new samples defies the conventional view, which holds that complex, over-parameterized networks would be prone to overfitting. This apparent discrepancy is exacerbated by our inability to inspect and interpret the high-dimensional, non-linear, latent representations they learn, which has led many to refer to neural networks as black-boxes''. The Law of Parsimony states that simpler solutions are more likely to be correct than complex ones''. Since they perform quite well in practice, a natural question to ask, then, is in what way are neural networks simple? We propose that compression is the answer. Since good generalization requires invariance to irrelevant variations in the input, it is necessary for a network to discard this irrelevant information. As a result, semantically similar samples are mapped to similar representations in neural network deep feature space, where they form simple, low-dimensional structures. Conversely, a network that overfits relies on memorizing individual samples. Such a network cannot discard information as easily. In this thesis we characterize the difference between such networks using the non-negative rank of activation matrices. Relying on the non-negativity of rectified-linear units, the non-negative rank is the smallest number that admits an exact non-negative matrix factorization. We derive an upper bound on the amount of memorization in terms of the non-negative rank, and show it is a natural complexity measure for rectified-linear units. With a focus on deep convolutional neural networks trained to perform object recognition, we show that the two non-negative factors derived from deep network layers decompose the information held therein in an interpretable way. The first of these factors provides heatmaps which highlight similarly encoded regions within an input image or image set. We find that these networks learn to detect semantic parts and form a hierarchy, such that parts are further broken down into sub-parts. We quantitatively evaluate the semantic quality of these heatmaps by using them to perform semantic co-segmentation and co-localization. In spite of the convolutional network we use being trained solely with image-level labels, we achieve results comparable or better than domain-specific state-of-the-art methods for these tasks. The second non-negative factor provides a bag-of-concepts representation for an image or image set. We use this representation to derive global image descriptors for images in a large collection. With these descriptors in hand, we perform two variations content-based image retrieval, i.e. reverse image search. Using information from one of the non-negative matrix factors we obtain descriptors which are suitable for finding semantically related images, i.e., belonging to the same semantic category as the query image. Combining information from both non-negative factors, however, yields descriptors that are suitable for finding other images of the specific instance depicted in the query image, where we again achieve state-of-the-art performance.

EPFL2019

SciLens: Evaluating the Quality of Scientific News Articles Using Social Media and Scientific Literature Indicators

Karl Aberer, Panagiotis Smeros

This paper describes, develops, and validates SciLens, a method to evaluate the quality of scientific news articles. The starting point for our work are structured methodologies that define a series of quality aspects for manually evaluating news. Based on these aspects, we describe a series of indicators of news quality. According to our experiments, these indicators help non-experts evaluate more accurately the quality of a scientific news article, compared to non-experts that do not have access to these indicators. Furthermore, SciLens can also be used to produce a completely automated quality score for an article, which agrees more with expert evaluators than manual evaluations done by non-experts. One of the main elements of SciLens is the focus on both content and context of articles, where context is provided by (1) explicit and implicit references on the article to scientific literature, and (2) reactions in social media referencing the article. We show that both contextual elements can be valuable sources of information for determining article quality. The validation of SciLens, done through a combination of expert and non-expert annotation, demonstrates its effectiveness for both semi-automatic and automatic quality evaluation of scientific news.

ASSOC COMPUTING MACHINERY2019

Digital Governance Drivers and Challenges for Resilience in AI-Driven Innovation of Public Sector

Gianluca Carlo Misuraca, Gianluigi Viscusi

Digitalization is not anymore an emergent phenomenon but the actual shape of everyday life interactions and transactions (Degryse 2016; Tilson et al. 2010; Yoo 2013). Compared to the private sector, where businesses have deployed initiatives to change their infrastructure, governance, and business models to create and exploit value from their digital assets, the public sector is still tied up to a consideration of technology as something separated to public sector reform and policy making. Accordingly, it is still preeminent in the public sector the focus on what can be considered the e-government rhetoric legacy, namely the provision of information and communication technology (ICT) enabled services mainly involving the public administration alone and the translation of administrative procedures in digital format. This view is still present in most countries having officially claimed the adoption of an open government stance towards their initiatives: the result is, in the best case, an efficient public administration, yet still neither inclusive nor fully open, in terms of transparency and accountability (Misuraca and Viscusi 2014). Moreover, the rhetoric is now encompassing the debate on the use of artificial intelligence (AI) solutions for innovating public sector services and decision-making (Androutsopoulou et al. 2019; van Noordt and Misuraca 2019; Sun and Medaglia 2019) with a new emphasis on the need for a governance of AI and not by AI (Sun and Medaglia 2019, p. 378) that resonates the former claims about governance of ICTs vs. governance by ICTs (Misuraca and Viscusi 2015). Thus, although some kind of public value can be gained (Benington 2011; Benington and Moore 2011), the switch to social value from co-production (Alford 2011; Cordella et al. 2018) by involving citizens and external actors is still an ongoing challenge for public administrations willing to have a role in social innovation through the appropriate exploitation of the unprecedented amount of data available from information production, inside and outside public sector information systems (Viscusi & Batini, 2016). This is particularly relevant when thinking about the use of crowdsourcing for deliberation, regulatory reviews, and policy initiatives (Aitamurto and Landemore 2016; Brabham 2009; Lodge and Wegrich 2015), the development of open innovation in the public sector (Viscusi et al. 2015) as well as the emergent challenges of using, e.g., machine learning for deciding on welfare issues (Reisman et al. 2018; West et al. 2019). Taking these issues into account, this article aims to contribute to the theoretical debate on the relationships between digital governance and social innovation, and their impact on policy making for creating and capturing value from effective solutions addressing societal challenges. In particular, we question the drivers and challenges specifically considering growing number of national strategies for innovation driven by artificial intelligence (AI) and the consequent wave of investments. To this end a conceptual model is presented aiming, on the one hand, to connect key dimensions and value drivers of digital governance for Pre-ICIS Workshop on E-Government, Munich, Germany 2019 1 Digital Governance in AI-Driven Innovation of Public Sector social innovation and complex systems methods for policy making; on the other hand, to position AI national initiatives with regard to welfare state initiatives. The model is expected to be applied to the analysis of case studied of AI-driven innovation initiatives in Europe to assess their impact on social innovation. To this end, first a conceptual model is presented aiming, on the one hand, to connect key dimensions and value drivers of digital governance for social innovation (Misuraca, Pasi, et al. 2018; Misuraca and Viscusi 2015) and complex systems methods for policy making formerly introduced by one of the authors in a framework called i-Frame (Misuraca, Geppert, et al. 2018); on the other hand, to position AI national initiatives with regard to welfare state initiatives for an appropriate analysis of their premises and effects. The paper concludes outlining future research directions and policy implications.