Online Dating Quantification Practices: A Human-Machine Learning Process

The phenomenon of online dating via web and mobile phone applications involves several actors: graphical interfaces, developers, algorithmic systems for user matching, and users. These actors have been studied in parallel by the social sciences and by computer science. While computer science research focuses on the design of more efficient recommendation systems and on the analysis of behavioral patterns, research in the humanities and social sciences is interested in the effects of platforms and their computational practices on the formation of affective relationships.This thesis explores how the mediation relations between the different actors of dating help to commonly establish the human-machine communication. Every actor learns in his-her own way a new conception of dating, mirroring the machineâs algorithmic efficiency. The study of these new dynamics of communication is bound to become key to software and algorithmic studies, for which the human-machine relation-ship has hitherto remained invisible in the study of dating platforms, or incomplete when it comes to analyzing developers' practices or algorithmic systems without taking into account the final uses of the platforms. Using a combination of qualitative and quantitative concepts and methodologies from sociology and data science, this thesis contributes to the comparative analysis of four actors who have a profound impact on the experience of online dating: graphical interfaces, developers, algorithmic matchmaking systems, and users. This is a study of more than 320 variables collected in more than 20 dating applications, both English and French, used in Switzerland and around the world. These variables mediate between the design of profile recommendation systems and their final use for dating. A case study is conducted on the massively-adopted dating application Tinder, which acts as an industry innovator, capable of establishing design conventions (e.g., swiping). This thesis further investigates, on the one hand, the development practices of nine founders and developers working in the industry, and on the other hand, the practices of 40 users of 26 different dating platforms. The results show that the interaction with dating platforms involves a collective learning process of experimentation that allows for communication between humans and machines. This fast-paced communication favors trial-and-error practices: actors either learn to adapt to the machine, or on the contrary to overcome it by accepting the mutual tolerance stemming from the uncertainty of actions. Learning is possible through the supposed social conventions of the actors, some of which are made explicit and quantifiable through graphical interfaces. This thesis contributes to the described research field with a theoretical framework and empirical data which can guide future research on online sociability at the intersection of computer and social sciences, like in the digital humanities.

The practical confrontation of engineers with a new design endeavour: The case of digital humanties

Frédéric Kaplan, Dominique Vinck

This chapter shows some of the practices of engineers use when they are confronted to completely new situations, when they enter into an emerging field where methods and paradigms are not yet stabilized. Following the engineers here would help to shed light on their practices when they are confronted to new fields and new interlocutors. This is the case for engineers and computer scientists who engage themselves with human and social sciences to imagine, design, develop and implement digital humanities (DH) with specific hardware, software and infrastructure.

CRC Press2013

Mining Conversational Social Video

Joan Isaac Biel Tres

The ubiquity of social media in our daily life, the intense user participation, and the explo- sion of multimedia content have generated an extraordinary interest from computer and social scientists to investigate the traces left by users to understand human behavior online. From this perspective, YouTube can be seen as the largest collection of audiovisual human behavioral data, among which conversational video blogs (vlogs) are one of the basic formats. Conversational vlogs have evolved from the initial "chat from your bedroom" format to a rich form of expression and communication that is expanding to innovative applications where a more natural and engaging way of reaching audiences is either necessary or might be beneficial. This video genre, available online in huge quantities, is a unique scenario for the study and characterization of complex human behavior in social media, that contrarily to social networks, text blogs, and microblogs, has remained unexplored so far. The automatic behavioral understanding of conversational vlogs is a new domain for multimedia research. In short, the goal of our research is the understanding of the processes involved in this social media type, based not only on the verbal channel – what is said – but also on the nonverbal channel – how it is said. The nonverbal channel includes prosody, gaze, facial expression, posture, gesture, etc. and has been studied in depth in the field of nonverbal communication. While the study of vlogging contributes to user behavior research in social media, it also adds to a larger research agenda in social computing by analyzing behavioral data at scales not previously achievable in other scenarios. These type of analysis pose important challenges regarding the development and integration of methods for robust and tractable audiovisual processing. In this thesis, we address the problem of mining user behavior inside conversational videos by addressing three main aspects. First, we integrate state-of-the art audio processing and computer vision techniques to analyze conversational social video. While the initial focus of the thesis is the nonverbal aspect of vlogger behavior, we also investigate the verbal content. Second, we study some of the interpersonal and social processes that link vlogger behavior and vlog consumption in social media platforms such as YouTube. In this context, we examine the phenomenon of social attention in vlogs, and we investigate the use of crowdsourcing as a scalable method to annotate large multimodal corpora with interpersonal perception impres- sions. Finally, we propose a computational framework to predict interpersonal impressions automatically using multimedia analysis, crowdsourced impressions, and machine learning techniques. We anticipate that the work presented in this dissertation will motivate future work in social and behavioral sciences, media analysis, natural language processing, and affective and social computing applied to the large-scale analysis of human interaction in social video.

EPFL2013

From recommender systems to spatio-temporal dynamics with network science

Kirell Maël Benzi

Networks are data structures that are fundamental for capturing and analyzing complex interactions between objects. While they have been used for decades to solve problems in virtually all scientific fields, their usage for data analysis in real-world practical applications deserves to be further investigated. In this thesis, we explore multiple aspects of network science and show how the design of new graph-based approaches offers an unprecedented depth for analyzing complex datasets. Through the study of practical applications, we demonstrate how to extract key findings in several domains such as digital humanities, recommender systems, social behavior, neuroscience or knowledge discovery. First, we propose to define in a concise manner the data science workflow. We present the tools, techniques, and questions that the practitioner needs to have in mind when addressing a new large-scale problem as they are of tremendous importance if one wants to apply network science concepts to real applications. Based on this foundation chapter, we begin by demonstrating the worth of networks for music recommendation with Genezik, our smart playlist application that adapts to user taste. Using signal processing, machine learning, and graphs, we show how to improve the performance of recommender systems as well as proposing a radically different user experience that has yet to be found in competing systems. We then move on to the introduction of the causal multilayer graph of activity, a novel graph method dedicated to the analysis of dynamical processes over networks. More than a data structure, we present a data analysis approach that tracks spreading or propagation of events through time in a scalable manner by efficiently combining a network with values associated with its vertices. Used in four different applications, the analysis of spatio-temporal patterns of activity extracted from the causal multilayer graph helps us better understand how rumors spread in social networks or how brain regions interact in resting states for instance. Finally, we study the browsing behavior of millions of people on Wikipedia and show how to extract contextual patterns of activity that reflect what is collectively remembered from past events. Based on their analysis, we confirm social studies on human behavior and conclude by revealing some of the rules that define human curiosity.