Dynamic Personalized Ranking | EPFL Graph Search

Personalized ranking methods are at the core of many systems that learn to produce recommendations from user feedbacks. Their primary objective is to identify relevant items from very large vocabularies and to assist users in discovering new content. These techniques have proven successful in stationary regimes, but the transition to an interactive and social Web, and the rise of user-generated content, increasingly require learning from dynamic factors. Existing approaches, based on distributed vector representations, notoriously fail in fast-changing contexts and sparse regimes; their static representation of users and items prevents them from adapting to contextual changes. Given this limitation, this thesis focuses on introducing new methods to make predictions based on contextual variables, exploit rich (social, temporal) signals, and maintain model consistency over time.First, we study a recommendation task on a live-streaming platform with a dynamically evolving set of available items. In this context, users have to choose from a limited subset of creators that are currently streaming content. To model this setting, we introduce a self-attentive approach that draws a dependency between available options and re-ranks the most promising candidates. We also show that repeat consumption, and the time at which it occurs, are predictive factors that we incorporate into our model.Second, we propose a dynamic embedding scheme to maintain a latent space consistent over multiple temporal slices of a dataset, by penalizing unnecessary differences between successive solutions of the model. In order to highlight the potential for analysis of our approach, we apply our methodology to a dataset of news production. Thanks to its ability to propagate information over several time epochs, our model sheds light on important changes in news coverage induced by acquisitions of media companies. Third, we propose an approach for modeling reciprocal interest on a bartering platform where users exchange goods, for which we design a trade recommendation system. We find that the social ties between members have a strong influence, as does the time at which they trade, therefore we extend our model to be socially- and temporally-aware. Finally, we study the collaboration between users on Reddit Place, a social experiment where users drew together on a virtual canvas. We propose to use personalization methods to model user actions, capture the latent structure of this emergent collaborative effort and provide an interpretable representation of its social structure.

Graph Representation Learning with Optimal Transport: Analysis and Applications

Effrosyni Simou

In several machine learning settings, the data of interest are well described by graphs. Examples include data pertaining to transportation networks or social networks. Further, biological data, such as proteins or molecules, lend themselves well to graph-structured descriptions. In such settings, it is desirable to design representation learning algorithms that can take into account the information captured by the underlying structure. This thesis focuses on providing new methods to ingrain geometrical information in end-to-end deep learning architectures for graphs through the use of elements from Optimal Transport theory.First, we consider the setting where the data can be described by a fixed graph. In this setting, each datapoint corresponds to a node of the graph and the edges among nodes signify their pairwise relations. We propose an autoencoder architecture that consists of a linear layer in the encoder and a novel Wasserstein barycentric layer at the decoder. Our proposed barycentric layer takes into account the underlying geometry through the diffusion distance of the graph and provides a way to obtain structure-aware, non-linear interpolations, which lead to directly interpretable node embeddings that are stable to perturbations of the graph structure. Further, the embeddings obtained with our proposed autoencoder achieve competitive or superior results compared to state-of-the-art methods in node classification tasks.Second, we consider the setting where the data consist of multiple graphs. In that setting, the graphs can be of varying size and, therefore, a global pooling operation is needed in order to obtain representations of fixed size and enable end-to-end learning with deep networks. We propose a global pooling operation that optimally preserves the statistical properties of the representations. In order to do so, we take into account the geometry of the representation space and introduce a global pooling layer that minimizes the Wasserstein distance between a graph representation and its pooled counterpart, of fixed size. This is achieved by performing a Wasserstein gradient flow with respect to the pooled representation. Our proposed method demonstrates promising results in the task of graph classification.Overall, in this thesis we provide new methods for incorporating geometrical information in end-to-end deep learning architectures for graph structured data. We believe that our proposals will contribute to the development of algorithms that learn meaningful representations and that take fully into account the geometry of the data under consideration.

EPFL2022

Robust audio segmentation

Jitendra Ajmera

Audio segmentation, in general, is the task of segmenting a continuous audio stream in terms of acoustically homogenous regions, where the rule of homogeneity depends on the task. This thesis aims at developing and investigating efficient, robust and unsupervised techniques for three important tasks related to audio segmentation, namely speech/music segmentation, speaker change detection and speaker clustering. The speech/music segmentation technique proposed in this thesis is based on the functioning of a HMM/ANN hybrid ASR system where an MLP estimates the posterior probabilities of different phonemes. These probabilities exhibit a particular pattern when the input is a speech signal. This pattern is captured in the form of feature vectors, which are then integrated in a HMM framework. The technique thus segments the audio data in terms of recognizable and non-recognizable segments. The efficiency of the proposed technique is demonstrated by a number of experiments conducted on broadcast news data exhibiting real-life scenarios (different speech and music styles, overlapping speech and music, non-speech sounds other than music, etc.). A novel distance metric is proposed in this thesis for the purpose of finding speaker segment boundaries (speaker change detection). The proposed metric can be seen as special case of Log Likelihood Ratio (LLR) or Bayesian Information Criterion (BIC), where the number of parameters in the two models (or hypotheses) is forced to be equal. However, the advantage of the proposed metric over LLR, BIC and other metric based approaches is that it achieves comparable performance without requiring an adjustable threshold/penalty term, hence also eliminating the need for a development dataset. Speaker clustering is the task of unsupervised classification of the audio data in terms of speakers. For this purpose, a novel HMM based agglomerative clustering algorithm is proposed where, starting from a large number of clusters, closest clusters are merged in an iterative process. A novel merging criterion is proposed for this purpose, which does not require an adjustable threshold value and hence the stopping criterion is also automatically met when there are no more clusters left for merging. The efficiency of the proposed algorithm is demonstrated with various experiments on broadcast news data and it is shown that the proposed criterion outperforms the use of LLR, when LLR is used with an optimal threshold value. These tasks obviously play an important role in the pre-processing stages of ASR. For example, correctly identifying non-recognizable segments in the audio stream and excluding them from recognition saves computation time in ASR and results in more meaningful transcriptions. Moreover, researchers have clearly shown the positive impact of further clustering of identified speech segments in terms of speakers (speaker clustering) on the transcription accuracy. However, we note that this processing has various other interesting and practical applications. For example, this provides characteristic information about the data (metadata), which is useful for the indexing of audio documents. One such application is investigated in this thesis which extracts this metadata and combines it with the ASR output, resulting in Rich Transcription (RT) which is much easier to understand for an end-user. In a further application, speaker clustering was combined with precise location information available in scenarios like smart meeting rooms to segment the meeting recordings jointly in terms of speakers and their locations in a meeting room. This is useful for automatic meeting summarization as it enables answering of questions like "who is speaking and where". This could be used to access, for example, a specific presentation made by a particular speaker or all the speech segments belonging to a particular speaker.

EPFL2005

Three essays on the role of proximity in science and innovation

Stefano Horst Baruffaldi

This thesis proposes three studies that provide novel empirical evidence on how different types of proximity can affect innovation and science activities through various mechanisms and in different contexts. In the first study (second chapter of this thesis), in collaboration with Julio Raffo, we analyze the relationship between geography and the likelihood of duplication in inventive activities. We argue that the uneven diffusion of knowledge means that the duplication of inventions will not be randomly distributed geographically and over time. First, as knowledge diffuses over time and competitive incentives decrease, the probability of a claimed invention duplicating an existing one will decrease in the time distance between the two. Second, for recent and upcoming inventions, competitive incentives are high, and localized knowledge flows increase the probability of duplication. Therefore, over a brief time period, the probability of duplication decreases with geographic distance. Conversely, the duplication of less recent inventions is more likely to occur at long distances as a consequence of less awareness of a technology existing due to missing knowledge flows. We test our hypotheses on European Patent Office (EPO) patent bibliographical data on patent citation categories. Geographic distance matters significantly less in sectors in which patents are known to be more effective as a source of information such as discrete technologies. In the second study (third chapter), in collaboration with Annamaria Conti and Fabiana Visentin, we investigate the effects of professorsâ social proximity with external universities on the level of productivity of PhD students hired from these universities. Researchers hired from external environments tend to have high scientific productivity compared to those who completed their studies in the same institution where they are employed. In a population of 4,666 PhD students, we further study the scientific productivity of external students from professorsâ networks, defined as students with a master's degree from a different university from that of their PhD, and also from a university with which their supervisors' co-authors are affiliated. We find that these students are significantly more productive, both compared to other students with a master's degree from a different university, and to students with a master's degree from the same university as that of their PhD. In our analyses, we control for the heterogeneity of supervisors and the heterogeneity of institutions where the students obtained their master's degrees, including proxies for the specific relevance of these universities for a given supervisor. Thus, we conclude that professors hire students with higher scientific productivity from universities where their co-authors are affiliated. Additional analyses further suggest that the reduction of information asymmetries is the main mechanism to explain this finding. In the third study (fourth chapter), in collaboration with Guillaume Burghouwt, we investigate the role of interregional knowledge integration as a driver of firm innovative performance. We adopt an unbalanced panel of 3,871 innovative companies in Germany between 1992 and 2010, for a total of 15,819 observations, and we study their innovative productivity. [...]

EPFL2015