Publications associées à Apprentissage organisationnel

On the (In)security of Peer-to-Peer Decentralized Machine Learning

Carmela González Troncoso, Mathilde Aliénor Raynal, Dario Pasquini

In this work, we carry out the first, in-depth, privacy analysis of Decentralized Learning-a collaborative machine learning framework aimed at addressing the main limitations of federated learning. We introduce a suite of novel attacks for both passive and ...

IEEE COMPUTER SOC2023

Improving Generalization of Pretrained Language Models

Rabeeh Karimi Mahabadi

In this dissertation, we propose multiple methods to improve transfer learning for pretrained language models (PLMs). Broadly, transfer learning is a powerful technique in natural language processing, where a language model is first pre-trained on a data-r ...

EPFL2023

Beyond Spectral Gap: The Role of the Topology in Decentralized Learning

Martin Jaggi, Thijs Vogels, Hadrien Hendrikx

In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rates and optimize faster. In the decentralized setting, in which workers commu ...

Microtome Publishing2023

Lifelong Machine Learning with Data Efficiency and Knowledge Retention

Fei Mi

Artificial intelligence (AI) and machine learning (ML) have become de facto tools in many real-life applications to offer a wide range of benefits for individuals and our society. A classic ML model is typically trained with a large-scale static dataset in ...

EPFL2021

Word-level Embeddings for Cross-Task Transfer Learning in Speech Processing

Pierre Anton Beckmann, Milos Cernak

Recent breakthroughs in deep learning often rely on representation learning and knowledge transfer. In recent years, unsupervised and self-supervised techniques for learning speech representation were developed to foster automatic speech recognition. Up to ...

EUROPEAN ASSOC SIGNAL SPEECH & IMAGE PROCESSING-EURASIP2021

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Martin Jaggi, Amirkeivan Mohtashami, Sebastian Urban Stich

It has been experimentally observed that the efficiency of distributed training with stochastic gradient (SGD) depends decisively on the batch size and—in asynchronous implementations—on the gradient staleness. Especially, it has been observed that the spe ...

MICROTOME PUBLISHING2021

Approaching Ontology Alignment through Representation Learning to Bridge the Semantic Gap in Engineering Applications

Prodromos Kolyvakis

The current information landscape is characterised by a vast amount of relatively semantically homogeneous, when observed in isolation, data silos that are, however, drastically semantically fragmented when considered as a whole. Within each data silo, inf ...

EPFL2020

Imagining with and in Space: Structural support as a cognitive device in architectural education

Dieter Dietz, Julien Lafontaine Carboni, Aurélie Dupuis, Darío Negueruela Del Castillo

In this paper, we will argue that the structure of the spatio-temporal pedagogical set-up and the elaboration of an evolutionary support for operations of cognition and imagination are powerful in transforming learning cultures. We develop this argument in ...

2020

Age-Dependent Statistical Learning Trajectories Reveal Differences in Information Weighting

Steffen Alexander Herff

Statistical learning (SL) is the ability to generate predictions based on probabilistic dependencies in the environment, an ability that is present throughout life. The effect of aging on SL is still unclear. Here, we explore statistical learning in health ...

AMER PSYCHOLOGICAL ASSOC2020

Knowledge Transfer with Jacobian Matching

François Fleuret, Suraj Srinivas

Classical distillation methods transfer representations from a “teacher” neural network to a “student” network by matching their output activations. Recent methods also match the Jacobians, or the gradient of output activations with the input. However, thi ...

2018