Publications associées à Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

EPFL2024

Exploiting the Signal-Leak Bias in Diffusion Models

Sabine Süsstrunk, Radhakrishna Achanta, Mahmut Sami Arpa, Martin Nicolas Everaert, Athanasios Fitsios

There is a bias in the inference pipeline of most diffusion models. This bias arises from a signal leak whose distribution deviates from the noise distribution, creating a discrepancy between training and inference processes. We demonstrate that this signa ...

2024

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Scott William Pesme

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Transportation-based functional ANOVA and PCA for covariance operators

Victor Panaretos, Yoav Zemel, Valentina Masarotto

We consider the problem of comparing several samples of stochastic processes with respect to their second-order structure, and describing the main modes of variation in this second order structure, if present. These tasks can be seen as an Analysis of Vari ...

Inst Mathematical Statistics-Ims2024

On the Generalization of Stochastic Gradient Descent with Momentum

Volkan Cevher, Kimon Antonakopoulos

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Brookline2024

Biases in Information Selection and Processing: Survey Evidence from the Pandemic

Andreas Fuster

We conduct two survey experiments to study which information people choose to consume and how it affects their beliefs. In the first experiment, respondents choose between optimistic and pessimistic article headlines related to the COVID-19 pandemic and ar ...

2024

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov, Scott William Pesme

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

Understanding generalization and robustness in modern deep learning

Maksym Andriushchenko

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

Gaia's brightest very metal-poor (VMP) stars Metallicity catalogue of a thousand VMP stars from Gaia's radial velocity spectrometer spectra

Nicolas Lawrence Etienne Longeard

Context. Gaia DR3 has offered the scientific community a remarkable dataset of approximately one million spectra acquired with the radial velocity spectrometer (RVS) in the calcium II triplet region, which is well suited to identify very metal-poor (VMP) s ...

Edp Sciences S A2024

It’s All Relative: Learning Interpretable Models for Scoring Subjective Bias in Documents from Pairwise Comparisons

Matthias Grossglauser, Aswin Suresh, Chi Hsuan Wu

We propose an interpretable model to score the subjective bias present in documents, based only on their textual content. Our model is trained on pairs of revisions of the same Wikipedia article, where one version is more biased than the other. Although pr ...

2024

Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

Graph Chatbot

Chattez avec Graph Search

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Exploiting the Signal-Leak Bias in Diffusion Models

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Transportation-based functional ANOVA and PCA for covariance operators

On the Generalization of Stochastic Gradient Descent with Momentum

Biases in Information Selection and Processing: Survey Evidence from the Pandemic

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Understanding generalization and robustness in modern deep learning

Gaia's brightest very metal-poor (VMP) stars Metallicity catalogue of a thousand VMP stars from Gaia's radial velocity spectrometer spectra

It’s All Relative: Learning Interpretable Models for Scoring Subjective Bias in Documents from Pairwise Comparisons

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Exploiting the Signal-Leak Bias in Diffusion Models

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Transportation-based functional ANOVA and PCA for covariance operators

Biases in Information Selection and Processing: Survey Evidence from the Pandemic

On the Generalization of Stochastic Gradient Descent with Momentum

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Understanding generalization and robustness in modern deep learning

Gaia's brightest very metal-poor (VMP) stars Metallicity catalogue of a thousand VMP stars from Gaia's radial velocity spectrometer spectra

It’s All Relative: Learning Interpretable Models for Scoring Subjective Bias in Documents from Pairwise Comparisons