Publications related to Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms

Quantifying the Unknown: Data-Driven Approaches and Applications in Energy Systems

In light of the challenges posed by climate change and the goals of the Paris Agreement, electricity generation is shifting to a more renewable and decentralized pattern, while the operation of systems like buildings is increasingly electrified. This calls ...

EPFL2024

Random matrix methods for high-dimensional machine learning models

Antoine Philippe Michel Bodin

In the rapidly evolving landscape of machine learning research, neural networks stand out with their ever-expanding number of parameters and reliance on increasingly large datasets. The financial cost and computational resources required for the training p ...

EPFL2024

On the Generalization of Stochastic Gradient Descent with Momentum

Volkan Cevher, Kimon Antonakopoulos

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Brookline2024

Bayes-optimal Learning of Deep Random Networks of Extensive-width

Florent Gérard Krzakala, Lenka Zdeborová, Hugo Chao Cui

We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights. We consider the asymptotic limit where the number of samples, the input dimension and the network width ...

2023

Distributional Regression and Autoregression via Optimal Transport

Laya Ghodrati

We present a framework for performing regression when both covariate and response are probability distributions on a compact and convex subset of

\R^d

. Our regression model is based on the theory of optimal transport and links the conditional Fr'echet m ...

EPFL2023

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

The statistical complexity of early-stopped mirror descent

Tomas Vaskevicius, Varun Kanade

Recently there has been a surge of interest in understanding implicit regularization properties of iterative gradient-based optimization algorithms. In this paper, we study the statistical guarantees on the excess risk achieved by early-stopped unconstrain ...

Oxford2023

DARE-GRAM : Unsupervised Domain Adaptation Regression by Aligning Inverse Gram Matrices

Olga Fink, Ismail Nejjar

Unsupervised Domain Adaptation Regression (DAR) aims to bridge the domain gap between a labeled source dataset and an unlabelled target dataset for regression problems. Recent works mostly focus on learning a deep feature encoder by minimizing the discrepa ...

IEEE2023

Local Kernel Regression and Neural Network Approaches to the Conformational Landscapes of Oligopeptides

Michele Ceriotti, Alberto Fabrizio, Benjamin André René Meyer, Edgar Albert Engel, Raimon Fabregat I De Aguilar-Amat, Veronika Juraskova

The application of machine learning to theoretical chemistry has made it possible to combine the accuracy of quantum chemical energetics with the thorough sampling of finite-temperature fluctuations. To reach this goal, a diverse set of methods has been pr ...

2022

Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms

Graph Chatbot

Chat with Graph Search

Quantifying the Unknown: Data-Driven Approaches and Applications in Energy Systems

Random matrix methods for high-dimensional machine learning models

On the Generalization of Stochastic Gradient Descent with Momentum

Bayes-optimal Learning of Deep Random Networks of Extensive-width

Distributional Regression and Autoregression via Optimal Transport

Universal and adaptive methods for robust stochastic optimization

Deep Learning Generalization with Limited and Noisy Labels

The statistical complexity of early-stopped mirror descent

DARE-GRAM : Unsupervised Domain Adaptation Regression by Aligning Inverse Gram Matrices

Local Kernel Regression and Neural Network Approaches to the Conformational Landscapes of Oligopeptides

Quantifying the Unknown: Data-Driven Approaches and Applications in Energy Systems

Bayes-optimal Learning of Deep Random Networks of Extensive-width

Deep Learning Generalization with Limited and Noisy Labels

Random matrix methods for high-dimensional machine learning models

Universal and adaptive methods for robust stochastic optimization

Distributional Regression and Autoregression via Optimal Transport

On the Generalization of Stochastic Gradient Descent with Momentum

The statistical complexity of early-stopped mirror descent

DARE-GRAM : Unsupervised Domain Adaptation Regression by Aligning Inverse Gram Matrices

Local Kernel Regression and Neural Network Approaches to the Conformational Landscapes of Oligopeptides