Publications associées à Accelerated And Inexact Forward-Backward Algorithms

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

EPFL2024

Residual-based attention in physics-informed neural networks

Nikolaos Stergiopoulos, Sokratis Anagnostopoulos

Driven by the need for more efficient and seamless integration of physical models and data, physics -informed neural networks (PINNs) have seen a surge of interest in recent years. However, ensuring the reliability of their convergence and accuracy remains ...

Lausanne2024

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov, Scott William Pesme

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

Understanding generalization and robustness in modern deep learning

Maksym Andriushchenko

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

Distributional Regression and Autoregression via Optimal Transport

Laya Ghodrati

We present a framework for performing regression when both covariate and response are probability distributions on a compact and convex subset of

\R^d

. Our regression model is based on the theory of optimal transport and links the conditional Fr'echet m ...

EPFL2023

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Functional data analysis with rough sample paths?

Victor Panaretos, Neda Mohammadi Jouzdani

Functional data are typically modeled as sample paths of smooth stochastic processes in order to mitigate the fact that they are often observed discretely and noisily, occasionally irregularly and sparsely. The smoothness assumption is imposed to allow for ...

TAYLOR & FRANCIS LTD2023

Impact of phylogeny on structural contact inference from protein sequence data

Anne-Florence Raphaëlle Bitbol, Umberto Lupo, Nicola Dietler

Local and global inference methods have been developed to infer structural contacts from multiple sequence alignments of homologous proteins. They rely on correlations in amino acid usage at contacting sites. Because homologous proteins share a common ance ...

ROYAL SOC2023

Stable Nonconvex-Nonconcave Training via Linear Interpolation

Volkan Cevher, Thomas Michaelsen Pethick, Wanyun Xie

This paper presents a theoretical analysis of linear interpolation as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss lan ...

2023

Accelerated And Inexact Forward-Backward Algorithms

Graph Chatbot

Chattez avec Graph Search

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Residual-based attention in physics-informed neural networks

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Understanding generalization and robustness in modern deep learning

Distributional Regression and Autoregression via Optimal Transport

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Universal and adaptive methods for robust stochastic optimization

Functional data analysis with rough sample paths?

Impact of phylogeny on structural contact inference from protein sequence data

Stable Nonconvex-Nonconcave Training via Linear Interpolation

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Residual-based attention in physics-informed neural networks

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Understanding generalization and robustness in modern deep learning

Universal and adaptive methods for robust stochastic optimization

Functional data analysis with rough sample paths?

Impact of phylogeny on structural contact inference from protein sequence data

Stable Nonconvex-Nonconcave Training via Linear Interpolation

Distributional Regression and Autoregression via Optimal Transport