Publications related to Learning rate

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov, Scott William Pesme

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

Social Opinion Formation and Decision Making Under Communication Trends

Ali H. Sayed, Mert Kayaalp, Virginia Bordignon

This work studies the learning process over social networks under partial and random information sharing. In traditional social learning models, agents exchange full belief information with each other while trying to infer the true state of nature. We stud ...

Piscataway2024

Understanding generalization and robustness in modern deep learning

Maksym Andriushchenko

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

GANDALF: Graph-based transformer and Data Augmentation Active Learning Framework with interpretable features for multi-label chest Xray classification

Informative sample selection in an active learning (AL) setting helps a machine learning system attain optimum performance with minimum labeled samples, thus reducing annotation costs and boosting performance of computer-aided diagnosis systems in the pres ...

Amsterdam2024

Enabling Uncertainty Estimation in Iterative Neural Networks

Pascal Fua, Nikita Durasov, Doruk Oner, Minh Hieu Lê

Turning pass-through network architectures into iterative ones, which use their own output as input, is a well-known approach for boosting performance. In this paper, we argue that such architectures offer an additional benefit: The convergence rate of the ...

2024

Phenomenological theory of variational quantum ground-state preparation

Ivano Tavernelli, Giuseppe Carleo

The variational approach is a cornerstone of computational physics, considering both conventional and quantum computing computational platforms. The variational quantum eigensolver algorithm aims to prepare the ground state of a Hamiltonian exploiting para ...

College Pk2023

XTab: Cross-table Pretraining for Tabular Transformers

Mahsa Shoaran, Bingzhao Zhu

The success of self-supervised learning in computer vision and natural language processing has motivated pretraining methods on tabular data. However, most existing tabular self-supervised learning models fail to leverage information across multiple data t ...

JMLR.org2023

Text Representation Learning for Low Cost Natural Language Understanding

Jan Frederik Jonas Florian Mai

Natural language processing and other artificial intelligence fields have witnessed impressive progress over the past decade. Although some of this progress is due to algorithmic advances in deep learning, the majority has arguably been enabled by scaling ...

EPFL2023

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

The statistical complexity of early-stopped mirror descent

Tomas Vaskevicius, Varun Kanade

Recently there has been a surge of interest in understanding implicit regularization properties of iterative gradient-based optimization algorithms. In this paper, we study the statistical guarantees on the excess risk achieved by early-stopped unconstrain ...

Oxford2023

Learning rate

Graph Chatbot

Chat with Graph Search

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Social Opinion Formation and Decision Making Under Communication Trends

Understanding generalization and robustness in modern deep learning

GANDALF: Graph-based transformer and Data Augmentation Active Learning Framework with interpretable features for multi-label chest Xray classification

Enabling Uncertainty Estimation in Iterative Neural Networks

Phenomenological theory of variational quantum ground-state preparation

XTab: Cross-table Pretraining for Tabular Transformers

Text Representation Learning for Low Cost Natural Language Understanding

Universal and adaptive methods for robust stochastic optimization

The statistical complexity of early-stopped mirror descent

Understanding generalization and robustness in modern deep learning

GANDALF: Graph-based transformer and Data Augmentation Active Learning Framework with interpretable features for multi-label chest Xray classification

Enabling Uncertainty Estimation in Iterative Neural Networks

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Social Opinion Formation and Decision Making Under Communication Trends

Phenomenological theory of variational quantum ground-state preparation

XTab: Cross-table Pretraining for Tabular Transformers

Universal and adaptive methods for robust stochastic optimization

The statistical complexity of early-stopped mirror descent

Text Representation Learning for Low Cost Natural Language Understanding