Publications related to Hyperparameter (machine learning)

Momentum-Based Policy Gradient with Second-Order Information

Patrick Thiran, Negar Kiyavash, Mohammadsadegh Khorasani, Saber Salehkaleybar

Variance-reduced gradient estimators for policy gradient methods have been one of the main focus of research in the reinforcement learning in recent years as they allow acceleration of the estimation process. We propose a variance-reduced policy-gradient m ...

2022

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Optimal recovery of unsecured debt via interpretable reinforcement learning

Thomas Alois Weber, Michael Mark, Huanxi Liu

This paper addresses the issue of interpretability and auditability of reinforcement-learning agents employed in the recovery of unsecured consumer debt. To this end, we develop a deterministic policy-gradient method that allows for a natural integration o ...

2022

Adversarial training with informed data selection

Pascal Frossard, Javier Alejandro Maroto Morales, Marcele Oliveira Kuhfuss de Mendonça

With the increasing amount of available data and advances in computing capabilities, deep neural networks (DNNs) have been successfully employed to solve challenging tasks in various areas, including healthcare, climate, and finance. Nevertheless, state-of ...

IEEE2022

Disparity Between Batches as a Signal for Early Stopping

Patrick Thiran, Mahsa Forouzesh

We propose a metric for evaluating the generalization ability of deep neural networks trained with mini-batch gradient descent. Our metric, called gradient disparity, is the l2 norm distance between the gradient vectors of two mini-batches drawn from the t ...

Springer2021

Counting using deep learning regression gives value to ecological surveys

Devis Tuia, Benjamin Alexander Kellenberger

Many ecological studies rely on count data and involve manual counting of objects of interest, which is time-consuming and especially disadvantageous when time in the field or lab is limited. However, an increasing number of works uses digital imagery, whi ...

2021

Conditional LSTM-GAN for Melody Generation from Lyrics

Simon Canales, Yi Yu

Melody generation from lyrics has been a challenging research issue in the field of artificial intelligence and music, which enables us to learn and discover latent relationships between interesting lyrics and accompanying melodies. Unfortunately, the limi ...

2021

Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent

Rachid Guerraoui, El Mahdi El Mhamdi, Sébastien Louis Alexandre Rouault

Byzantine-resilient Stochastic Gradient Descent (SGD) aims at shielding model training from Byzantine faults, be they ill-labeled training datapoints, exploited software/hardware vulnerabilities, or malicious worker nodes in a distributed setting. Two rece ...

2021

Learning Saliency From Single Noisy Labelling: A Robust Model Fitting Perspective

Tong Zhang

The advances made in predicting visual saliency using deep neural networks come at the expense of collecting large-scale annotated data. However, pixel-wise annotation is labor-intensive and overwhelming. In this paper, we propose to learn saliency predict ...

2021

Optimizer Benchmarking Needs to Account for Hyperparameter Tuning

François Fleuret, Martin Jaggi, Thijs Vogels, Jan Frederik Jonas Florian Mai, Prabhu Teja Sivaprasad

The performance of optimizers, particularly in deep learning, depends considerably on their chosen hyperparameter configuration. The efficacy of optimizers is often studied under near-optimal problem-specific hyperparameters, and finding these settings may ...

2020

Hyperparameter (machine learning)

Graph Chatbot

Chat with Graph Search

Momentum-Based Policy Gradient with Second-Order Information

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Optimal recovery of unsecured debt via interpretable reinforcement learning

Adversarial training with informed data selection

Disparity Between Batches as a Signal for Early Stopping

Counting using deep learning regression gives value to ecological surveys

Conditional LSTM-GAN for Melody Generation from Lyrics

Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent

Learning Saliency From Single Noisy Labelling: A Robust Model Fitting Perspective

Optimizer Benchmarking Needs to Account for Hyperparameter Tuning

Disparity Between Batches as a Signal for Early Stopping

Counting using deep learning regression gives value to ecological surveys

Conditional LSTM-GAN for Melody Generation from Lyrics

Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Optimal recovery of unsecured debt via interpretable reinforcement learning

Adversarial training with informed data selection

Momentum-Based Policy Gradient with Second-Order Information

Learning Saliency From Single Noisy Labelling: A Robust Model Fitting Perspective

Optimizer Benchmarking Needs to Account for Hyperparameter Tuning