Publications related to Global Convergence and Variance Reduction for a Class of Nonconvex-Nonconcave Minimax Problems

Towards Verifiable, Generalizable and Efficient Robust Deep Neural Networks.

In the last decade, deep neural networks have achieved tremendous success in many fields of machine learning.However, they are shown vulnerable against adversarial attacks: well-designed, yet imperceptible, perturbations can make the state-of-the-art deep ...

EPFL2022

Masked Training of Neural Networks with Partial Gradients

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extra-gradient), limiting SGD updates to a sub ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Volkan Cevher, Kimon Antonakopoulos, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi, Ali Kavis

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

2022

Sparse-RS: A Versatile Framework for Query-Efficient Sparse Black-Box Adversarial Attacks

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko, Francesco Croce

We propose a versatile framework based on random search, Sparse-RS, for score-based sparse targeted and untargeted attacks in the black-box setting. Sparse-RS does not rely on substitute models and achieves state-of-the-art success rate and query efficienc ...

ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE2022

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Ahmet Alacaoglu

Stochastic gradient descent (SGD) and randomized coordinate descent (RCD) are two of the workhorses for training modern automated decision systems. Intriguingly, convergence properties of these methods are not well-established as we move away from the spec ...

EPFL2021

Unsupervised Domain Adaptation for Cross-Subject Few-Shot Neurological Symptom Detection

Mahsa Shoaran, Bingzhao Zhu

Modern machine learning tools have shown promise in detecting symptoms of neurological disorders. However, current approaches typically train a unique classifier for each subject. This subject-specific training scheme requires long labeled recordings from ...

IEEE2021

On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them

Sabine Süsstrunk, Mathieu Salzmann, Tao Lin, Chen Liu

We analyze the influence of adversarial training on the loss landscape of machine learning models. To this end, we first provide analytical studies of the properties of adversarial loss functions under different adversarial budgets. We then demonstrate tha ...

2021

Wasserstein Adversarial Regularization for learning with label noise

Devis Tuia, Sylvain Lobry, Nicolas Courty

Noisy labels often occur in vision datasets, especially when they are obtained from crowdsourcing or Web scraping. We propose a new regularization method, which enables learning robust classifiers in presence of noisy data. To achieve this goal, we propose ...

2021

A Catalyst Framework for Minimax Optimization

Negar Kiyavash

We introduce a generic two-loop scheme for smooth minimax optimization with strongly-convex-concave objectives. Our approach applies the accelerated proximal point framework (or Catalyst) to the associated dual problem and takes full advantage of existing ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2020

Global Convergence and Variance Reduction for a Class of Nonconvex-Nonconcave Minimax Problems

Graph Chatbot

Chat with Graph Search

Towards Verifiable, Generalizable and Efficient Robust Deep Neural Networks.

Masked Training of Neural Networks with Partial Gradients

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Sparse-RS: A Versatile Framework for Query-Efficient Sparse Black-Box Adversarial Attacks

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Unsupervised Domain Adaptation for Cross-Subject Few-Shot Neurological Symptom Detection

On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them

Wasserstein Adversarial Regularization for learning with label noise

A Catalyst Framework for Minimax Optimization

Masked Training of Neural Networks with Partial Gradients

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Towards Verifiable, Generalizable and Efficient Robust Deep Neural Networks.

Sparse-RS: A Versatile Framework for Query-Efficient Sparse Black-Box Adversarial Attacks

Unsupervised Domain Adaptation for Cross-Subject Few-Shot Neurological Symptom Detection

Wasserstein Adversarial Regularization for learning with label noise

A Catalyst Framework for Minimax Optimization

On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them