Publications related to Stochastic gradient descent

Lipschitz constant estimation for Neural Networks via sparse polynomial optimization

Volkan Cevher, Paul Thierry Yves Rolland, Fabian Ricardo Latorre Gomez

We introduce LiPopt, a polynomial optimization framework for computing increasingly tighter upper bounds on the Lipschitz constant of neural networks. The underlying optimization problems boil down to either linear (LP) or semidefinite (SDP) programming. W ...

2020

Supply-Power-Constrained Cable Capacity Maximization Using Multi-Layer Neural Networks

Erixhen Sula

We experimentally solve the problem of maximizing capacity under a total supply power constraint in a massively parallel submarine cable context, i.e., for a spatially uncoupled system in which fiber Kerr nonlinearity is not a dominant limitation. By using ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Square Attack: a query-efficient black-box adversarial attack via random search

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko, Francesco Croce

We propose the Square Attack, a new score-based black-box

l_2

and

l_\infty

adversarial attack that does not rely on local gradient information and thus is not affected by gradient masking. The Square Attack is based on a randomized search scheme where ...

2020

Robust Distributed Learning

El Mahdi El Mhamdi

Whether it occurs in artificial or biological substrates, {\it learning} is a {distributed} phenomenon in at least two aspects. First, meaningful data and experiences are rarely found in one location, hence {\it learners} have a strong incentive to work t ...

EPFL2020

FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction

Rachid Guerraoui, Anne-Marie Kermarrec, Rhicheek Patra, Georgios Damaskinos, Vlad-Tiberiu Nitu

Federated Learning (FL) is very appealing for its privacy benefits: essentially, a global model is trained with updates computed on mobile devices while keeping the data of users local. Standard FL infrastructures are however designed to have no energy or ...

Association for Computing Machinery2020

Minimizing Regret of Bandit Online Optimization in Unconstrained Action Spaces

Maryam Kamgarpour

We consider online convex optimization with a zero-order oracle feedback. In particular, the decision maker does not know the explicit representation of the time-varying cost functions, or their gradients. At each time step, she observes the value of the c ...

2020

MATHICSE Technical Report : Sparse Polynomial Chaos expansions using Variational Relevance Vector Machines

Fabio Nobile, Panagiotis Tsilifis

The challenges for non-intrusive methods for Polynomial Chaos modeling lie in the computational efficiency and accuracy under a limited number of model simulations. These challenges can be addressed by enforcing sparsity in the series representation throug ...

MATHICSE2020

Sparse Polynomial Chaos expansions using variational relevance vector machines

Fabio Nobile, Panagiotis Tsilifis

The challenges for non-intrusive methods for Polynomial Chaos modeling lie in the computational efficiency and accuracy under a limited number of model simulations. These challenges can be addressed by enforcing sparsity in the series representation throug ...

2020

Principal Component Analysis By Optimization Of Symmetric Functions Has No Spurious Local Optima

Armin Eftekhari

Principal component analysis (PCA) finds the best linear representation of data and is an indispensable tool in many learning and inference tasks. Classically, principal components of a dataset are interpreted as the directions that preserve most of its "e ...

SIAM PUBLICATIONS2020

Accurate deep neural network inference using computational phase-change memory

Irem Boybat Kara, Evangelos Eleftheriou, Abu Sebastian

In-memory computing using resistive memory devices is a promising non-von Neumann approach for making energy-efficient deep learning inference hardware. However, due to device variability and noise, the network needs to be trained in a specific way so that ...

2020

Stochastic gradient descent

Graph Chatbot

Chat with Graph Search

Lipschitz constant estimation for Neural Networks via sparse polynomial optimization

Supply-Power-Constrained Cable Capacity Maximization Using Multi-Layer Neural Networks

Square Attack: a query-efficient black-box adversarial attack via random search

Robust Distributed Learning

FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction

Minimizing Regret of Bandit Online Optimization in Unconstrained Action Spaces

MATHICSE Technical Report : Sparse Polynomial Chaos expansions using Variational Relevance Vector Machines

Sparse Polynomial Chaos expansions using variational relevance vector machines

Principal Component Analysis By Optimization Of Symmetric Functions Has No Spurious Local Optima

Accurate deep neural network inference using computational phase-change memory

FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction

Robust Distributed Learning

Square Attack: a query-efficient black-box adversarial attack via random search

Supply-Power-Constrained Cable Capacity Maximization Using Multi-Layer Neural Networks

MATHICSE Technical Report : Sparse Polynomial Chaos expansions using Variational Relevance Vector Machines

Sparse Polynomial Chaos expansions using variational relevance vector machines

Minimizing Regret of Bandit Online Optimization in Unconstrained Action Spaces

Principal Component Analysis By Optimization Of Symmetric Functions Has No Spurious Local Optima

Accurate deep neural network inference using computational phase-change memory

Lipschitz constant estimation for Neural Networks via sparse polynomial optimization