Publications related to Laboratory for Information and Inference Systems

Robustness and invariance properties of image classifiers

Deep neural networks have achieved impressive results in many image classification tasks. However, since their performance is usually measured in controlled settings, it is important to ensure that their decisions remain correct when deployed in noisy envi ...

EPFL2022

Learning of Continuous and Piecewise-Linear Functions With Hessian Total-Variation Regularization

Michaël Unser, Shayan Aziznejad, Joaquim Gonçalves Garcia Barreto Campos

We develop a novel 2D functional learning framework that employs a sparsity-promoting regularization based on second-order derivatives. Motivated by the nature of the regularizer, we restrict the search space to the span of piecewise-linear box splines shi ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Stochastic distributed learning with gradient quantization and double-variance reduction

Sebastian Urban Stich, Konstantin Mishchenko

We consider distributed optimization over several devices, each sending incremental model updates to a central server. This setting is considered, for instance, in federated learning. Various schemes have been designed to compress the model updates in orde ...

TAYLOR & FRANCIS LTD2022

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Theory of Deep Learning: Neural Tangent Kernel and Beyond

Arthur Ulysse Jacot-Guillarmod

In the recent years, Deep Neural Networks (DNNs) have managed to succeed at tasks that previously appeared impossible, such as human-level object recognition, text synthesis, translation, playing games and many more. In spite of these major achievements, o ...

EPFL2022

Proximal Point Imitation Learning

Volkan Cevher, Luca Viano, Igor Krawczuk, Angeliki Kamoutsi

This work develops new algorithms with rigorous efficiency guarantees for infinite horizon imitation learning (IL) with linear function approximation without restrictive coherence assumptions. We begin with the minimax formulation of the problem and then o ...

2022

Iterative pre-conditioning for expediting the distributed gradient-descent method: The case of linear least-squares problem

Nirupam Gupta

This paper considers the multi-agent linear least-squares problem in a server-agent network architecture. The system comprises multiple agents, each with a set of local data points. The agents are connected to a server, and there is no inter-agent communic ...

PERGAMON-ELSEVIER SCIENCE LTD2022

Neural Set Function Extensions: Learning with Discrete Functions in High Dimensions

Andreas Loukas, Nikolaos Karalias

Integrating functions on discrete domains into neural networks is key to developing their capability to reason about discrete objects. But, discrete domains are (I) not naturally amenable to gradient-based optimization, and (II) incompatible with deep lear ...

2022

Adversarial training with informed data selection

Pascal Frossard, Javier Alejandro Maroto Morales, Marcele Oliveira Kuhfuss de Mendonça

With the increasing amount of available data and advances in computing capabilities, deep neural networks (DNNs) have been successfully employed to solve challenging tasks in various areas, including healthcare, climate, and finance. Nevertheless, state-of ...

IEEE2022

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Volkan Cevher, Alp Yurtsever, Maria-Luiza Vladarean

We propose a stochastic conditional gradient method (CGM) for minimizing convex finitesum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or require carefully inc ...

2022

Laboratory for Information and Inference Systems

Graph Chatbot

Chat with Graph Search

Robustness and invariance properties of image classifiers

Learning of Continuous and Piecewise-Linear Functions With Hessian Total-Variation Regularization

Stochastic distributed learning with gradient quantization and double-variance reduction

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Theory of Deep Learning: Neural Tangent Kernel and Beyond

Proximal Point Imitation Learning

Iterative pre-conditioning for expediting the distributed gradient-descent method: The case of linear least-squares problem

Neural Set Function Extensions: Learning with Discrete Functions in High Dimensions

Adversarial training with informed data selection

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Stochastic distributed learning with gradient quantization and double-variance reduction

Theory of Deep Learning: Neural Tangent Kernel and Beyond

Learning of Continuous and Piecewise-Linear Functions With Hessian Total-Variation Regularization

Robustness and invariance properties of image classifiers

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Proximal Point Imitation Learning

Neural Set Function Extensions: Learning with Discrete Functions in High Dimensions

Iterative pre-conditioning for expediting the distributed gradient-descent method: The case of linear least-squares problem

Adversarial training with informed data selection

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization