Publications related to Second-order guarantees in centralized, federated and decentralized nonconvex optimization

Optimization Over Banach Spaces: A Unified View on Supervised Learning and Inverse Problems

In this thesis, we reveal that supervised learning and inverse problems share similar mathematical foundations. Consequently, we are able to present a unified variational view of these tasks that we formulate as optimization problems posed over infinite-di ...

EPFL2022

Improving the Training of Compact Neural Networks for Visual Recognition

Shuxuan Guo

During the Artificial Intelligence (AI) revolution of the past decades, deep neural networks have been widely used and have achieved tremendous success in visual recognition. Unfortunately, deploying deep models is challenging because of their huge model s ...

EPFL2022

Advances In Morphological Neural Networks: Training, Pruning And Enforcing Shape Constraints

Nikolaos Dimitriadis

In this paper, we study an emerging class of neural networks, the Morphological Neural networks, from some modern perspectives. Our approach utilizes ideas from tropical geometry and mathematical morphology. First, we state the training of a binary morphol ...

IEEE2021

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Ahmet Alacaoglu

Stochastic gradient descent (SGD) and randomized coordinate descent (RCD) are two of the workhorses for training modern automated decision systems. Intriguingly, convergence properties of these methods are not well-established as we move away from the spec ...

EPFL2021

Neural Tangent Kernel: Convergence and Generalization in Neural Networks (Invited Paper)

Clément Hongler, Franck Raymond Gabriel, Arthur Jacot

The Neural Tangent Kernel is a new way to understand the gradient descent in deep neural networks, connecting them with kernel methods. In this talk, I'll introduce this formalism and give a number of results on the Neural Tangent Kernel and explain how th ...

ASSOC COMPUTING MACHINERY2021

Large Steps in Inverse Rendering of Geometry

Wenzel Alban Jakob, Baptiste Alexandre Marie Philippe Claude Nicolet

Inverse reconstruction from images is a central problem in many scientific and engineering disciplines. Recent progress on differentiable rendering has led to methods that can efficiently differentiate the full process of image formation with respect to mi ...

ASSOC COMPUTING MACHINERY2021

Active Subdivision Surfaces for the Semiautomatic Segmentation of Biomedical Volumes

Michaël Unser, Anaïs Laure Marie-Thérèse Badoual

We present a new family of active surfaces for the semiautomatic segmentation of volumetric objects in 3D biomedical images. We represent our deformable model by a subdivision surface encoded by a small set of control points and generated through a geometr ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search

Mathieu Salzmann, Kaicheng Yu

Weight sharing has become a de facto standard in neural architecture search because it enables the search to be done on commodity hardware. However, recent works have empirically shown a ranking disorder between the performance of stand-alone architectures ...

IEEE COMPUTER SOC2021

Convergence of adaptive algorithms for constrained weakly convex optimization

Volkan Cevher, Ahmet Alacaoglu, Yurii Malitskyi

We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective. We prove the

\tilde O(t^{-1/2})

rate of convergence for the squared norm of the gradient of Moreau envelope, ...

2021

Estimation of discrete choice models with hybrid stochastic adaptive batch size algorithms

Michel Bierlaire, Timothy Michael Hillel, Virginie Janine Camille Lurkin, Gael Lederrey

The emergence of Big Data has enabled new research perspectives in the discrete choice community. While the techniques to estimate Machine Learning models on a massive amount of data are well established, these have not yet been fully explored for the esti ...

2020

Second-order guarantees in centralized, federated and decentralized nonconvex optimization

Graph Chatbot

Chat with Graph Search

Optimization Over Banach Spaces: A Unified View on Supervised Learning and Inverse Problems

Improving the Training of Compact Neural Networks for Visual Recognition

Advances In Morphological Neural Networks: Training, Pruning And Enforcing Shape Constraints

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Neural Tangent Kernel: Convergence and Generalization in Neural Networks (Invited Paper)

Large Steps in Inverse Rendering of Geometry

Active Subdivision Surfaces for the Semiautomatic Segmentation of Biomedical Volumes

Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search

Convergence of adaptive algorithms for constrained weakly convex optimization

Estimation of discrete choice models with hybrid stochastic adaptive batch size algorithms

Optimization Over Banach Spaces: A Unified View on Supervised Learning and Inverse Problems

Large Steps in Inverse Rendering of Geometry

Estimation of discrete choice models with hybrid stochastic adaptive batch size algorithms

Adaptation in Stochastic Algorithms: From Nonsmooth Optimization to Min-Max Problems and Beyond

Convergence of adaptive algorithms for constrained weakly convex optimization

Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search

Neural Tangent Kernel: Convergence and Generalization in Neural Networks (Invited Paper)

Advances In Morphological Neural Networks: Training, Pruning And Enforcing Shape Constraints

Active Subdivision Surfaces for the Semiautomatic Segmentation of Biomedical Volumes

Improving the Training of Compact Neural Networks for Visual Recognition