Concept

Algorithme du gradient stochastique

Publications associées (431)

Linear Speedup In Saddle-Point Escape For Decentralized Non-Convex Optimization

Under appropriate cooperation protocols and parameter choices, fully decentralized solutions for stochastic optimization have been shown to match the performance of centralized solutions and result in linear speedup (in the number of agents) relative to no ...

IEEE2020

Inverse design and demonstration of high-performance wide-angle diffractive optical elements

Toralf Scharf, Dong Cheon Kim

Diffractive optical elements are ultra-thin optical components required for constructing very compact optical 3D sensors. However, the required wide-angle diffractive 2D fan-out gratings have been elusive due to design challenges. Here, we introduce a new ...

OPTICAL SOC AMER2020

Optimizer Benchmarking Needs to Account for Hyperparameter Tuning

François Fleuret, Martin Jaggi, Thijs Vogels, Jan Frederik Jonas Florian Mai, Prabhu Teja Sivaprasad

The performance of optimizers, particularly in deep learning, depends considerably on their chosen hyperparameter configuration. The efficacy of optimizers is often studied under near-optimal problem-specific hyperparameters, and finding these settings may ...

2020

A Shared-Memory Parallel Implementation of the RePlAce Global Cell Placer

Mirjana Stojilovic, Philip Brisk, Frédéric Gessler

RePlAce is a state-of-the-art prototype of a flat, analytic, and nonlinear global cell placement algorithm, which models a placement instance as an electrostatic system with positively charged objects. It can handle large-scale standard-cell and mixed-cell ...

IEEE2020

Linear convergence of primal-dual gradient methods and their performance in distributed optimization

Ali H. Sayed, Sulaiman A S A E Alghunaim

In this work, we revisit a classical incremental implementation of the primal-descent dual-ascent gradient method used for the solution of equality constrained optimization problems. We provide a short proof that establishes the linear (exponential) conver ...

PERGAMON-ELSEVIER SCIENCE LTD2020

FEM Based Statistical Data-Driven Modeling Approach for MFT Design Optimization

Drazen Dujic, Marko Mogorovic

This paper proposes a novel class of neural-network inspired statistical data-driven models, especially derived for the purpose of design optimization of medium frequency transformers. These models allow for an efficient (3-5 orders of magnitude faster com ...

2020

Local SGD Converges Fast and Communicates Little

Sebastian Urban Stich

Mini-batch stochastic gradient descent (SGD) is state of the art in large scale distributed training. The scheme can reach a linear speedup with respect to the number of workers, but this is rarely seen in practice as the scheme often suffers from large ne ...

2019

Stochastic Gradient Descent for Spectral Embedding with Implicit Orthogonality Constraint

Pascal Frossard, Mireille El Gheche, Giovanni Chierchia

In this paper, we propose a scalable algorithm for spectral embedding. The latter is a standard tool for graph clustering. However, its computational bottleneck is the eigendecomposition of the graph Laplacian matrix, which prevents its application to larg ...

IEEE2019

Stochastic Optimization with Adaptive Batch Size: Discrete Choice Models as a Case Study

Michel Bierlaire, Timothy Michael Hillel, Virginie Janine Camille Lurkin, Gael Lederrey

The 2.5 quintillion bytes of data created each day brings new opportunities, but also new stimulating challenges for the discrete choice community. Opportunities because more and more new and larger data sets will undoubtedly become available in the future ...

2019

Stochastic Learning Under Random Reshuffling With Constant Step-Sizes

Ali H. Sayed, Stefan Vlaski, Bicheng Ying, Kun Yuan

In empirical risk optimization, it has been observed that stochastic gradient implementations that rely on random reshuffling of the data achieve better performance than implementations that rely on sampling the data uniformly. Recent works have pursued ju ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Graph Chatbot

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Connectez-vous pour utiliser Chat avec Graph Search