Concept

Méthode de Newton

Publications associées (32)

Pareto-like sequential sampling heuristic for global optimisation

In this paper, we propose a simple global optimisation algorithm inspired by Pareto’s principle. This algorithm samples most of its solutions within prominent search domains and is equipped with a self-adaptive mechanism to control the dynamic tightening o ...

2021

Efficient second-order methods for model compression

Sidak Pal Singh

Second-order information, in the form of Hessian- or Inverse-Hessian-vector products, is a fundamental tool for solving optimization problems. Recently, there has been a tremendous amount of work on utilizing this information for the current compute and me ...

2020

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Florent Gérard Krzakala, Lenka Zdeborová

Despite the widespread use of gradient-based algorithms for optimizing high-dimensional non-convex functions, understanding their ability of finding good minima instead of being trapped in spurious ones remains to a large extent an open problem. Here we fo ...

Curran Associates, Inc.2020

Adaptive Gradient Descent without Descent

Konstantin Mishchenko

We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don’t increase the stepsize too fast and 2) don’t overstep the local curvature. No need for functional values, no line search, no information about the func ...

2020

Vulnérabilité sismique à grande échelle de la ville de Lausanne

Loris Silva Brites

The assessment of the risk maps for the seismic vulnerability at large scale is based on the vulnerability of each building. In order to determine these vulnerabilities, it is first required to assign to each building its construction class. The constructi ...

2020

Adaptation and Learning Over Networks Under Subspace Constraints-Part I: Stability Analysis

Ali H. Sayed, Stefan Vlaski, Roula Nassif

This paper considers optimization problems over networks where agents have individual objectives to meet, or individual parameter vectors to estimate, subject to subspace constraints that require the objectives across the network to lie in low-dimensional ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Compress-and-restart block Krylov subspace methods for Sylvester matrix equations

Daniel Kressner, Stefano Massei, Kathryn Dianne Lund

Block Krylov subspace methods (KSMs) comprise building blocks in many state-of-the-art solvers for large-scale matrix equations as they arise, for example, from the discretization of partial differential equations. While extended and rational block Krylov ...

2020

On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems

Volkan Cevher, Ali Kavis, Shaul Nadav Hallak

This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm’s convergence properties in non-convex problems. We first show that the sequence of iterates generated by SGD remains bounded and converges with prob ...

2020

The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Updates

Sebastian Urban Stich, Sai Praneeth Reddy Karimireddy

We analyze (stochastic) gradient descent (SGD) with delayed updates on smooth quasi-convex and non-convex functions and derive concise, non-asymptotic, convergence rates. We show that the rate of convergence in all cases consists of two terms: (i) a stocha ...

MICROTOME PUBL2020

On the choice of metric in gradient-based theories of brain function

Wulfram Gerstner, Johanni Michael Brea, Jean-Pascal Théodor Pfister

The idea that the brain functions so as to minimize certain costs pervades theoretical neuroscience. Because a cost function by itself does not predict how the brain finds its minima, additional assumptions about the optimization method need to be made to ...

2020

Graph Chatbot

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Connectez-vous pour utiliser Chat avec Graph Search