Concept

Gradient

Related publications (90)

Efficient Continual Finite-Sum Minimization

Volkan Cevher, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi

Given a sequence of functions

f_1,\ldots,f_n

with

f_i:\mathcal{D}\mapsto \mathbb{R}

, finite-sum minimization seeks a point

{x}^\star \in \mathcal{D}

minimizing

\sum_{j=1}^nf_j(x)/n

. In this work, we propose a key twist into the finite-sum minimizat ...

2024

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov, Scott William Pesme

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

Rank-Adaptive Time Integration Of Tree Tensor Networks

Gianluca Ceruti

A rank-adaptive integrator for the approximate solution of high-order tensor differential equations by tree tensor networks is proposed and analyzed. In a recursion from the leaves to the root, the integrator updates bases and then evolves connection tenso ...

SIAM PUBLICATIONS2023

Phenomenological theory of variational quantum ground-state preparation

Ivano Tavernelli, Giuseppe Carleo

The variational approach is a cornerstone of computational physics, considering both conventional and quantum computing computational platforms. The variational quantum eigensolver algorithm aims to prepare the ground state of a Hamiltonian exploiting para ...

College Pk2023

Spatial symmetries in nonlocal multipolar metasurfaces

Olivier Martin, Karim Achouri, Ville Tuovi Tiukuvaara

We propose a framework that connects the spatial symmetries of a metasurface to its material parameter tensors and its scattering matrix. This provides a simple and universal way to effortlessly determine the properties of a metasurface scattering response ...

SPIE-SOC PHOTO-OPTICAL INSTRUMENTATION ENGINEERS2023

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Giancarlo Ferrari Trecate, Luca Furieri, Clara Lucía Galimberti, Liang Xu

Deep Neural Networks (DNNs) training can be difficult due to vanishing and exploding gradients during weight optimization through backpropagation. To address this problem, we propose a general class of Hamiltonian DNNs (H-DNNs) that stem from the discretiz ...

2023

Anomalies from the covariant derivative expansion

Timothy Cohen

We revisit the calculation of chiral anomalies for global and gauge symmetries in the framework of the covariant derivative expansion (CDE). Due to the presence of UV divergences, the result is an ambiguous quantity that depends on the regularization proce ...

College Pk2023

Implementation of multi-component Zhdanov closure in SOLEDGE3X

Davide Galassi, Madhusudan Raghunathan, Eric Serre

The multi-component fluid closure derived by Zhdanov (2002 Transport Processes in Multicomponent Plasma (London: Taylor and Francis)) is implemented in the fluid code SOLEDGE3X-EIRENE to deal with arbitrary edge plasma composition. The closure assumes no d ...

2022

A comprehensive framework for hard-magnetic beams: Reduced-order theory, 3D simulations, and experiments

Pedro Miguel Nunes Pereira de Almeida Reis, Dong Yan, Arefeh Abbasi

Thin beams made of magnetorheological elastomers embedded with hard-magnetic particles (hard-MREs) are capable of large deflections under an applied magnetic field. We propose a comprehensive framework, comprising a beam model and 3D finite element modelin ...

PERGAMON-ELSEVIER SCIENCE LTD2022

Masked Training of Neural Networks with Partial Gradients

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extra-gradient), limiting SGD updates to a sub ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.