Publications associées à Gradient

Rank-Adaptive Time Integration Of Tree Tensor Networks

A rank-adaptive integrator for the approximate solution of high-order tensor differential equations by tree tensor networks is proposed and analyzed. In a recursion from the leaves to the root, the integrator updates bases and then evolves connection tenso ...

SIAM PUBLICATIONS2023

Phenomenological theory of variational quantum ground-state preparation

Ivano Tavernelli, Giuseppe Carleo

The variational approach is a cornerstone of computational physics, considering both conventional and quantum computing computational platforms. The variational quantum eigensolver algorithm aims to prepare the ground state of a Hamiltonian exploiting para ...

Amer Physical Soc2023

Spatial symmetries in nonlocal multipolar metasurfaces

Olivier Martin, Karim Achouri, Ville Tuovi Tiukuvaara

We propose a framework that connects the spatial symmetries of a metasurface to its material parameter tensors and its scattering matrix. This provides a simple and universal way to effortlessly determine the properties of a metasurface scattering response ...

SPIE-SOC PHOTO-OPTICAL INSTRUMENTATION ENGINEERS2023

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Giancarlo Ferrari Trecate, Luca Furieri, Clara Lucía Galimberti, Liang Xu

Deep Neural Networks (DNNs) training can be difficult due to vanishing and exploding gradients during weight optimization through backpropagation. To address this problem, we propose a general class of Hamiltonian DNNs (H-DNNs) that stem from the discretiz ...

2023

Robust Collaborative Learning with Linear Gradient Overhead

Rachid Guerraoui, John Stephan, Sadegh Farhadkhani, Le Nguyen Hoang, Nirupam Gupta, Rafaël Benjamin Pinot

Collaborative learning algorithms, such as distributed SGD (or D-SGD), are prone to faulty machines that may deviate from their prescribed algorithm because of software or hardware bugs, poisoned data or malicious behaviors. While many solutions have been ...

PLMR2023

Implementation of multi-component Zhdanov closure in SOLEDGE3X

Davide Galassi, Madhusudan Raghunathan, Eric Serre

The multi-component fluid closure derived by Zhdanov (2002 Transport Processes in Multicomponent Plasma (London: Taylor and Francis)) is implemented in the fluid code SOLEDGE3X-EIRENE to deal with arbitrary edge plasma composition. The closure assumes no d ...

2022

A comprehensive framework for hard-magnetic beams: Reduced-order theory, 3D simulations, and experiments

Pedro Miguel Nunes Pereira de Almeida Reis, Dong Yan, Arefeh Abbasi

Thin beams made of magnetorheological elastomers embedded with hard-magnetic particles (hard-MREs) are capable of large deflections under an applied magnetic field. We propose a comprehensive framework, comprising a beam model and 3D finite element modelin ...

PERGAMON-ELSEVIER SCIENCE LTD2022

Masked Training of Neural Networks with Partial Gradients

Martin Jaggi, Amirkeivan Mohtashami, Sebastian Urban Stich

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extra-gradient), limiting SGD updates to a sub ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

An Accelerated First-Order Method for Non-convex Optimization on Manifolds

Nicolas Boumal, Christopher Arnold Criscitiello

We describe the first gradient methods on Riemannian manifolds to achieve accelerated rates in the non-convex case. Under Lipschitz assumptions on the Riemannian gradient and Hessian of the cost function, these methods find approximate first-order critical ...

SPRINGER2022

Towards Understanding Sharpness-Aware Minimization

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko

Sharpness-Aware Minimization (SAM) is a recent training method that relies on worst-case weight perturbations which significantly improves generalization in various settings. We argue that the existing justifications for the success of SAM which are based ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022