Publications associées à Dérivation automatique

Efficient local linearity regularization to overcome catastrophic overfitting

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Elias Abad Rocamora

Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-step AT, it has been observed that the loss function behaves locally linearly w ...

2024

Efficient Continual Finite-Sum Minimization

Volkan Cevher, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi

Given a sequence of functions

f_1,\ldots,f_n

with

f_i:\mathcal{D}\mapsto \mathbb{R}

, finite-sum minimization seeks a point

{x}^\star \in \mathcal{D}

minimizing

\sum_{j=1}^nf_j(x)/n

. In this work, we propose a key twist into the finite-sum minimizat ...

2024

Heterogeneity and developmental dynamics of mammalian neocortical progenitors

Gioele La Manno, Antonio Herrera Camacho, Leila Haj Abdullah Alieh

The central nervous system develops from a pool of neural progenitors which, depending on their location and time of division, generate cells committed to differentiate into specific kinds of neurons or glia. In the last decades, the developmental neurobio ...

Amsterdam2023

Robust Collaborative Learning with Linear Gradient Overhead

Rachid Guerraoui, Nirupam Gupta, John Stephan, Sadegh Farhadkhani, Le Nguyen Hoang, Rafaël Benjamin Pinot

Collaborative learning algorithms, such as distributed SGD (or D-SGD), are prone to faulty machines that may deviate from their prescribed algorithm because of software or hardware bugs, poisoned data or malicious behaviors. While many solutions have been ...

PLMR2023

Layer-Wise Learning Framework for Efficient DNN Deployment in Biomedical Wearable Systems

David Atienza Alonso, Amir Aminifar, Tomas Teijeiro Campo, Alireza Amirshahi, Saleh Baghersalimi

The development of low-power wearable systems requires specialized techniques to accommodate their unique requirements and constraints. While significant advancements have been made in the inference phase of artificial intelligence, the training phase rema ...

2023

Contextual Stochastic Bilevel Optimization

Daniel Kuhn, Andreas Krause, Yifan Hu, Jie Wang

We introduce contextual stochastic bilevel optimization (CSBO) -- a stochastic bilevel optimization framework with the lower-level problem minimizing an expectation conditioned on some contextual information and the upper-level decision variable. This fram ...

2023

Constrained Form-Finding of Tension-Compression Structures using Automatic Differentiation

Stefana Parascho, Pierluigi D'Acunto

This paper proposes a computational approach to form-find pin-jointed bar structures subjected to combinations of tension and compression forces. The generated equilibrium states can meet structural and geometrical constraints via gradient-based optimizati ...

ELSEVIER SCI LTD2023

Autoencoders reloaded

Hervé Bourlard, Selen Hande Kabil

In Bourlard and Kamp (Biol Cybern 59(4):291-294, 1998), it was theoretically proven that autoencoders (AE) with single hidden layer (previously called "auto-associative multilayer perceptrons") were, in the best case, implementing singular value decomposit ...

SPRINGER2022

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Efficient Continual Finite-Sum Minimization

Volkan Cevher, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi

Given a sequence of functions

f_1,\ldots,f_n

with

f_i:\mathcal{D}\mapsto \mathbb{R}

, finite-sum minimization seeks a point

{x}^\star \in \mathcal{D}

minimizing

\sum_{j=1}^nf_j(x)/n

. In this work, we propose a key twist into the finite-sum minimizat ...

2024

Dérivation automatique

Efficient local linearity regularization to overcome catastrophic overfitting

Efficient Continual Finite-Sum Minimization

Heterogeneity and developmental dynamics of mammalian neocortical progenitors

Robust Collaborative Learning with Linear Gradient Overhead

Layer-Wise Learning Framework for Efficient DNN Deployment in Biomedical Wearable Systems

Contextual Stochastic Bilevel Optimization

Deep Learning for 3D Surface Modelling and Reconstruction

Constrained Form-Finding of Tension-Compression Structures using Automatic Differentiation

Autoencoders reloaded

Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Graph Chatbot

Chattez avec Graph Search

Efficient Continual Finite-Sum Minimization

Efficient local linearity regularization to overcome catastrophic overfitting

Heterogeneity and developmental dynamics of mammalian neocortical progenitors

Layer-Wise Learning Framework for Efficient DNN Deployment in Biomedical Wearable Systems

Contextual Stochastic Bilevel Optimization

Autoencoders reloaded

Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Constrained Form-Finding of Tension-Compression Structures using Automatic Differentiation

Robust Collaborative Learning with Linear Gradient Overhead