Publications related to Automatic differentiation

Efficient local linearity regularization to overcome catastrophic overfitting

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Elias Abad Rocamora

Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-step AT, it has been observed that the loss function behaves locally linearly w ...

2024

Efficient Continual Finite-Sum Minimization

Volkan Cevher, Efstratios Panteleimon Skoulakis, Leello Tadesse Dadi

Given a sequence of functions

f_1,\ldots,f_n

with

f_i:\mathcal{D}\mapsto \mathbb{R}

, finite-sum minimization seeks a point

{x}^\star \in \mathcal{D}

minimizing

\sum_{j=1}^nf_j(x)/n

. In this work, we propose a key twist into the finite-sum minimizat ...

2024

Heterogeneity and developmental dynamics of mammalian neocortical progenitors

Gioele La Manno, Antonio Herrera Camacho, Leila Haj Abdullah Alieh

The central nervous system develops from a pool of neural progenitors which, depending on their location and time of division, generate cells committed to differentiate into specific kinds of neurons or glia. In the last decades, the developmental neurobio ...

Amsterdam2023

Robust Collaborative Learning with Linear Gradient Overhead

Rachid Guerraoui, Nirupam Gupta, John Stephan, Sadegh Farhadkhani, Le Nguyen Hoang, Rafaël Benjamin Pinot

Collaborative learning algorithms, such as distributed SGD (or D-SGD), are prone to faulty machines that may deviate from their prescribed algorithm because of software or hardware bugs, poisoned data or malicious behaviors. While many solutions have been ...

PLMR2023

Layer-Wise Learning Framework for Efficient DNN Deployment in Biomedical Wearable Systems

David Atienza Alonso, Amir Aminifar, Tomas Teijeiro Campo, Alireza Amirshahi, Saleh Baghersalimi

The development of low-power wearable systems requires specialized techniques to accommodate their unique requirements and constraints. While significant advancements have been made in the inference phase of artificial intelligence, the training phase rema ...

2023

Contextual Stochastic Bilevel Optimization

Daniel Kuhn, Andreas Krause, Yifan Hu, Jie Wang

We introduce contextual stochastic bilevel optimization (CSBO) -- a stochastic bilevel optimization framework with the lower-level problem minimizing an expectation conditioned on some contextual information and the upper-level decision variable. This fram ...

2023

Deep Learning for 3D Surface Modelling and Reconstruction

Benoît Alain René Guillard

In recent years, there has been a significant revolution in the field of deep learning, which has demonstrated its effectiveness in automatically capturing intricate patterns from large datasets. However, the majority of these successes in Computer Vision ...

EPFL2023

Constrained Form-Finding of Tension-Compression Structures using Automatic Differentiation

Stefana Parascho, Pierluigi D'Acunto

This paper proposes a computational approach to form-find pin-jointed bar structures subjected to combinations of tension and compression forces. The generated equilibrium states can meet structural and geometrical constraints via gradient-based optimizati ...

ELSEVIER SCI LTD2023

Autoencoders reloaded

Hervé Bourlard, Selen Hande Kabil

In Bourlard and Kamp (Biol Cybern 59(4):291-294, 1998), it was theoretically proven that autoencoders (AE) with single hidden layer (previously called "auto-associative multilayer perceptrons") were, in the best case, implementing singular value decomposit ...

SPRINGER2022

Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Jonathan Paul Sauder

Countless signal processing applications include the reconstruction of signals from few indirect linear measurements. The design of effective measurement operators is typically constrained by the underlying hardware and physics, posing a challenging and of ...

2022

Automatic differentiation

Graph Chatbot

Chat with Graph Search

Efficient local linearity regularization to overcome catastrophic overfitting

Efficient Continual Finite-Sum Minimization

Heterogeneity and developmental dynamics of mammalian neocortical progenitors

Robust Collaborative Learning with Linear Gradient Overhead

Layer-Wise Learning Framework for Efficient DNN Deployment in Biomedical Wearable Systems

Contextual Stochastic Bilevel Optimization

Deep Learning for 3D Surface Modelling and Reconstruction

Constrained Form-Finding of Tension-Compression Structures using Automatic Differentiation

Autoencoders reloaded

Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Heterogeneity and developmental dynamics of mammalian neocortical progenitors

Efficient Continual Finite-Sum Minimization

Efficient local linearity regularization to overcome catastrophic overfitting

Deep Learning for 3D Surface Modelling and Reconstruction

Layer-Wise Learning Framework for Efficient DNN Deployment in Biomedical Wearable Systems

Constrained Form-Finding of Tension-Compression Structures using Automatic Differentiation

Robust Collaborative Learning with Linear Gradient Overhead

Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Contextual Stochastic Bilevel Optimization

Autoencoders reloaded