Publications associées à Dérivation automatique

Efficient local linearity regularization to overcome catastrophic overfitting

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Elias Abad Rocamora

Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-step AT, it has been observed that the loss function behaves locally linearly w ...

2024

Heterogeneity and developmental dynamics of mammalian neocortical progenitors

Gioele La Manno, Antonio Herrera Camacho, Leila Haj Abdullah Alieh

The central nervous system develops from a pool of neural progenitors which, depending on their location and time of division, generate cells committed to differentiate into specific kinds of neurons or glia. In the last decades, the developmental neurobio ...

Elsevier2023

Robust Collaborative Learning with Linear Gradient Overhead

Rachid Guerraoui, John Stephan, Sadegh Farhadkhani, Le Nguyen Hoang, Nirupam Gupta, Rafaël Benjamin Pinot

Collaborative learning algorithms, such as distributed SGD (or D-SGD), are prone to faulty machines that may deviate from their prescribed algorithm because of software or hardware bugs, poisoned data or malicious behaviors. While many solutions have been ...

PLMR2023

Layer-Wise Learning Framework for Efficient DNN Deployment in Biomedical Wearable Systems

David Atienza Alonso, Amir Aminifar, Tomas Teijeiro Campo, Alireza Amirshahi, Saleh Baghersalimi

The development of low-power wearable systems requires specialized techniques to accommodate their unique requirements and constraints. While significant advancements have been made in the inference phase of artificial intelligence, the training phase rema ...

2023

Contextual Stochastic Bilevel Optimization

Daniel Kuhn, Yifan Hu, Andreas Krause, Jie Wang

We introduce contextual stochastic bilevel optimization (CSBO) -- a stochastic bilevel optimization framework with the lower-level problem minimizing an expectation conditioned on some contextual information and the upper-level decision variable. This fram ...

2023

Deep Learning for 3D Surface Modelling and Reconstruction

Benoît Alain René Guillard

In recent years, there has been a significant revolution in the field of deep learning, which has demonstrated its effectiveness in automatically capturing intricate patterns from large datasets. However, the majority of these successes in Computer Vision ...

EPFL2023

Constrained Form-Finding of Tension-Compression Structures using Automatic Differentiation

Stefana Parascho, Pierluigi D'Acunto

This paper proposes a computational approach to form-find pin-jointed bar structures subjected to combinations of tension and compression forces. The generated equilibrium states can meet structural and geometrical constraints via gradient-based optimizati ...

ELSEVIER SCI LTD2023

Autoencoders reloaded

Hervé Bourlard, Selen Hande Kabil

In Bourlard and Kamp (Biol Cybern 59(4):291-294, 1998), it was theoretically proven that autoencoders (AE) with single hidden layer (previously called "auto-associative multilayer perceptrons") were, in the best case, implementing singular value decomposit ...

SPRINGER2022

Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Jonathan Paul Sauder

Countless signal processing applications include the reconstruction of signals from few indirect linear measurements. The design of effective measurement operators is typically constrained by the underlying hardware and physics, posing a challenging and of ...

2022

Continual Test-Time Domain Adaptation

Olga Fink, Qin Wang

Test-time domain adaptation aims to adapt a source pretrained model to a target domain without using any source data. Existing works mainly consider the case where the target domain is static. However, real-world machine perception systems are running in n ...

IEEE COMPUTER SOC2022