Concept
Dérivation automatique
Publications associées (41)
Efficient local linearity regularization to overcome catastrophic overfitting
Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Elias Abad Rocamora
Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-step AT, it has been observed that the loss function behaves locally linearly w ...
2024Robust Collaborative Learning with Linear Gradient Overhead
Rachid Guerraoui, Nirupam Gupta, John Stephan, Sadegh Farhadkhani, Le Nguyen Hoang, Rafaël Benjamin Pinot
Collaborative learning algorithms, such as distributed SGD (or D-SGD), are prone to faulty machines that may deviate from their prescribed algorithm because of software or hardware bugs, poisoned data or malicious behaviors. While many solutions have been ...
PLMR2023Layer-Wise Learning Framework for Efficient DNN Deployment in Biomedical Wearable Systems
David Atienza Alonso, Amir Aminifar, Tomas Teijeiro Campo, Alireza Amirshahi, Saleh Baghersalimi
The development of low-power wearable systems requires specialized techniques to accommodate their unique requirements and constraints. While significant advancements have been made in the inference phase of artificial intelligence, the training phase rema ...
2023Contextual Stochastic Bilevel Optimization
Daniel Kuhn, Andreas Krause, Yifan Hu, Jie Wang
We introduce contextual stochastic bilevel optimization (CSBO) -- a stochastic bilevel optimization framework with the lower-level problem minimizing an expectation conditioned on some contextual information and the upper-level decision variable. This fram ...
2023