Publications related to Backpropagation

Machine Learning Methods for Robust Uncertainty Quantification and Controller Approximation

This thesis is situated at the crossroads between machine learning and control engineering. Our contributions are both theoretical, through proposing a new uncertainty quantification methodology in a kernelized context; and experimental, through investigat ...

EPFL2023

NMDA-driven dendritic modulation enables multitask representation learning in hierarchical sensory processing pathways

Willem Anna Mark Wybo, Bernd Albert Illing, Matthias Chinyen Tsai, Walter Senn

While sensory representations in the brain depend on context, it remains unclear how such modulations are implemented at the biophysical level, and how processing layers further in the hierarchy can extract useful features for each possible contex-tual sta ...

Washington2023

An exact mapping from ReLU networks to spiking neural networks

Wulfram Gerstner, Stanislaw Andrzej Wozniak, Ana Stanojevic, Giovanni Cherubini, Angeliki Pantazi

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an ...

2023

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Michaël Unser

The minimization of a data-fidelity term and an additive regularization functional gives rise to a powerful framework for supervised learning. In this paper, we present a unifying regularization functional that depends on an operator L\documentclass[12pt]{ ...

New York2023

Probabilistic methods for neural combinatorial optimization

Nikolaos Karalias

The monumental progress in the development of machine learning models has led to a plethora of applications with transformative effects in engineering and science. This has also turned the attention of the research community towards the pursuit of construc ...

EPFL2023

Towards Stable and Efficient Adversarial Training against $l_1$ Bounded Adversarial Attacks

Sabine Süsstrunk, Mathieu Salzmann, Yulun Jiang, Chen Liu, Zhuoyi Huang

We address the problem of stably and efficiently training a deep neural network robust to adversarial perturbations bounded by an

l_1

norm. We demonstrate that achieving robustness against

l_1

-bounded perturbations is more challenging than in the

l_2

...

2023

Fast Adversarial Training With Adaptive Step Size

Sabine Süsstrunk, Mathieu Salzmann, Chen Liu, Zhuoyi Huang, Yong Zhang, Jue Wang

While adversarial training and its variants have shown to be the most effective algorithms to defend against adversarial attacks, their extremely slow training process makes it hard to scale to large datasets like ImageNet. The key idea of recent works to ...

Piscataway2023

New Perspectives on Regularization and Computation in Optimal Transport-Based Distributionally Robust Optimization

Daniel Kuhn, Florian Dörfler, Soroosh Shafieezadeh Abadeh

We study optimal transport-based distributionally robust optimization problems where a fictitious adversary, often envisioned as nature, can choose the distribution of the uncertain problem parameters by reshaping a prescribed reference distribution at a f ...

2023

Approximating Relu Networks By Single-Spike Computation

Wulfram Gerstner, Stanislaw Andrzej Wozniak, Ana Stanojevic, Evangelos Eleftheriou, Giovanni Cherubini, Angeliki Pantazi

Developing energy-saving neural network models is a topic of rapidly increasing interest in the artificial intelligence community. Spiking neural networks (SNNs) are biologically inspired models that strive to leverage the energy efficiency stemming from a ...

IEEE2022

Adversarial training with informed data selection

Pascal Frossard, Javier Alejandro Maroto Morales, Marcele Oliveira Kuhfuss de Mendonça

With the increasing amount of available data and advances in computing capabilities, deep neural networks (DNNs) have been successfully employed to solve challenging tasks in various areas, including healthcare, climate, and finance. Nevertheless, state-of ...

IEEE2022

Backpropagation

Graph Chatbot

Chat with Graph Search

Machine Learning Methods for Robust Uncertainty Quantification and Controller Approximation

NMDA-driven dendritic modulation enables multitask representation learning in hierarchical sensory processing pathways

An exact mapping from ReLU networks to spiking neural networks

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Probabilistic methods for neural combinatorial optimization

Towards Stable and Efficient Adversarial Training against $l_1$ Bounded Adversarial Attacks

Fast Adversarial Training With Adaptive Step Size

New Perspectives on Regularization and Computation in Optimal Transport-Based Distributionally Robust Optimization

Approximating Relu Networks By Single-Spike Computation

Adversarial training with informed data selection

Adversarial training with informed data selection

NMDA-driven dendritic modulation enables multitask representation learning in hierarchical sensory processing pathways

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Machine Learning Methods for Robust Uncertainty Quantification and Controller Approximation

Fast Adversarial Training With Adaptive Step Size

Approximating Relu Networks By Single-Spike Computation

An exact mapping from ReLU networks to spiking neural networks

Towards Stable and Efficient Adversarial Training against $l_1$ Bounded Adversarial Attacks

Probabilistic methods for neural combinatorial optimization

New Perspectives on Regularization and Computation in Optimal Transport-Based Distributionally Robust Optimization