Publications related to Time delay neural network

Intraday solar irradiance forecasting using public cameras

Demetri Psaltis, Mario Paolone, Christophe Moser, Luisa Lambertini

With the significant increase in photovoltaic (PV) electricity generation, more attention has been given to PV power forecasting. Indeed, accurate forecasting allows power grid operators to better schedule and dispatch their assets, such as energy storage ...

Pergamon-Elsevier Science Ltd2024

Linear Complexity Self-Attention With 3rd Order Polynomials

Grigorios Chrysos, Filippos Kokkinos

Self-attention mechanisms and non-local blocks have become crucial building blocks for state-of-the-art neural architectures thanks to their unparalleled ability in capturing long-range dependencies in the input. However their cost is quadratic with the nu ...

Los Alamitos2023

Towards Stable and Efficient Adversarial Training against $l_1$ Bounded Adversarial Attacks

Sabine Süsstrunk, Mathieu Salzmann, Yulun Jiang, Chen Liu, Zhuoyi Huang

We address the problem of stably and efficiently training a deep neural network robust to adversarial perturbations bounded by an

l_1

norm. We demonstrate that achieving robustness against

l_1

-bounded perturbations is more challenging than in the

l_2

...

2023

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins.All these successful methods depend on a gradient-based learning algorithm to train a model on massive a ...

EPFL2023

NMDA-driven dendritic modulation enables multitask representation learning in hierarchical sensory processing pathways

Willem Anna Mark Wybo, Bernd Albert Illing, Matthias Chinyen Tsai, Walter Senn

While sensory representations in the brain depend on context, it remains unclear how such modulations are implemented at the biophysical level, and how processing layers further in the hierarchy can extract useful features for each possible contex-tual sta ...

Washington2023

Estimating and Improving the Robustness of Attributions in Text

Ádám Dániel Ivánkay

End-to-end learning methods like deep neural networks have been the driving force in the remarkable progress of machine learning in recent years. However, despite their success, the deployment process of such networks in safety-critical use cases, such as ...

EPFL2023

De novo protein design by inversion of the AlphaFold structure prediction network

Bruno Emanuel Ferreira De Sousa Correia, Hamed Khakzad, Casper Alexander Goverde, Stéphane Rosset, Benedict Dieter Gregor Wolf

De novo protein design enhances our understanding of the principles that govern protein folding and interactions, and has the potential to revolutionize biotechnology through the engineering of novel protein functionalities. Despite recent progress in comp ...

2023

An exact mapping from ReLU networks to spiking neural networks

Wulfram Gerstner, Stanislaw Andrzej Wozniak, Ana Stanojevic, Giovanni Cherubini, Angeliki Pantazi

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an ...

2023

Sharp asymptotics on the compression of two-layer neural networks

Marco Mondelli

In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M < N nodes. More precisely, we consider the setting in which the weights of the target network are i.i.d. sub-Gaussian, and we minimiz ...

IEEE2022

Improving the Training of Compact Neural Networks for Visual Recognition

Shuxuan Guo

During the Artificial Intelligence (AI) revolution of the past decades, deep neural networks have been widely used and have achieved tremendous success in visual recognition. Unfortunately, deploying deep models is challenging because of their huge model s ...

EPFL2022

Time delay neural network

Graph Chatbot

Chat with Graph Search

Intraday solar irradiance forecasting using public cameras

Linear Complexity Self-Attention With 3rd Order Polynomials

Towards Stable and Efficient Adversarial Training against $l_1$ Bounded Adversarial Attacks

Transformer Models for Vision

NMDA-driven dendritic modulation enables multitask representation learning in hierarchical sensory processing pathways

Estimating and Improving the Robustness of Attributions in Text

De novo protein design by inversion of the AlphaFold structure prediction network

An exact mapping from ReLU networks to spiking neural networks

Sharp asymptotics on the compression of two-layer neural networks

Improving the Training of Compact Neural Networks for Visual Recognition

Intraday solar irradiance forecasting using public cameras

Linear Complexity Self-Attention With 3rd Order Polynomials

Towards Stable and Efficient Adversarial Training against $l_1$ Bounded Adversarial Attacks

Transformer Models for Vision

NMDA-driven dendritic modulation enables multitask representation learning in hierarchical sensory processing pathways

Estimating and Improving the Robustness of Attributions in Text

De novo protein design by inversion of the AlphaFold structure prediction network

An exact mapping from ReLU networks to spiking neural networks

Sharp asymptotics on the compression of two-layer neural networks

Improving the Training of Compact Neural Networks for Visual Recognition