Concept

Time delay neural network

Related publications (29)

Linear Complexity Self-Attention With 3rd Order Polynomials

Self-attention mechanisms and non-local blocks have become crucial building blocks for state-of-the-art neural architectures thanks to their unparalleled ability in capturing long-range dependencies in the input. However their cost is quadratic with the nu ...

Ieee Computer Soc2023

Linear Complexity Self-Attention With 3rd Order Polynomials

Grigorios Chrysos, Filippos Kokkinos

Ieee Computer Soc2023

Towards Stable and Efficient Adversarial Training against $l_1$ Bounded Adversarial Attacks

Sabine Süsstrunk, Mathieu Salzmann, Yulun Jiang, Chen Liu, Zhuoyi Huang

We address the problem of stably and efficiently training a deep neural network robust to adversarial perturbations bounded by an

l_1

norm. We demonstrate that achieving robustness against

l_1

-bounded perturbations is more challenging than in the

l_2

...

2023

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

EPFL2023

NMDA-driven dendritic modulation enables multitask representation learning in hierarchical sensory processing pathways

Willem Anna Mark Wybo, Bernd Albert Illing, Matthias Chinyen Tsai, Walter Senn

While sensory representations in the brain depend on context, it remains unclear how such modulations are implemented at the biophysical level, and how processing layers further in the hierarchy can extract useful features for each possible contex-tual sta ...

National Academy of Sciences2023

Estimating and Improving the Robustness of Attributions in Text

Ádám Dániel Ivánkay

End-to-end learning methods like deep neural networks have been the driving force in the remarkable progress of machine learning in recent years. However, despite their success, the deployment process of such networks in safety-critical use cases, such as ...

EPFL2023

De novo protein design by inversion of the AlphaFold structure prediction network

Bruno Emanuel Ferreira De Sousa Correia, Hamed Khakzad, Casper Alexander Goverde, Stéphane Rosset, Benedict Dieter Gregor Wolf

De novo protein design enhances our understanding of the principles that govern protein folding and interactions, and has the potential to revolutionize biotechnology through the engineering of novel protein functionalities. Despite recent progress in comp ...

2023

Sharp asymptotics on the compression of two-layer neural networks

Marco Mondelli

In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M < N nodes. More precisely, we consider the setting in which the weights of the target network are i.i.d. sub-Gaussian, and we minimiz ...

IEEE2022

Improving the Training of Compact Neural Networks for Visual Recognition

Shuxuan Guo

During the Artificial Intelligence (AI) revolution of the past decades, deep neural networks have been widely used and have achieved tremendous success in visual recognition. Unfortunately, deploying deep models is challenging because of their huge model s ...

EPFL2022

Assessment of idiopathic inflammatory myopathy using a deep learning method for muscle T2 mapping segmentation

Tobias Kober, Qian Wang, Tom Hilbert, Yao Zhang, Yubo Zhao

Objective To investigate the utility of an automatic deep learning (DL) method for segmentation of T2 maps in patients with idiopathic inflammatory myopathy (IIM) against healthy controls, and also the association of quantitative T2 values in patients with ...

SPRINGER2022

Towards Verifiable, Generalizable and Efficient Robust Deep Neural Networks

Chen Liu

In the last decade, deep neural networks have achieved tremendous success in many fields of machine learning. However, they are shown vulnerable against adversarial attacks: well-designed, yet imperceptible, perturbations can make the state-of-the-art deep ...

EPFL2022

Towards Stable and Efficient Adversarial Training against $l_1$ Bounded Adversarial Attacks

Sabine Süsstrunk, Mathieu Salzmann, Yulun Jiang, Chen Liu, Zhuoyi Huang

We address the problem of stably and efficiently training a deep neural network robust to adversarial perturbations bounded by an

l_1

norm. We demonstrate that achieving robustness against

l_1

-bounded perturbations is more challenging than in the

l_2

...

2023