Publication

Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention

Related publications (58)

About
Privacy
Disclaimer

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention

Graph Chatbot

Chat with Graph Search

Short-term energy use prediction of solar-assisted water heating system: Application case of combined attention-based LSTM and time-series decomposition

Supplementary Material - AL2: Progressive Activation Loss for Learning General Representations in Classification Neural Networks

Capsule networks as recurrent models of grouping and segmentation

Fast Transformers with Clustered Attention

Non-filamentary non-volatile memory elements as synapses in neuromorphic systems

Evaluating and Interpreting Deep Convolutional Neural Networks via Non-negative Matrix Factorization

Learning Trajectory Dependencies for Human Motion Prediction

A geometry-inspired decision-based attack

Implicit discourse relation classification with syntax-aware contextualized word representations

Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem