Publications de Angelos Katharopoulos

Stop Wasting my FLOPS: Improving the Efficiency of Deep Learning Models

Deep neural networks have completely revolutionized the field of machine learning by achieving state-of-the-art results on various tasks ranging from computer vision to protein folding. However, their application is hindered by their large computational an ...

EPFL2022

Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks

Angelos Katharopoulos, Andreas Geiger, Despoina Paschalidou

Impressive progress in 3D shape extraction led to representations that can capture object geometries with high fidelity. In parallel, primitive-based methods seek to represent objects as semantically consistent part arrangements. However, due to the simpli ...

IEEE COMPUTER SOC2021

Fast Transformers with Clustered Attention

François Fleuret, Angelos Katharopoulos, Apoorv Vyas

Transformers have been proven a successful model for a variety of tasks in sequence modeling. However, computing the attention matrix, which is their key component, has quadratic complexity with respect to the sequence length, thus making them prohibitivel ...

2020

Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention

François Fleuret, Angelos Katharopoulos

Transformers achieve remarkable performance in several tasks but due to their quadratic complexity, with respect to the input's length, they are prohibitively slow for very long sequences. To address this limitation, we express the self-attention as a line ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2020