Publications by Jean-Baptiste Francis Marie Juliette Cordonnier

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

EPFL2023

Point-of-care lung ultrasonography for early identification of mild COVID-19: a prospective cohort of outpatients in a Swiss screening center

Mary-Anne Hartley, Jean-Baptiste Francis Marie Juliette Cordonnier, Jean-Yves Meuwly, Luca Bosso, Tanguy Espejo, Siméon Schaad

Objectives Early identification of SARS-CoV-2 infection is important to guide quarantine and reduce transmission. This study evaluates the diagnostic performance of lung ultrasound (LUS), an affordable, consumable-free point-of-care tool, for COVID-19 scre ...

BMJ PUBLISHING GROUP2022

Attention is not all you need: pure attention loses rank doubly exponentially with depth

Andreas Loukas, Jean-Baptiste Francis Marie Juliette Cordonnier

Attention-based architectures have become ubiquitous in machine learning. Yet, our understanding of the reasons for their effectiveness remains limited. This work proposes a new way to understand self-attention networks: we show that their output can be de ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Differentiable Patch Selection for Image Recognition

Jean-Baptiste Francis Marie Juliette Cordonnier, Aravindh Mahendran

Neural Networks require large amounts of memory and compute to process high resolution images, even when only a small part of the image is actually informative for the task at hand. We propose a method based on a differentiable Top-K operator to select the ...

IEEE COMPUTER SOC2021

On the Relationship between Self-Attention and Convolutional Layers

Martin Jaggi, Jean-Baptiste Francis Marie Juliette Cordonnier, Andreas Loukas

Recent trends of incorporating attention mechanisms in vision have led re- searchers to reconsider the supremacy of convolutional layers as a primary build- ing block. Beyond helping CNNs to handle long-range dependencies, Ramachandran et al. (2019) showed ...

2020

Extrapolating Paths with Graph Neural Networks

Andreas Loukas, Jean-Baptiste Francis Marie Juliette Cordonnier

We consider the problem of path inference: given a path prefix, i.e., a partially observed sequence of nodes in a graph, we want to predict which nodes are in the missing suffix. In particular, we focus on natural paths occurring as a by-product of the int ...

IJCAI, Inc.2019

Sparsified SGD with Memory

Martin Jaggi, Sebastian Urban Stich, Jean-Baptiste Francis Marie Juliette Cordonnier

Huge scale machine learning problems are nowadays tackled by distributed optimization algorithms, i.e. algorithms that leverage the compute power of many devices for training. The communication overhead is a key bottleneck that hinders perfect scalability. ...

2018