Publications by Martin Jaggi | EPFL Graph Search

Attention with Markov: A Curious Case of Single-layer Transformers

Michael Christoph Gastpar, Martin Jaggi, Marco Bondaschi

Attention-based transformers have achieved tremendous success across a variety of disciplines including natural languages. To deepen our understanding of their sequential modeling capabilities, there is a growing interest in using Markov input processes to ...

ICLR2025

CoBo: Collaborative Learning via Bilevel Optimization

Martin Jaggi, Lie He

Collaborative learning is an important tool to train multiple clients more effectively by enabling communication among clients. Identifying helpful clients, however, presents challenging and often introduces significant overhead. In this paper, we model cl ...

2024

The Privacy Power of Correlated Noise in Decentralized Learning

Rachid Guerraoui, Martin Jaggi, Youssef Allouah, Anastasiia Koloskova, Aymane El Firdoussi

Decentralized learning is appealing as it enables the scalable usage of large amounts of distributed data and resources (without resorting to any central entity), while promoting privacy since every user minimizes the direct exposure of their data. Yet, wi ...

PMLR2024

MultiModN- Multimodal, Multi-Task, Interpretable Modular Networks

Martin Jaggi, Mary-Anne Hartley, Vinitra Swamy, Thierry Bossy, Tanja Christina Käser Jacober, Thijs Vogels, Malika Satayeva, Jibril Albachir Frej

Predicting multiple real-world tasks in a single model often requires a particularly diverse feature space. Multimodal (MM) models aim to extract the synergistic predictive potential of multiple data types to create a shared feature space with aligned sema ...

2023

Beyond Spectral Gap: The Role of the Topology in Decentralized Learning

Martin Jaggi, Thijs Vogels, Hadrien Hendrikx

In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rates and optimize faster. In the decentralized setting, in which workers commu ...

Microtome Publishing2023

Modular Clinical Decision Support Networks (MoDN)-Updatable, interpretable, and portable predictions for evolving clinical environments

Martin Jaggi, Mary-Anne Hartley, Thijs Vogels

Clinical Decision Support Systems (CDSS) have the potential to improve and standardise care with probabilistic guidance. However, many CDSS deploy static, generic rule-based logic, resulting in inequitably distributed accuracy and inconsistent performance ...

PUBLIC LIBRARY SCIENCE2023

SIMSUM: Document-level Text Simplification via Simultaneous Summarization

Martin Jaggi

Document-level text simplification is a specific type of simplification which involves simplifying documents consisting of several sentences by rewriting them into fewer or more sentences. In this paper, we propose a new two-stage framework SIMSUM for auto ...

Assoc Computational Linguistics-Acl2023

DeepBreath-automated detection of respiratory pathology from lung auscultation in 572 pediatric outpatients across 5 countries

Martin Jaggi, Mary-Anne Hartley, Jonathan Dönz, Juliane Dervaux, Tatjana Chavdarova, Daniel Mueller, Julien Niklas Heitmann, Daniel Hinjos García

The interpretation of lung auscultation is highly subjective and relies on non-specific nomenclature. Computer-aided analysis has the potential to better standardize and automate evaluation. We used 35.9 hours of auscultation audio from 572 pediatric outpa ...

NATURE PORTFOLIO2023

SKILL: Structured Knowledge Infusion for Large Language Models

Martin Jaggi, Fedor Moiseev

Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks. However, it is largely unexplored whether they can better internalize knowledge from a structured data, such as a knowledge graph, or from ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2022

Implicit Gradient Alignment in Distributed and Federated Learning

Martin Jaggi, Luis Felipe Barba Flores

A major obstacle to achieving global convergence in distributed and federated learning is the misalignment of gradients across clients, or mini-batches due to heterogeneity and stochasticity of the distributed data. In this work, we show that data heteroge ...

ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE2022