Anne-Marie Kermarrec

Biography

Anne-Marie Kermarrec is Professor at EPFL since January 2020. Before that she was the CEO of the Mediego startup that she founded in April 2015. Mediego provides content personalization services for online publishers. She was a Research Director at Inria, France from 2004 to 2015. She got a Ph.D. thesis from University of Rennes (France), and has been with Vrije Universiteit, NL and Microsoft Research Cambridge, UK. Anne-Marie received an ERC grant in 2008 and an ERC Proof of Concept in 2013. She received the Montpetit Award in 2011 and the Innovation Award in 2017 from the French Academy of Science. She has been elected to the European Academy in 2013 and named ACM Fellow in 2016. Her research interests are in large-scale distributed systems, epidemic algorithms, peer to peer networks and system support for machine learning.Google Scholar: https://scholar.google.com/citations?user=aIAy-qcAAAAJDBLP: https://dblp.org/pers/k/Kermarrec:Anne=Marie.html

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related units (9)

Related units

Courses taught by this person (4)

The course covers fundamental principles for understanding and building systems for managing and analyzing large amounts of data.

This course is intended for students who want to understand modern large-scale data analysis systems and database systems. The course covers fundamental principles for understanding and building systems for managing and analyzing large amounts of data. It covers a wide range of topics and technologi

This course will cover the latest technologies, platforms and research contributions in the area of machine learning systems. The students will read, review and present papers from recent venues across the systems for ML spectrum.

Courses taught by this person

Related research domains (44)

Related research domains

Related publications (61)

Related publications

People doing similar research (100)

People doing similar research

Decentralized learning made easy with DecentralizePy

Akash Balasaheb Dhasade, Anne-Marie Kermarrec, Rafael Pereira Pires, Rishi Sharma, Milos Vujasinovic

Decentralized learning (DL) has gained prominence for its potential benefits in terms of scalability, privacy, and fault tolerance. It consists of many nodes that coordinate without a central server and exchange millions of parameters in the inherently iterative process of machine learning (ML) training. In addition, these nodes are connected in complex and potentially dynamic topologies. Assessing the intricate dynamics of such networks is clearly not an easy task. Often in literature, researchers resort to simulated environments that do not scale and fail to capture practical and crucial behaviors, including the ones associated to parallelism, data transfer, network delays, and wall-clock time. In this paper, we propose DecentralizePy, a distributed framework for decentralized ML, which allows for the emulation of large-scale learning networks in arbitrary topologies. We demonstrate the capabilities of DecentralizePy by deploying techniques such as sparsification and secure aggregation on top of several topologies, including dynamic networks with more than one thousand nodes.

2023

Get More for Less in Decentralized Learning Systems

Akash Balasaheb Dhasade, Anne-Marie Kermarrec, Rafael Pereira Pires, Rishi Sharma, Milos Vujasinovic, Jeffrey Wigger

Decentralized learning (DL) systems have been gaining popularity because they avoid raw data sharing by communicating only model parameters, hence preserving data confidentiality. However, the large size of deep neural networks poses a significant challenge for decentralized training, since each node needs to exchange gigabytes of data, overloading the network. In this paper, we address this challenge with JWINS, a communication-efficient and fully decentralized learning system that shares only a subset of parameters through sparsification. JWINS uses wavelet transform to limit the information loss due to sparsification and a randomized communication cut-off that reduces communication usage without damaging the performance of trained models. We demonstrate empirically with 96 DL nodes on non-IID datasets that JWINS can achieve similar accuracies to full-sharing DL while sending up to 64% fewer bytes. Additionally, on low communication budgets, JWINS outperforms the state-of-the-art communication-efficient DL algorithm CHOCO-SGD by up to 4x in terms of network savings and time.

2023

D-Cliques: Compensating for Data Heterogeneity with Topology in Decentralized Federated Learning

Anne-Marie Kermarrec, Erick Lavoie

The convergence speed of machine learning models trained with Federated Learning is significantly affected by non-independent and identically distributed (non-IID) data partitions, even more so in a fully decentralized setting without a central server. In this paper, we show that the impact of local class bias, an important type of data non-IIDness, can be significantly reduced by carefully designing the underlying communication topology. We present D-Cliques, a novel topology that reduces gradient bias by grouping nodes in interconnected cliques such that the local joint distribution in a clique is representative of the global class distribution. We also show how to adapt the updates of decentralized SGD to obtain unbiased gradients and implement an effective momentum with D-Cliques. Our empirical evaluation on MNIST and CIFAR10 demonstrates that our approach provides similar convergence speed as a fully-connected topology with a significant reduction in the number of edges and messages. In a 1000-node topology, D-Cliques requires 98% less edges and 96% less total messages, with further possible gains using a small-world topology across cliques.

2022