Publications by Tao Lin | EPFL Graph Search

On Pitfalls of Test-time Adaptation

Alexandre Massoud Alahi, Tao Lin, Hao Zhao, Yuejiang Liu

2023

Algorithms for Efficient and Robust Distributed Deep Learning

The success of deep learning may be attributed in large part to remarkable growth in the size and complexity of deep neural networks. However, present learning systems raise significant efficiency concerns and privacy: (1) currently, training systems are l ...

EPFL2022

Learning Disentangled Behaviour Patterns for Wearable-based Human Activity Recognition

Tao Lin

In wearable-based human activity recognition (HAR) research, one of the major challenges is the large intra-class variability problem. The collected activity signal is often, if not always, coupled with noises or bias caused by personal, environmental, or ...

ASSOC COMPUTING MACHINERY2022

Consensus Control for Decentralized Deep Learning

Martin Jaggi, Tao Lin, Sebastian Urban Stich, Anastasiia Koloskova, Lingjing Kong

Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters. Experiments in earlier works reveal that, even in a data-center setup, decentralized training often suffers fro ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data

Martin Jaggi, Tao Lin, Sebastian Urban Stich, Sai Praneeth Reddy Karimireddy

Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of heterogeneity across different clients' local datasets poses an optimization ch ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them

Sabine Süsstrunk, Mathieu Salzmann, Tao Lin, Chen Liu

We analyze the influence of adversarial training on the loss landscape of machine learning models. To this end, we first provide analytical studies of the properties of adversarial loss functions under different adversarial budgets. We then demonstrate tha ...

2021

Dynamic Model Pruning with Feedback

Martin Jaggi, Luis Felipe Barba Flores, Sebastian Urban Stich, Tao Lin, Daniil Dmitriev

Deep neural networks often have millions of parameters. This can hinder their deployment to low-end devices, not only due to high memory requirements but also because of increased latency at inference. We propose a novel model compression method that gener ...

2020

GA-Par: Dependable Microservice Orchestration Framework for Geo-Distributed Clouds

Tao Lin, Jie Xu

Recent advances in composing Cloud applications have been driven by deployments of inter-networking heterogeneous microservices across multiple Cloud datacenters. System dependability has been of the upmost importance and criticality to both service vendor ...

IEEE COMPUTER SOC2020

Ensemble Distillation for Robust Model Fusion in Federated Learning

Martin Jaggi, Sebastian Urban Stich, Tao Lin, Lingjing Kong

Federated Learning (FL) is a machine learning setting where many devices collaboratively train a machine learning model while keeping the training data decentralized. In most of the current training schemes the central model is refined by averaging the par ...

2020

Extrapolation for Large-batch Training in Deep Learning

Martin Jaggi, Sebastian Urban Stich, Tao Lin, Lingjing Kong

Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fraction of the training data. A major roadblock faced when increasing the batc ...

2020