Concept
Multiplication operator
Publications associées (6)
Stochastic distributed learning with gradient quantization and double-variance reduction
Sebastian Urban Stich, Konstantin Mishchenko
We consider distributed optimization over several devices, each sending incremental model updates to a central server. This setting is considered, for instance, in federated learning. Various schemes have been designed to compress the model updates in orde ...
TAYLOR & FRANCIS LTD2022Error Feedback Fixes SignSGD and other Gradient Compression Schemes
Martin Jaggi, Sebastian Urban Stich, Quentin Rebjock, Sai Praneeth Reddy Karimireddy
Sign-based algorithms (e.g. signSGD) have been proposed as a biased gradient compression technique to alleviate the communication bottleneck in training large neural networks across multiple workers. We show simple convex counter-examples where signSGD doe ...
PMLR2019Error Feedback Fixes SignSGD and other Gradient Compression Schemes
Martin Jaggi, Sebastian Urban Stich, Quentin Rebjock, Sai Praneeth Reddy Karimireddy
Sign-based algorithms (e.g. signSGD) have been proposed as a biased gradient compression technique to alleviate the communication bottleneck in training large neural networks across multiple workers. We show simple convex counter-examples where signSGD doe ...
2019