Concept

Shannon's source coding theorem

Related publications (87)

Error Feedback Fixes SignSGD and other Gradient Compression Schemes

Martin Jaggi, Sebastian Urban Stich, Quentin Rebjock, Sai Praneeth Reddy Karimireddy

Sign-based algorithms (e.g. signSGD) have been proposed as a biased gradient compression technique to alleviate the communication bottleneck in training large neural networks across multiple workers. We show simple convex counter-examples where signSGD doe ...

PMLR2019

Error Feedback Fixes SignSGD and other Gradient Compression Schemes

Martin Jaggi, Sebastian Urban Stich, Quentin Rebjock, Sai Praneeth Reddy Karimireddy

2019

Communication Versus Computation: Duality for Multiple-Access Channels and Source Coding

Michael Christoph Gastpar, Sung Hoon Lim, Jingge Zhu

Computation codes in network information theory are designed for scenarios where the decoder is not interested in recovering the information sources themselves, but only a function thereof. Körner and Marton showed for distributed source coding (DSC) that ...

2019

Predictive Validity of Thin-Slice Nonverbal Behavior from Social Interactions

Laurent Son Nguyen, Denise Frauendorfer

We present five studies investigating the predictive validity of thin slices of nonverbal behavior (NVB). Predictive validity of thin slices refers to how well behavior slices excerpted from longer video predict other measured variables. Using six NVBs, we ...

SAGE PUBLICATIONS INC2019

A New Coding Paradigm for the Primitive Relay Channel

Rüdiger Urbanke, Seyed Hamed Hassani, Marco Mondelli

We present a coding paradigm that provides a new achievable rate for the primitive relay channel by combining compress-and-forward and decode-and-forward with a chaining construction. In the primitive relay channel model, the source broadcasts a message to ...

IEEE2018

Near-Optimal Noisy Group Testing via Separate Decoding of Items

Volkan Cevher, Jonathan Mark Scarlett

The group testing problem consists of determining a small set of defective items from a larger set of items based on a number of tests, and is relevant in applications such as medical testing, communication protocols, pattern matching, and more. In this pa ...

2018

On the Dispersions of the Gel'fand-Pinsker Channel and Dirty Paper Coding

Jonathan Mark Scarlett

This paper studies the second-order coding rates for memoryless channels with a state sequence known non-causally at the encoder. In the case of finite alphabets, an achievability result is obtained using constant-composition random coding, and by using a ...

Institute of Electrical and Electronics Engineers2015

Second-Order Rate Region of Constant-Composition Codes for the Multiple-Access Channel

Jonathan Mark Scarlett, Albert Guillén i Fàbregas

This paper studies the second-order asymptotics of coding rates for the discrete memoryless multiple-access channel (MAC) with a fixed target error probability. Using constant-composition random coding, coded time-sharing, and a variant of Hoeffding's comb ...

Ieee-Inst Electrical Electronics Engineers Inc2015

Pliable Index Coding

Christina Fragouli, Siddhartha Brahma

We formulate a new variant of the index coding problem, where instead of demanding a specific message, clients are pliable, and are interested in receiving any t messages that they do not have. We term this problem pliable index coding or PICOD(t). We prov ...

Ieee-Inst Electrical Electronics Engineers Inc2015

On Distributed Successive Refinement with Lossless Recovery

Michael Christoph Gastpar, Chien-Yi Wang

The problem of successive refinement in distributed source coding and in joint source-channel coding is considered. The emphasis is placed on the case where the sources have to be recovered losslessly in the second stage. In distributed source coding, it i ...

2014

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.