Publications related to Conditional entropy

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establishes novel bounds on the differential entropy of the co ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

The Gray-Wyner Network and Wyner's Common Information for Gaussian Sources

Michael Christoph Gastpar, Erixhen Sula

This paper presents explicit solutions for two related non-convex information extremization problems due to Gray and Wyner in the Gaussian case. The first problem is the Gray-Wyner network subject to a sum-rate constraint on the two private links. Here, ou ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Differential Entropy of the Conditional Expectation under Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

This paper considers an additive Gaussian noise channel with arbitrarily distributed finite variance input signals. It studies the differential entropy of the minimum mean-square error (MMSE) estimator and provides a new lower bound which connects the diff ...

IEEE2021

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Nicolas Macris, Florent Gérard Krzakala, Jean François Emmanuel Barbier, Mohamad Baker Dia

We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections. A few examples where this problem is relevant are compressed sensing, sparse superposition codes, and code division multiple access. There has been a ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Reed-Muller codes polarize

Emmanuel Abbé, Min Ye

Reed-Muller (RM) codes were introduced in 1954 and have long been conjectured to achieve Shannon's capacity on symmetric channels. The activity on this conjecture has recently been revived with the emergence of polar codes. RM codes and polar codes are gen ...

IEEE COMPUTER SOC2019

Fisher–Shannon Complexity Analysis of High-Frequency Urban Wind Speed Time Series

Dasaraden Mauree, Fabian Guignard

One-hertz wind time series recorded at different levels (from 1.5–25.5 m) in an urban area are investigated by using the Fisher–Shannon (FS) analysis. FS analysis is a well-known method to gain insight into the complex behavior of nonlinear systems, by qua ...

2019

Chaining Mutual Information and Tightening Generalization Bounds

Emmanuel Abbé

Bounding the generalization error of learning algorithms has a long history, which yet falls short in explaining various generalization successes including those of deep learning. Two important difficulties are (i) exploiting the dependencies between the h ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018

Lower Bounds on Active Learning for Graphical Model Selection

Volkan Cevher, Jonathan Mark Scarlett

We consider the problem of estimating the underlying graph associated with a Markov random field, with the added twist that the decoding algorithm can iteratively choose which subsets of nodes to sample based on the previous samples, resulting in an active ...

2017

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Conditional entropy

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

The Gray-Wyner Network and Wyner's Common Information for Gaussian Sources

Differential Entropy of the Conditional Expectation under Gaussian Noise

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Reed-Muller codes polarize

Towards Measuring the Amount of Discriminatory Information in Finger Vein Biometric Characteristics Using a Relative Entropy Estimator

Fisher–Shannon Complexity Analysis of High-Frequency Urban Wind Speed Time Series

Dimension-free Information Concentration via Exp-Concavity

Chaining Mutual Information and Tightening Generalization Bounds

Lower Bounds on Active Learning for Graphical Model Selection

Graph Chatbot

Chat with Graph Search

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

The Gray-Wyner Network and Wyner's Common Information for Gaussian Sources

Differential Entropy of the Conditional Expectation under Gaussian Noise

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Towards Measuring the Amount of Discriminatory Information in Finger Vein Biometric Characteristics Using a Relative Entropy Estimator

Reed-Muller codes polarize

Fisher–Shannon Complexity Analysis of High-Frequency Urban Wind Speed Time Series

Lower Bounds on Active Learning for Graphical Model Selection

Chaining Mutual Information and Tightening Generalization Bounds

Dimension-free Information Concentration via Exp-Concavity