Publications associées à Entropie min

Information Spectrum Converse for Minimum Entropy Couplings and Functional Representations

Given two jointly distributed random variables (X,Y), a functional representation of X is a random variable Z independent of Y, and a deterministic function g(⋅,⋅) such that X=g(Y,Z). The problem of finding a minimum entropy functional representation is kn ...

2023

A Functional Perspective on Information Measures

Amedeo Roberto Esposito

Since the birth of Information Theory, researchers have defined and exploited various information measures, as well as endowed them with operational meanings. Some were born as a "solution to a problem", like Shannon's Entropy and Mutual Information. Other ...

EPFL2022

Shannon entropy and degree of polarization of a speckle pattern

Abhijit Roy

The dependence of the Shannon entropy (SE) of a speckle pattern on the degree of polarization (DoP) of the pattern is investigated both experimentally and numerically. The superposition of two uncorrelated speckle patterns with polarization diversity is ut ...

OPTICAL SOC AMER2021

From Data to Decisions: Distributionally Robust Optimization is Optimal

Daniel Kuhn, Peyman Mohajerin Esfahani

We study stochastic programs where the decision-maker cannot observe the distribution of the exogenous uncertainties but has access to a finite set of independent samples from this distribution. In this setting, the goal is to find a procedure that transfo ...

2021

Robust Generalization via f−Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f-divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals ...

IEEE2020

Gaussian Process Regression for Maximum Entropy Distribution

Mohammadhossein Gorji, Mohsen Sadr

Maximum-Entropy Distributions offer an attractive family of probability densities suitable for moment closure problems. Yet finding the Lagrange multipliers which parametrize these distributions, turns out to be a computational bottleneck for practical clo ...

2020

Gaussian Process Regression for Maximum Entropy Distribution

Mohsen Sadr

Maximum-Entropy Distributions offer an attractive family of probability densities suitable for moment closure problems. Yet finding the Lagrange multipliers which parametrize these distributions, turns out to be a computational bottleneck for practical clo ...

2019

Negentropic linguistic evolution: A comparison of seven languages

Frédéric Kaplan, Vincent Christian Buntinx

The relationship between the entropy of language and its complexity has been the subject of much speculation – some seeing the increase of linguistic entropy as a sign of linguistic complexification or interpreting entropy drop as a marker of greater regul ...

Red de Humanidades Digitales A. C.2018

Polarization of the Renyi Information Dimension With Applications to Compressed Sensing

Emmanuel Abbé, Saeid Haghighatshoar

In this paper, we show that the Hadamard matrix acts as an extractor over the reals of the Renyi Information Dimension (RID), in an analogous way to how it acts as an extractor of the discrete entropy over finite fields. More precisely, we prove that the R ...

Ieee-Inst Electrical Electronics Engineers Inc2017

Spatial Coupling as a Proof Technique and Three Applications

Rüdiger Urbanke, Nicolas Macris, Andrei Giurgiu

The aim of this paper is to show that spatial coupling can be viewed not only as a means to build better graphical models, but also as a tool to better understand uncoupled models. The starting point is the observation that some asymptotic properties of gr ...

Ieee-Inst Electrical Electronics Engineers Inc2016