Publications related to Wavelet packet best basis search using generalized Renyi entropy

Information Spectrum Converse for Minimum Entropy Couplings and Functional Representations

Given two jointly distributed random variables (X,Y), a functional representation of X is a random variable Z independent of Y, and a deterministic function g(⋅,⋅) such that X=g(Y,Z). The problem of finding a minimum entropy functional representation is kn ...

2023

A Functional Perspective on Information Measures

Amedeo Roberto Esposito

Since the birth of Information Theory, researchers have defined and exploited various information measures, as well as endowed them with operational meanings. Some were born as a "solution to a problem", like Shannon's Entropy and Mutual Information. Other ...

EPFL2022

From Generalisation Error to Transportation-cost Inequalities and Back

Michael Christoph Gastpar, Amedeo Roberto Esposito

In this work, we connect the problem of bounding the expected generalisation error with transportation-cost inequalities. Exposing the underlying pattern behind both approaches we are able to generalise them and go beyond Kullback- Leibler Divergences/Mutu ...

2022

From Data to Decisions: Distributionally Robust Optimization is Optimal

Daniel Kuhn, Peyman Mohajerin Esfahani

We study stochastic programs where the decision-maker cannot observe the distribution of the exogenous uncertainties but has access to a finite set of independent samples from this distribution. In this setting, the goal is to find a procedure that transfo ...

2020

Information Theoretic Characterization of Uncertainty Distinguishes Surprise From Accuracy Signals in the Brain

Kerstin Preuschoff, Leyla Loued-Khenissi

Uncertainty presents a problem for both human and machine decision-making. While utility maximization has traditionally been viewed as the motive force behind choice behavior, it has been theorized that uncertainty minimization may supersede reward motivat ...

2020

Robust Generalization via f−Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f-divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals ...

IEEE2020

Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Networks

Emmanuel Abbé

We derive generalization and excess risk bounds for neural networks using a family of complexity measures based on a multilevel relative entropy. The bounds are obtained by introducing the notion of generated hierarchical coverings of neural networks and b ...

MICROTOME PUBL2020

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

Generalized maximum entropy estimation

John Lygeros, Tobias Sutter

We consider the problem of estimating a probability distribution that maximizes the entropy while satisfying a finite number of moment constraints, possibly corrupted by noise. Based on duality of convex programming, we present a novel approximation scheme ...

2019

Statistical Aspects of Wasserstein Distances

Victor Panaretos, Yoav Zemel

Wasserstein distances are metrics on probability distributions inspired by the problem of optimal mass transportation. Roughly speaking, they measure the minimal effort required to reconfigure the probability mass of one distribution in order to recover th ...

ANNUAL REVIEWS2019

Wavelet packet best basis search using generalized Renyi entropy

Graph Chatbot

Chat with Graph Search

Information Spectrum Converse for Minimum Entropy Couplings and Functional Representations

A Functional Perspective on Information Measures

From Generalisation Error to Transportation-cost Inequalities and Back

From Data to Decisions: Distributionally Robust Optimization is Optimal

Information Theoretic Characterization of Uncertainty Distinguishes Surprise From Accuracy Signals in the Brain

Robust Generalization via f−Mutual Information

Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Networks

Robust Generalization via $\alpha$-Mutual Information

Generalized maximum entropy estimation

Statistical Aspects of Wasserstein Distances

Information Spectrum Converse for Minimum Entropy Couplings and Functional Representations

From Generalisation Error to Transportation-cost Inequalities and Back

From Data to Decisions: Distributionally Robust Optimization is Optimal

Statistical Aspects of Wasserstein Distances

Robust Generalization via f−Mutual Information

A Functional Perspective on Information Measures

Information Theoretic Characterization of Uncertainty Distinguishes Surprise From Accuracy Signals in the Brain

Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Networks

Generalized maximum entropy estimation

Robust Generalization via $\alpha$-Mutual Information