Publications related to ON THE INEQUALITIES IN INFORMATION THEORY

Common Information Components Analysis

Wyner's common information is a measure that quantifies and assesses the commonality between two random variables. Based on this, we introduce a novel two-step procedure to construct features from data, referred to as Common Information Components Analysis ...

MDPI2021

Secrecy by Design With Applications to Privacy and Compression

Yanina Yurina Shkel

Secrecy by design is examined as an approach to information-theoretic secrecy. The main idea behind this approach is to design an information processing system from the ground up to be perfectly secure with respect to an explicit secrecy constraint. The pr ...

2021

Towards Interactions Through Differentiable-Non-Differentiable Scale Transitions In Scale Relativity Theory

Maria-Alexandra Paun

It is shown that, in the framework of Scale Relativity Theory, correlations of type informational entropy/cross entropy - probability density, in the description of the dynamics of any complex system, can be perceived as interactions. Explaining these inte ...

UNIV POLITEHNICA BUCHAREST, SCI BULL2021

Learning in Volatile Environments With the Bayes Factor Surprise

Wulfram Gerstner, Johanni Michael Brea, Alireza Modirshanechi, Vasiliki Liakoni

Surprise-based learning allows agents to rapidly adapt to nonstationary stochastic environments characterized by sudden changes. We show that exact Bayesian inference in a hierarchical model gives rise to a surprise-modulated trade-off between forgetting o ...

MIT PRESS2021

Distributional Robustness in Mechanism Design

Cagil Kocyigit

Mechanism design theory examines the design of allocation mechanisms or incentive systems involving multiple rational but self-interested agents and plays a central role in many societally important problems in economics. In mechanism design problems, agen ...

EPFL2020

Information Theoretic Characterization of Uncertainty Distinguishes Surprise From Accuracy Signals in the Brain

Kerstin Preuschoff, Leyla Loued-Khenissi

Uncertainty presents a problem for both human and machine decision-making. While utility maximization has traditionally been viewed as the motive force behind choice behavior, it has been theorized that uncertainty minimization may supersede reward motivat ...

2020

An Operational Approach to Information Leakage

Ibrahim Issa

Given two random variables X and Y , an operational approach is undertaken to quantify the "leakage" of information from X to Y . The resulting measure L (X -> Y) is called maximal leakage, and is defined as the multiplicative increase, upon observing Y , ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

Strong convergence of multivariate maxima

Simone Padoan, Stefano Rizzelli

It is well known and readily seen that the maximum of n independent and uniformly on [0, 1] distributed random variables, suitably standardised, converges in total variation distance, as n increases, to the standard negative exponential distribution. We ex ...

CAMBRIDGE UNIV PRESS2020

On some consistent tests of mutual independence among several random vectors of arbitrary dimensions

Soham Sarkar

Testing for mutual independence among several random vectors is a challenging problem, and in recent years, it has gained significant attention in statistics and machine learning literature. Most of the existing tests of independence deal with only two ran ...

SPRINGER2020

ON THE INEQUALITIES IN INFORMATION THEORY

Graph Chatbot

Chat with Graph Search

Common Information Components Analysis

Secrecy by Design With Applications to Privacy and Compression

Towards Interactions Through Differentiable-Non-Differentiable Scale Transitions In Scale Relativity Theory

Learning in Volatile Environments With the Bayes Factor Surprise

Distributional Robustness in Mechanism Design

Information Theoretic Characterization of Uncertainty Distinguishes Surprise From Accuracy Signals in the Brain

An Operational Approach to Information Leakage

Robust Generalization via $\alpha$-Mutual Information

Strong convergence of multivariate maxima

On some consistent tests of mutual independence among several random vectors of arbitrary dimensions

Secrecy by Design With Applications to Privacy and Compression

Common Information Components Analysis

An Operational Approach to Information Leakage

Strong convergence of multivariate maxima

Towards Interactions Through Differentiable-Non-Differentiable Scale Transitions In Scale Relativity Theory

Learning in Volatile Environments With the Bayes Factor Surprise

Information Theoretic Characterization of Uncertainty Distinguishes Surprise From Accuracy Signals in the Brain

Robust Generalization via $\alpha$-Mutual Information

On some consistent tests of mutual independence among several random vectors of arbitrary dimensions

Distributional Robustness in Mechanism Design