Publications related to Dimension-free Information Concentration via Exp-Concavity

A Functional Perspective on Information Measures

Since the birth of Information Theory, researchers have defined and exploited various information measures, as well as endowed them with operational meanings. Some were born as a "solution to a problem", like Shannon's Entropy and Mutual Information. Other ...

EPFL2022

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establishes novel bounds on the differential entropy of the co ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Measurement in the Age of Information

Marilyne Andersen, Forrest Simon Webler

Information is the resolution of uncertainty and manifests itself as patterns. Although complex, most observable phenomena are not random and instead are associated with deterministic, chaotic systems. The underlying patterns and symmetries expressed from ...

2022

Differential Entropy of the Conditional Expectation under Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

This paper considers an additive Gaussian noise channel with arbitrarily distributed finite variance input signals. It studies the differential entropy of the minimum mean-square error (MMSE) estimator and provides a new lower bound which connects the diff ...

IEEE2021

A methodology to design measurement systems when multiple model classes are plausible

Ian Smith, Numa Joy Bertola, Sai Ganesh Sarvotham Pai

The management of existing civil infrastructure is challenging due to evolving functional requirements, aging and climate change. Civil infrastructure often has hidden reserve capacity because of conservative approaches used in design and during constructi ...

2021

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

Double-Loop Unadjusted Langevin Algorithm

Volkan Cevher, Paul Thierry Yves Rolland, Ali Kavis, Armin Eftekhari

A well-known first-order method for sampling from log-concave probability distributions is the Unadjusted Langevin Algorithm (ULA). This work proposes a new annealing step-size schedule for ULA, which allows to prove new convergence guarantees for sampling ...

2020

Tighter thermodynamic bound on the speed limit in systems with unidirectional transitions

Daniel Maria Busiello, Deepak Gupta

We consider a general discrete state-space system with both unidirectional and bidirectional links. In contrast to bidirectional links, there is no reverse transition along the unidirectional links Herein, we first compute the statistical length and the th ...

2020

Information Theoretic Characterization of Uncertainty Distinguishes Surprise From Accuracy Signals in the Brain

Kerstin Preuschoff, Leyla Loued-Khenissi

Uncertainty presents a problem for both human and machine decision-making. While utility maximization has traditionally been viewed as the motive force behind choice behavior, it has been theorized that uncertainty minimization may supersede reward motivat ...

2020

Framework to Approximate the Value of Information of Bridge Load Testing for Reserve Capacity Assessment

Ian Smith, Numa Joy Bertola, Marco Proverbio

In developed countries, structural assessment of existing bridges should not be performed using the same conservative models that are used at the design stage. Field measurements of real behavior provide additional information for the inference of previous ...

2020

Dimension-free Information Concentration via Exp-Concavity

Graph Chatbot

Chat with Graph Search

A Functional Perspective on Information Measures

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

Measurement in the Age of Information

Differential Entropy of the Conditional Expectation under Gaussian Noise

A methodology to design measurement systems when multiple model classes are plausible

Robust Generalization via $\alpha$-Mutual Information

Double-Loop Unadjusted Langevin Algorithm

Tighter thermodynamic bound on the speed limit in systems with unidirectional transitions

Information Theoretic Characterization of Uncertainty Distinguishes Surprise From Accuracy Signals in the Brain

Framework to Approximate the Value of Information of Bridge Load Testing for Reserve Capacity Assessment

Measurement in the Age of Information

A methodology to design measurement systems when multiple model classes are plausible

Differential Entropy of the Conditional Expectation under Gaussian Noise

A Functional Perspective on Information Measures

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

Double-Loop Unadjusted Langevin Algorithm

Framework to Approximate the Value of Information of Bridge Load Testing for Reserve Capacity Assessment

Robust Generalization via $\alpha$-Mutual Information

Tighter thermodynamic bound on the speed limit in systems with unidirectional transitions

Information Theoretic Characterization of Uncertainty Distinguishes Surprise From Accuracy Signals in the Brain