Concept

Principle of maximum entropy

Related publications (139)

Relaxing the Additivity Constraints in Decentralized No-Regret High-Dimensional Bayesian Optimization

Bayesian Optimization (BO) is typically used to optimize an unknown function f that is noisy and costly to evaluate, by exploiting an acquisition function that must be maximized at each optimization step. Even if provably asymptotically optimal BO algorith ...

2024

Information Spectrum Converse for Minimum Entropy Couplings and Functional Representations

Given two jointly distributed random variables (X,Y), a functional representation of X is a random variable Z independent of Y, and a deterministic function g(⋅,⋅) such that X=g(Y,Z). The problem of finding a minimum entropy functional representation is kn ...

2023

Learning curves for the multi-class teacher-student perceptron

Lenka Zdeborová, Elisabetta Cornacchia, Bruno Loureiro, Bruno Loureiro, Francesca Mignacco

One of the most classical results in high-dimensional learning theory provides a closed-form expression for the generalisation error of binary classification with a single-layer teacher-student perceptron on i.i.d. Gaussian inputs. Both Bayes-optimal (BO) ...

IOP Publishing Ltd2023

T-5 configurations and hyperbolic systems

Carl Johan Peter Johansson, Riccardo Tione

In this paper, we study the rank-one convex hull of a differential inclusion associated to entropy solutions of a hyperbolic system of conservation laws. This was introduced in [B. Kirchheim, S. Muller and V. S(sic)ver & aacute;k, Studying Nonlinear PDE by ...

WORLD SCIENTIFIC PUBL CO PTE LTD2023

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establishes novel bounds on the differential entropy of the co ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Alpha-NML Universal Predictors

Michael Christoph Gastpar, Marco Bondaschi

Inspired by Sibson’s alpha-mutual information, we introduce a new parametric class of universal predictors. This class interpolates two well-known predictors, the mixture estimator, that includes the Laplace and the Krichevsky-Trofimov predictors, and the ...

2022

First-Principles Density Limit Scaling in Tokamaks Based on Edge Turbulent Transport and Implications for ITER

Olivier Sauter, Paolo Ricci, Alessandro Pau, Maurizio Giacomin

A first-principles scaling law, based on turbulent transport considerations, and a multimachine database of density limit discharges from the ASDEX Upgrade, JET, and TCV tokamaks, show that the increase of the boundary turbulent transport with the plasma c ...

AMER PHYSICAL SOC2022

Relative stability toward diffeomorphisms indicates performance in deep nets

Matthieu Wyart, Mario Geiger, Leonardo Petrini, Alessandro Favero

Understanding why deep nets can classify data in large dimensions remains a challenge. It has been proposed that they do so by becoming stable to diffeomorphisms, yet existing empirical measurements support that it is often not the case. We revisit this qu ...

2021

An Energy-Autonomous Wireless Sensor With Simultaneous Energy Harvesting and Ambient Light Sensing

Andreas Peter Burg, Catherine Dehollain, Roberto La Rosa

Wireless sensor nodes (WSNs) are generally powered by batteries, which results in a substantial limitation to the places where the nodes can be installed, to the maximum number of deployable devices, and to the node lifetime. To meet the demand for Interne ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Gaussian Process Regression for Maximum Entropy Distribution

Mohsen Sadr

Maximum-Entropy Distributions offer an attractive family of probability densities suitable for moment closure problems. Yet finding the Lagrange multipliers which parametrize these distributions, turns out to be a computational bottleneck for practical clo ...

2020

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.