Concept

Entropie de Shannon

Publications associées (378)

On Maintaining Linear Convergence of Distributed Learning and Optimization Under Limited Communication

In distributed optimization and machine learning, multiple nodes coordinate to solve large problems. To do this, the nodes need to compress important algorithm information to bits so that it can be communicated over a digital channel. The communication tim ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Robust Generalization via f−Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f-divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals ...

IEEE2020

From Data to Decisions: Distributionally Robust Optimization is Optimal

Daniel Kuhn, Peyman Mohajerin Esfahani

We study stochastic programs where the decision-maker cannot observe the distribution of the exogenous uncertainties but has access to a finite set of independent samples from this distribution. In this setting, the goal is to find a procedure that transfo ...

2020

The Devil is in the Detail: A Framework for Macroscopic Prediction via Microscopic Models

Negar Kiyavash

Macroscopic data aggregated from microscopic events are pervasive in machine learning, such as country-level COVID-19 infection statistics based on city-level data. Yet, many existing approaches for predicting macroscopic behavior only use aggregated data, ...

2020

Gaussian Process Regression for Maximum Entropy Distribution

Mohsen Sadr

Maximum-Entropy Distributions offer an attractive family of probability densities suitable for moment closure problems. Yet finding the Lagrange multipliers which parametrize these distributions, turns out to be a computational bottleneck for practical clo ...

2020

Common Information Components Analysis

Michael Christoph Gastpar, Erixhen Sula

We give an information-theoretic interpretation of Canonical Correlation Analysis (CCA) via (relaxed) Wyner's common information. CCA permits to extract from two high-dimensional data sets low-dimensional descriptions (features) that capture the commonalit ...

IEEE2020

EEG microstates are a candidate endophenotype for schizophrenia

Michael Herzog, Christine Mohr, Maya Roinishvili, Ophélie Gladys Favrod, Patricia Figueiredo

Electroencephalogram microstates are recurrent scalp potential configurations that remain stable for around 90 ms. The dynamics of two of the four canonical classes of microstates, commonly labeled as C and D, have been suggested as a potential endophenoty ...

2020

On quantifying the quality of acoustic models in hybrid DNN-HMM ASR

Hervé Bourlard, Afsaneh Asaei, Pranay Dighe

We propose an information theoretic framework for quantitative assessment of acoustic models used in hidden Markov model (HMM) based automatic speech recognition (ASR). The HMM backend expects that (i) the acoustic model yields accurate state conditional e ...

ELSEVIER2020

Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Networks

Emmanuel Abbé

We derive generalization and excess risk bounds for neural networks using a family of complexity measures based on a multilevel relative entropy. The bounds are obtained by introducing the notion of generated hierarchical coverings of neural networks and b ...

MICROTOME PUBL2020

Deep Learning-Based Image Classification through a Multimode Fiber in the Presence of Wavelength Drift

Demetri Psaltis, Christophe Moser, Navid Borhani, Eirini Kakkava, Babak Rahmani, Ugur Tegin

Deep neural networks (DNNs) are employed to recover information after its propagation through a multimode fiber (MMF) in the presence of wavelength drift. The intensity distribution of the speckle patterns generated at the output of an MMF when an input wa ...

2020

Graph Chatbot

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Connectez-vous pour utiliser Chat avec Graph Search