Natural exponential family

Résumé

In probability and statistics, a natural exponential family (NEF) is a class of probability distributions that is a special case of an exponential family (EF). Definition Univariate case The natural exponential families (NEF) are a subset of the exponential families. A NEF is an exponential family in which the natural parameter η and the natural statistic T(x) are both the identity. A distribution in an exponential family with parameter θ can be written with probability density function (PDF) : f_X(x\mid \theta) = h(x)\ \exp\Big(\ \eta(\theta) T(x) - A(\theta)\ \Big) ,! , where h(x) and A(\theta) are known functions. A distribution in a natural exponential family with parameter θ can thus be written with PDF : f_X(x\mid \theta) = h(x)\ \exp\Big(\ \theta x - A(\theta)\ \Big) ,! . [Note that slightly different notation is used by the originator of the NEF, Carl Morris. Morris uses &omega

Source officielle

https://en.wikipedia.org/wiki/Natural_exponential_family

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Publications associées (4)

Publications associées

Chargement

Personnes associées

Chargement

Unités associées

Chargement

Concepts associés (12)

Concepts associés

Chargement

Cours associés (2)

Cours associés

Chargement

Séances de cours associées (8)

Séances de cours associées

Chargement

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

Ahmet Arda Atalik, Michael Christoph Gastpar, Alper Köse

The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establishes novel bounds on the differential entropy of the conditional mean in the case of finite-variance input signals and additive Gaussian noise. The main result is a new lower bound in terms of the differential entropies of the input signal and the noisy observation. The main results are also extended to the vector Gaussian channel and to the natural exponential family. Various other properties such as upper bounds, asymptotics, Taylor series expansion, and connection to Fisher Information are obtained. Two applications of the lower bound in the remote-source coding and CEO problem are discussed.

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Chargement

Generalized Spatial Regression with Differential Penalization

Matthieu Wilhelm

We propose a novel method for the analysis of spatially distributed data from an exponential family distribution, able to efficiently treat data occurring over irregularly shaped domains. We consider a generalized linear framework and extend the work of Sangalli et al. (2013) to distributions other than the Gaussian. In particular, we can handle all distributions within the exponential family, including binomial, Poisson and Gamma outcomes, hence leading to a very broad applicability of the proposed model. We maximize a penalized log-likelihood function. The roughness penalty term involves a suitable differential operator of the spatial field over the domain of interest. This maximization is done via a penalized iterative least square approach (see Wood (2006)). Covariate information can also be included in the model in a semi-parametric setting. The proposed models exploit advanced scientific computing techniques and specifically make use of the Finite Element Method, that provides a basis for piecewise polynomial surfaces and allows to impose boundary conditions on the space distribution of the probability. Finally, we extend theoretically the model to deal with data occurring on a two dimensional manifold.

2013

Chargement

Although it is widely believed that reinforcement learning is a suitable tool for describing behavioral learning, the mechanisms by which it can be implemented in networks of spiking neurons are not fully understood. Here, we show that different learning rules emerge from a policy gradient approach depending on which features of the spike trains are assumed to influence the reward signals, i.e., depending on which neural code is in effect. We use the framework of Williams (1992) to derive learning rules for arbitrary neural codes. For illustration, we present policy-gradient rules for three different example codes - a spike count code, a spike timing code and the most general “full spike train” code - and test them on simple model problems. In addition to classical synaptic learning, we derive learning rules for intrinsic parameters that control the excitability of the neuron. The spike count learning rule has structural similarities with established Bienenstock-Cooper-Munro rules. If the distribution of the relevant spike train features belongs to the natural exponential family, the learning rules have a characteristic shape that raises interesting prediction problems.

Massachusetts Institute of Technology Press2009

Chargement