Natural exponential family

In probability and statistics, a natural exponential family (NEF) is a class of probability distributions that is a special case of an exponential family (EF). Definition Univariate case The natural exponential families (NEF) are a subset of the exponential families. A NEF is an exponential family in which the natural parameter η and the natural statistic T(x) are both the identity. A distribution in an exponential family with parameter θ can be written with probability density function (PDF) : f_X(x\mid \theta) = h(x)\ \exp\Big(\ \eta(\theta) T(x) - A(\theta)\ \Big) ,! , where h(x) and A(\theta) are known functions. A distribution in a natural exponential family with parameter θ can thus be written with PDF : f_X(x\mid \theta) = h(x)\ \exp\Big(\ \theta x - A(\theta)\ \Big) ,! . [Note that slightly different notation is used by the originator of the NEF, Carl Morris. Morris uses &omega

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications (4)

Related publications

Related people

Related units

Related concepts (12)

Related concepts

Related courses (2)

Related courses

Related lectures (8)

Related lectures

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

Ahmet Arda Atalik, Michael Christoph Gastpar, Alper Köse

The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establishes novel bounds on the differential entropy of the conditional mean in the case of finite-variance input signals and additive Gaussian noise. The main result is a new lower bound in terms of the differential entropies of the input signal and the noisy observation. The main results are also extended to the vector Gaussian channel and to the natural exponential family. Various other properties such as upper bounds, asymptotics, Taylor series expansion, and connection to Fisher Information are obtained. Two applications of the lower bound in the remote-source coding and CEO problem are discussed.

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Generalized Spatial Regression with Differential Penalization

Matthieu Wilhelm

We propose a novel method for the analysis of spatially distributed data from an exponential family distribution, able to efficiently treat data occurring over irregularly shaped domains. We consider a generalized linear framework and extend the work of Sangalli et al. (2013) to distributions other than the Gaussian. In particular, we can handle all distributions within the exponential family, including binomial, Poisson and Gamma outcomes, hence leading to a very broad applicability of the proposed model. We maximize a penalized log-likelihood function. The roughness penalty term involves a suitable differential operator of the spatial field over the domain of interest. This maximization is done via a penalized iterative least square approach (see Wood (2006)). Covariate information can also be included in the model in a semi-parametric setting. The proposed models exploit advanced scientific computing techniques and specifically make use of the Finite Element Method, that provides a basis for piecewise polynomial surfaces and allows to impose boundary conditions on the space distribution of the probability. Finally, we extend theoretically the model to deal with data occurring on a two dimensional manifold.

2013

Code-specific policy gradient rules for spiking neurons

Wulfram Gerstner, Guillaume Hennequin

Although it is widely believed that reinforcement learning is a suitable tool for describing behavioral learning, the mechanisms by which it can be implemented in networks of spiking neurons are not fully understood. Here, we show that different learning rules emerge from a policy gradient approach depending on which features of the spike trains are assumed to influence the reward signals, i.e., depending on which neural code is in effect. We use the framework of Williams (1992) to derive learning rules for arbitrary neural codes. For illustration, we present policy-gradient rules for three different example codes - a spike count code, a spike timing code and the most general “full spike train” code - and test them on simple model problems. In addition to classical synaptic learning, we derive learning rules for intrinsic parameters that control the excitability of the neuron. The spike count learning rule has structural similarities with established Bienenstock-Cooper-Munro rules. If the distribution of the relevant spike train features belongs to the natural exponential family, the learning rules have a characteristic shape that raises interesting prediction problems.

Massachusetts Institute of Technology Press2009

Gamma distribution

In probability theory and statistics, the gamma distribution is a two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared di

Negative binomial distribution

In probability theory and statistics, the negative binomial distribution is a discrete probability distribution that models the number of failures in a sequence of independent and identically distri

Exponential family

In probability and statistics, an exponential family is a parametric set of probability distributions of a certain form, specified below. This special form is chosen for mathematical convenience, in

COM-406: Foundations of Data Science

We discuss a set of topics that are important for the understanding of modern data science but that are typically not taught in an introductory ML course. In particular we discuss fundamental ideas and techniques that come from probability, information theory as well as signal processing.

MATH-408: Regression methods

General graduate course on regression methods