Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.
Concept
Jeffreys prior
Summary
In Bayesian probability, the Jeffreys prior, named after Sir Harold Jeffreys, is a non-informative prior distribution for a parameter space; its density function is proportional to the square root of the determinant of the Fisher information matrix:
It has the key feature that it is invariant under a change of coordinates for the parameter vector . That is, the relative probability assigned to a volume of a probability space using a Jeffreys prior will be the same regardless of the parameterization used to define the Jeffreys prior. This makes it of special interest for use with scale parameters.
In maximum likelihood estimation of exponential family models, penalty terms based on the Jeffreys prior were shown to reduce asymptotic bias in point estimates.
If and are two possible parametrizations of a statistical model, and is a continuously differentiable function of , we say that the prior is "invariant" under a reparametrization if
that is, if the priors and are related by the usual change of variables theorem.
Since the Fisher information transforms under reparametrization as
defining the priors as and gives us the desired "invariance".
Analogous to the one-parameter case, let and be two possible parametrizations of a statistical model, with a continuously differentiable function of . We call the prior "invariant" under reparametrization if
where is the Jacobian matrix with entries
Since the Fisher information matrix transforms under reparametrization as
we have that
and thus defining the priors as and gives us the desired "invariance".
From a practical and mathematical standpoint, a valid reason to use this non-informative prior instead of others, like the ones obtained through a limit in conjugate families of distributions, is that the relative probability of a volume of the probability space is not dependent upon the set of parameter variables that is chosen to describe parameter space.
Sometimes the Jeffreys prior cannot be normalized, and is thus an improper prior.
Official source
About this result
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.