Marginal likelihood

Summary

A marginal likelihood is a likelihood function that has been integrated over the parameter space. In Bayesian statistics, it represents the probability of generating the observed sample from a prior and is therefore often referred to as model evidence or simply evidence. Concept Given a set of independent identically distributed data points \mathbf{X}=(x_1,\ldots,x_n), where x_i \sim p(x|\theta) according to some probability distribution parameterized by \theta, where \theta itself is a random variable described by a distribution, i.e. \theta \sim p(\theta\mid\alpha), the marginal likelihood in general asks what the probability p(\mathbf{X}\mid\alpha) is, where \theta has been marginalized out (integrated out): :p(\mathbf{X}\mid\alpha) = \int_\theta p(\mathbf{X}\mid\theta) , p(\theta\mid\alpha)\ \operatorname{d}!\theta The above definition is phrased in the context of

Official source

https://en.wikipedia.org/wiki/Marginal_likelihood

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications (4)

Related publications

Related people (1)

Related people

Related units

Related concepts (7)

Bayesian inference (ˈbeɪziən or ˈbeɪʒən ) is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes avail

In statistics, Gibbs sampling or a Gibbs sampler is a Markov chain Monte Carlo (MCMC) algorithm for obtaining a sequence of observations which are approximated from a specified multivariate probabil

In statistical inference, the likelihood function quantifies the plausibility of parameter values characterizing a statistical model in light of observed data. Its most typical usage is to compare po

Related concepts

Related courses (6)

Related courses

Related lectures (20)

Related lectures

Latent Gaussian models (LGMs) are widely used in statistics and machine learning. Bayesian inference in non-conjugate LGMs is difficult due to intractable integrals in- volving the Gaussian prior and non-conjugate likelihoods. Algorithms based on variational Gaussian (VG) approximations are widely employed since they strike a favorable bal- ance between accuracy, generality, speed, and ease of use. However, the structure of the optimization problems associated with these approximations remains poorly understood, and standard solvers take too long to con- verge. We derive a novel dual variational in- ference approach that exploits the convexity property of the VG approximations. We ob- tain an algorithm that solves a convex op- timization problem, reduces the number of variational parameters, and converges much faster than previous methods. Using real- world data, we demonstrate these advantages on a variety of LGMs, including Gaussian process classification, and latent Gaussian Markov random fields.

Omni Press2013

Bayesian Inference and Optimal Design in the Sparse Linear Model

Matthias Seeger

The linear model with sparsity-favouring prior on the coefficients has important applications in many different domains. In machine learning, most methods to date search for maximum a posteriori sparse solutions and neglect to represent posterior uncertainties. In this paper, we address problems of Bayesian optimal design (or experiment planning), for which accurate estimates of uncertainty are essential. To this end, we employ expectation propagation approximate inference for the linear model with Laplace prior, giving new insight into numerical stability properties and proposing a robust algorithm. We also show how to estimate model hyperparameters by empirical Bayesian maximisation of the marginal likelihood, and propose ideas in order to scale up the method to very large underdetermined problems. We demonstrate the versatility of our framework on the application of gene regulatory network identification from micro-array expression data, where both the Laplace prior and the active experimental design approach are shown to result in significant improvements. We also address the problem of sparse coding of natural images, and show how our framework can be used for compressive sensing tasks.

Massachusetts Institute of Technology Press2008

Multiple Kernel Learning: A Unifying Probabilistic Viewpoint

Matthias Seeger

We present a probabilistic viewpoint to multiple kernel learning unifying well-known regularised risk approaches and recent advances in approximate Bayesian inference relaxations. The framework proposes a general objective function suitable for regression, robust regression and classification that is lower bound of the marginal likelihood and contains many regularised risk approaches as special cases. Furthermore, we derive an efficient and provably convergent optimisation algorithm.

2011