Marginal likelihood

Résumé

A marginal likelihood is a likelihood function that has been integrated over the parameter space. In Bayesian statistics, it represents the probability of generating the observed sample from a prior and is therefore often referred to as model evidence or simply evidence. Concept Given a set of independent identically distributed data points \mathbf{X}=(x_1,\ldots,x_n), where x_i \sim p(x|\theta) according to some probability distribution parameterized by \theta, where \theta itself is a random variable described by a distribution, i.e. \theta \sim p(\theta\mid\alpha), the marginal likelihood in general asks what the probability p(\mathbf{X}\mid\alpha) is, where \theta has been marginalized out (integrated out): :p(\mathbf{X}\mid\alpha) = \int_\theta p(\mathbf{X}\mid\theta) , p(\theta\mid\alpha)\ \operatorname{d}!\theta The above definition is phrased in the context of

Source officielle

https://en.wikipedia.org/wiki/Marginal_likelihood

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Publications associées (4)

Publications associées

Chargement

Personnes associées (1)

Personnes associées

Chargement

Unités associées

Aucun résultat

Unités associées

Chargement

Concepts associés (7)

Concepts associés

Chargement

Cours associés (6)

Cours associés

Chargement

Séances de cours associées (20)

Afficher plus

Séances de cours associées

Chargement

Fast Dual Variational Inference for Non-Conjugate Latent Gaussian Models

Mohammad Emtiyaz Khan, Matthias Seeger

Latent Gaussian models (LGMs) are widely used in statistics and machine learning. Bayesian inference in non-conjugate LGMs is difficult due to intractable integrals in- volving the Gaussian prior and non-conjugate likelihoods. Algorithms based on variational Gaussian (VG) approximations are widely employed since they strike a favorable bal- ance between accuracy, generality, speed, and ease of use. However, the structure of the optimization problems associated with these approximations remains poorly understood, and standard solvers take too long to con- verge. We derive a novel dual variational in- ference approach that exploits the convexity property of the VG approximations. We ob- tain an algorithm that solves a convex op- timization problem, reduces the number of variational parameters, and converges much faster than previous methods. Using real- world data, we demonstrate these advantages on a variety of LGMs, including Gaussian process classification, and latent Gaussian Markov random fields.

Omni Press2013

Chargement

The linear model with sparsity-favouring prior on the coefficients has important applications in many different domains. In machine learning, most methods to date search for maximum a posteriori sparse solutions and neglect to represent posterior uncertainties. In this paper, we address problems of Bayesian optimal design (or experiment planning), for which accurate estimates of uncertainty are essential. To this end, we employ expectation propagation approximate inference for the linear model with Laplace prior, giving new insight into numerical stability properties and proposing a robust algorithm. We also show how to estimate model hyperparameters by empirical Bayesian maximisation of the marginal likelihood, and propose ideas in order to scale up the method to very large underdetermined problems. We demonstrate the versatility of our framework on the application of gene regulatory network identification from micro-array expression data, where both the Laplace prior and the active experimental design approach are shown to result in significant improvements. We also address the problem of sparse coding of natural images, and show how our framework can be used for compressive sensing tasks.

Massachusetts Institute of Technology Press2008

Chargement

Multiple Kernel Learning: A Unifying Probabilistic Viewpoint

Matthias Seeger

We present a probabilistic viewpoint to multiple kernel learning unifying well-known regularised risk approaches and recent advances in approximate Bayesian inference relaxations. The framework proposes a general objective function suitable for regression, robust regression and classification that is lower bound of the marginal likelihood and contains many regularised risk approaches as special cases. Furthermore, we derive an efficient and provably convergent optimisation algorithm.

2011

Chargement