**Are you an EPFL student looking for a semester project?**

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

Concept# Deviance information criterion

Summary

The deviance information criterion (DIC) is a hierarchical modeling generalization of the Akaike information criterion (AIC). It is particularly useful in Bayesian model selection problems where the posterior distributions of the models have been obtained by Markov chain Monte Carlo (MCMC) simulation. DIC is an asymptotic approximation as the sample size becomes large, like AIC. It is only valid when the posterior distribution is approximately multivariate normal.
Define the deviance as , where are the data, are the unknown parameters of the model and is the likelihood function. is a constant that cancels out in all calculations that compare different models, and which therefore does not need to be known.
There are two calculations in common usage for the effective number of parameters of the model. The first, as described in , is , where is the expectation of . The second, as described in , is . The larger the effective number of parameters is, the easier it is for the model to fit the data, and so the deviance needs to be penalized.
The deviance information criterion is calculated as
or equivalently as
From this latter form, the connection with AIC is more evident.
The idea is that models with smaller DIC should be preferred to models with larger DIC. Models are penalized both by the value of , which favors a good fit, but also (similar to AIC) by the effective number of parameters . Since will decrease as the number of parameters in a model increases, the term compensates for this effect by favoring models with a smaller number of parameters.
An advantage of DIC over other criteria in the case of Bayesian model selection is that the DIC is easily calculated from the samples generated by a Markov chain Monte Carlo simulation. AIC requires calculating the likelihood at its maximum over , which is not readily available from the MCMC simulation. But to calculate DIC, simply compute as the average of over the samples of , and as the value of evaluated at the average of the samples of . Then the DIC follows directly from these approximations.

Official source

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications (40)

Related people (8)

Related courses (5)

Related concepts (4)

Related units (10)

Related lectures (33)

Model selection

Model selection is the task of selecting a model from among various candidates on the basis of performance criterion to choose the best one. In the context of learning, this may be the selection of a statistical model from a set of candidate models, given data. In the simplest cases, a pre-existing set of data is considered. However, the task can also involve the design of experiments such that the data collected is well-suited to the problem of model selection.

Bayesian information criterion

In statistics, the Bayesian information criterion (BIC) or Schwarz information criterion (also SIC, SBC, SBIC) is a criterion for model selection among a finite set of models; models with lower BIC are generally preferred. It is based, in part, on the likelihood function and it is closely related to the Akaike information criterion (AIC). When fitting models, it is possible to increase the maximum likelihood by adding parameters, but doing so may result in overfitting.

Bayes factor

The Bayes factor is a ratio of two competing statistical models represented by their evidence, and is used to quantify the support for one model over the other. The models in questions can have a common set of parameters, such as a null hypothesis and an alternative, but this is not necessary; for instance, it could also be a non-linear model compared to its linear approximation. The Bayes factor can be thought of as a Bayesian analog to the likelihood-ratio test, although it uses the (integrated) marginal likelihood rather than the maximized likelihood.

MATH-413: Statistics for data science

Statistics lies at the foundation of data science, providing a unifying theoretical and methodological backbone for the diverse tasks enountered in this emerging field. This course rigorously develops

MATH-408: Regression methods

General graduate course on regression methods

MATH-342: Time series

A first course in statistical time series analysis and applications.

Model Selection Criteria: AIC, BIC, CpMATH-413: Statistics for data science

Explores model selection criteria like AIC, BIC, and Cp in statistics for data science.

Statistical Inference: Model Selection and Nuisance ParametersMATH-562: Statistical inference

Covers model selection, nuisance parameters, and higher-order inference methods in statistical inference.

The Information Criteria of Takeuchi and AkaikeMATH-412: Statistical machine learning

Covers the estimation of optimism for predictors via statistical models and the application of AIC to linear regression.

Maria Colombo, Massimo Sorella

The Obukhov-Corrsin theory of scalar turbulence [21, 54] advances quantitative predictions on passive-scalar advection in a turbulent regime and can be regarded as the analogue for passive scalars of Kolmogorov's K41 theory of fully developed turbulence [4 ...

The electron self-interaction is a long-standing problem in density functional theory and is particularly critical in the description of polarons. Polarons are quasiparticles involving charge localization coupled with self-induced lattice distortions. Sinc ...

Nikolaos Geroliminis, Raphael Ali Francis Lamotte

Traditional priority schemes reduce delays for some by increasing those of others. Yet, this might not be a necessity. Several works published over the last two decades have shown for a stylized set-up with homogeneous users that dynamic priority scheme ma ...

2020