Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
Concept
Importance sampling
Summary
Importance sampling is a Monte Carlo method for evaluating properties of a particular distribution, while only having samples generated from a different distribution than the distribution of interest. Its introduction in statistics is generally attributed to a paper by Teun Kloek and Herman K. van Dijk in 1978, but its precursors can be found in statistical physics as early as 1949. Importance sampling is also related to umbrella sampling in computational physics. Depending on the application, the term may refer to the process of sampling from this alternative distribution, the process of inference, or both.
Let be a random variable in some probability space . We wish to estimate the expected value of X under P, denoted E[X;P]. If we have statistically independent random samples , generated according to P, then an empirical estimate of E[X;P] is
and the precision of this estimate depends on the variance of X:
The basic idea of importance sampling is to sample the states from a different distribution to lower the variance of the estimation of E[X;P], or when sampling from P is difficult.
This is accomplished by first choosing a random variable such that E[L;P] = 1 and that P-almost everywhere .
With the variable L we define a probability that satisfies
The variable X/L will thus be sampled under P(L) to estimate E[X;P] as above and this estimation is improved when
When X is of constant sign over Ω, the best variable L would clearly be , so that X/L* is the searched constant E[X;P] and a single sample under P(L*) suffices to give its value. Unfortunately we cannot take that choice, because E[X;P] is precisely the value we are looking for! However this theoretical best case L* gives us an insight into what importance sampling does:
to the right, is one of the infinitesimal elements that sum up to E[X;P]:
therefore, a good probability change P(L) in importance sampling will redistribute the law of X so that its samples' frequencies are sorted directly according to their weights in E[X;P]. Hence the name "importance sampling.
Official source
About this result
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.