Laplace's method
In mathematics, Laplace's method, named after Pierre-Simon Laplace, is a technique used to approximate integrals of the form where is a twice-differentiable function, M is a large number, and the endpoints a and b could possibly be infinite. This technique was originally presented in . In Bayesian statistics, Laplace's approximation can refer to either approximating the posterior normalizing constant with Laplace's method or approximating the posterior distribution with a Gaussian centered at the maximum a posteriori estimate. Laplace approximations play a central role in the integrated nested Laplace approximations method for fast approximate Bayesian inference. Suppose the function has a unique global maximum at x0. Let be a constant and consider the following two functions: Note that x0 will be the global maximum of and as well. Now observe: As M increases, the ratio for will grow exponentially, while the ratio for does not change. Thus, significant contributions to the integral of this function will come only from points x in a neighbourhood of x0, which can then be estimated. To state and motivate the method, we need several assumptions. We will assume that x0 is not an endpoint of the interval of integration, that the values cannot be very close to unless x is close to x0, and that We can expand around x0 by Taylor's theorem, where (see: big O notation). Since has a global maximum at x0, and since x0 is not an endpoint, it is a stationary point, i.e. . Therefore, the second-order Taylor polynomial approximating is for x close to x0 (recall ). The assumptions ensure the accuracy of the approximation (see the picture on the right). This latter integral is a Gaussian integral if the limits of integration go from −∞ to +∞ (which can be assumed because the exponential decays very fast away from x0), and thus it can be calculated. We find A generalization of this method and extension to arbitrary precision is provided by . Suppose is a twice continuously differentiable function on and there exists a unique point such that: Then: Lower bound: Let .