Delta method

In statistics, the delta method is a result concerning the approximate probability distribution for a function of an asymptotically normal statistical estimator from knowledge of the limiting variance of that estimator. The delta method was derived from propagation of error, and the idea behind was known in the early 20th century. Its statistical application can be traced as far back as 1928 by T. L. Kelley. A formal description of the method was presented by J. L. Doob in 1935. Robert Dorfman also described a version of it in 1938. While the delta method generalizes easily to a multivariate setting, careful motivation of the technique is more easily demonstrated in univariate terms. Roughly, if there is a sequence of random variables Xn satisfying where θ and σ2 are finite valued constants and denotes convergence in distribution, then for any function g satisfying the property that its first derivative, evaluated at , exists and is non-zero valued. Demonstration of this result is fairly straightforward under the assumption that g′(θ) is continuous. To begin, we use the mean value theorem (i.e.: the first order approximation of a Taylor series using Taylor's theorem): where lies between Xn and θ. Note that since and , it must be that and since g′(θ) is continuous, applying the continuous mapping theorem yields where denotes convergence in probability. Rearranging the terms and multiplying by gives Since by assumption, it follows immediately from appeal to Slutsky's theorem that This concludes the proof. Alternatively, one can add one more step at the end, to obtain the order of approximation: This suggests that the error in the approximation converges to 0 in probability. By definition, a consistent estimator B converges in probability to its true value β, and often a central limit theorem can be applied to obtain asymptotic normality: where n is the number of observations and Σ is a (symmetric positive semi-definite) covariance matrix. Suppose we want to estimate the variance of a scalar-valued function h of the estimator B.