The Delta Method

If $Y_n$ has a limiting Normal distribution, then the delta method allows us to find the limiting distribution of $g(Y_n)$ where $g$ is any smooth function.

Theorem (The Delta Method)

Suppose that

$$\frac{\sqrt{n}(Y_n-\mu )}{\sigma }⇝N(0,1),$$

and that $g$ is a differentiable function such that $g^{\prime }(\mu )\ne 0$. Then,

$$\sqrt{n}\frac{\left(g(Y_n)-g(\mu )\right)}{|g^{\prime }(\mu )|\sigma }⇝N(0,1).$$

In other words,

$$Y_n\approx N\left(\mu ,\frac{{\sigma }^2}{n}\right)\text{imples that}g(Y_n)\approx N\left(g(\mu ),(g^{\prime }(\mu ){)}^2\frac{{\sigma }^2}{n}\right).$$

Example

Let $X_1,\dots ,X_n$ be IID with finite mean $\mu$ and finite variance ${\sigma }^2$. By the central limit theorem, $\sqrt{n}({\overline{X}}_n-\mu )/\sigma ⇝N(0,1)$. Let $W_n=e^{{\overline{X}}_n}$. Thus, $W_n=g({\overline{X}}_n)$ where $g(s)=e^s$. Since $g^{\prime }(s)=e^s$, the delta method implies that

$$W_n\approx N\left(e^{\mu },\frac{e^{2\mu }{\sigma }^2}{n}\right),$$

There is also a multivariant version of the Delta Method.

Theorem (The Multivariate Delta Method)

Suppose that $Y_n=(Y_{n_1},\dots ,Y_{nk})$ is a sequence of random vectors such that

$$\sqrt{n}(Y_n-\mu )⇝N(0,\Sigma ).$$

Let $g:{\mathbb{R}}^k\to \mathbb{R}$ and let

$$\nabla g(y)=\left(\begin{array}{c}\frac{\partial g}{\partial y_1}\\ ⋮\\ \frac{\partial g}{\partial y_k}\end{array}\right).$$

Let ${\nabla }_{\mu }$ denote $\nabla g(y)$ evaluated at $y=\mu$ and assume that the elements of ${\nabla }_{\mu }$ are nonzero. Then,

$$\sqrt{n}(g(Y_n)-g(\mu ))⇝N(0,{\nabla }_{\mu }^T\Sigma {\nabla }_{\mu }).$$