Method of Moments

The Method of Moments

Info

Suppose that the parameter $\theta =({\theta }_1,\dots ,{\theta }_k)$ has $k$ components. For $1\le j\le k$, define the $j$th moment

$${\alpha }_j\equiv a_j(\theta )={\mathbb{E}}_{\theta }(X^j)=\int x^{j}d{F}_{\theta }(x)$$

and the $j$th sample moment

$${\hat{\alpha }}_j=\frac{1}{n}\sum _{i=1}^n{X}_i^j.$$

Definition (Method of Moments Estimator)

The method of moments estimator ${\hat{\theta }}_n$ is defined to be the value of $\theta$ such that

$$\begin{array}{ccc}{\alpha }_1({\hat{\theta }}_n)& =& {\hat{\alpha }}_1\\ {\alpha }_2({\hat{\theta }}_n)& =& {\hat{\alpha }}_2\\ ⋮& ⋮& ⋮\\ {\alpha }_k({\hat{\theta }}_n)& =& {\hat{\alpha }}_k.\end{array}$$

This defines a system of $k$ equations with $k$ unknowns.

Theorem

Let ${\hat{\theta }}_n$ denote the method of moments estimator. Under appropriate conditions on the model, the following statements hold:

1. The estimate ${\hat{\theta }}_n$ exists with probability tending to 1.
2. The estimate is consistent: ${\hat{\theta }}_n\stackrel{\mathrm{P}}{\to }\theta$.
3. The estimate is asymptotically Normal:

$$\sqrt{n}({\hat{\theta }}_n-\theta )⇝N(0,\Sigma )$$

where $\Sigma =g{\mathbb{E}}_{\theta }(Y{Y}^T)g^T$, $Y=(X,X^2,\dots ,X^k{)}^T$, $g=g(g_1,\dots ,g_k)$ and $g_j=\partial {\alpha }_j^{-1}(\theta )/\partial \theta$.

The last statement above can be used to find standard errors and confidence intervals. However, bootstrapping provides an easier method.

Example

Let $X_1,\dots ,X_n\sim \mathrm{Bernoulli}(p)$. Then ${\alpha }_1={\mathbb{E}}_p(X)=p$ and ${\hat{\alpha }}_1=n^{-1}{\sum }_{i=1}^n{X}_i$. By equating these we get the estimator

$${\hat{p}}_n=\frac{1}{n}\sum _{i=1}^n{X}_i.$$

Example

Let $X_1,\dots ,X_n\sim \mathrm{Normal}(\mu ,{\sigma }^2)$. Then ${\alpha }_1={\mathbb{E}}_{\theta }(X)=\mu$ and ${\alpha }_2={\mathbb{E}}_{\theta }(X^2)={\mathbb{V}}_{\theta }(X)+({\mathbb{E}}_{\theta }[X]{)}^2={\sigma }^2+{\mu }^2$. We need to solve the equations

$$\begin{array}{rl}\hat{\mu }& =\frac{1}{n}\sum _{i=1}^n{X}_i\\ {\hat{\sigma }}^2+{\hat{\mu }}^2& =\frac{1}{n}\sum _{i=1}^n{X}_i^2.\end{array}$$

This is a system of 2 equations with 2 unknowns. The solution is

$$\begin{array}{rl}\hat{\mu }& ={\bar{X}}_n\\ {\hat{\sigma }}^2& =\frac{1}{n}\sum _{i=1}^n{\left(X_i-{\bar{X}}_n\right)}^2.\end{array}$$