Likelihood Ratio Test

The Likelihood Ratio Test

The Wald test is useful for testing a scalar parameter. The likelihood ratio test is more general and can be used for testing a vector-valued parameter.

Definition

Consider testing

$$H_0:\theta \in {\Theta }_0\mathrm{versus}{H}_1:\theta \notin {\Theta }_0$$

The likelihood ratio statistic is

$$\lambda =-2\log \left(\frac{{\sup }_{\theta \in {\Theta }_0}\mathcal{L}(\theta )}{{\sup }_{\theta \in \Theta }\mathcal{L}(\theta )}\right)=-2\log \left(\frac{\mathcal{L}({\hat{\theta }}_0)}{\mathcal{L}(\hat{\theta })}\right)$$

where $\hat{\theta }$ is the MLE and ${\hat{\theta }}_0$ is the MLE when $\theta$ is restricted to lie in ${\Theta }_0$.

The multiplication by $-2$ ensures that (by Wilk’s theorem) $\lambda$ converges asymptotically to being Chi-squared distributed if the null hypothesis happens to be true.

Interpretation

The numerator corresponds to the likelihood of an observed outcome under the null hypothesis. The denominator corresponds to the maximum likelihood of an observed outcome, varying the parameters over the whole parameter space. Because the numerator of the ratio is less than the denominator, the ratio is between 0 and 1. Low values of the likelihood ratio mean that the observed result was much less likely to occur under the null hypothesis as compared to the alternative. High values of the statistic mean that the observed outcome was nearly as likely to occur under the null hypothesis as the alternative, and so the null hypothesis cannot be rejected.