Types of Convergence

Convergence in Probability and Convergence in Distribution

There are two main types of convergence.

Definition

Let be a sequence of random variables, and let be another random variable. Let denote the CDF of and let denote the CDF of .

1. converges to in probability, written , if, for every ,

as .
2. converges to in distribution, written , if

for all for which is continuous.

In words:

• means the random variables themselves get close with high probability.
• means only the distributions get close; the random variables do not have to be close on the same outcome.
• In probability, we compare the values of and at the same outcome .

When the limiting factor is a point mass, we change the notation slightly. If and , then we write . Similarly, if we write .

Same distribution, different convergence

Let and define .

Since the standard normal distribution is symmetric about , we have . So each has the same distribution as , and therefore .

But does not converge to in probability, because for odd we have on the same outcome . If , then , not . So

This does not get small with probability going to , so .

Convergence in both probability and distribution

Let and write

Then .

To show convergence in probability to , fix . Then

Since and has a fixed distribution, this probability goes to . So . Interestingly, this can also be shown using Markov’s Inequality. For any ,

as . Hence, .

To show convergence in distribution, let be the CDF of . For any ,

where is the standard normal CDF.

If , then , so .
If , then , so .
This is exactly the CDF of a point mass at , so .

Convergence in Quadratic Mean

There is another type of convergence which is useful for proving convergence in probability:

Definition

converges to in quadratic mean (also called convergence in ), written , if

As .

Again, if is a point mass at , we write instead of .

Convergence Properties

The following theorem gives the relationship between the types of convergence.

Theorem

The following relationships hold:

• a) implies that .
• b) implies that .
• c) If and if for some real number , then .

There are also some additional useful properties.

Theorem

Let be random variables. Let be a continuous function.

• (a) If and , then .
• (b) If and , then .
• (c) If and , then .
• (d) If and , then .
• (e) If and , then .
• (f) If , then .
• (g) If , then .

Sources

• Wasserman, L. (2010). All of Statistics: A concise Course in Statistical Inference. Chapter 5.