The Law of Large Numbers

The following theorem states that, the distribution of becomes more concentrated around as gets large.

Theorem (The Weak Law of Large Numbers)

If are IID, then .

Proof

Assume that . This is not necessary but it simplifies the proof. Using Chebyshev’s inequality,

which tends to 0 as .

Example

Consider flipping a coin for which the probability of heads is . Let denote the outcome of a single toss (0 or 1). Suppose that . How large should be so that ? Since and , from Chebyshev’s inequality,

The last expression will be larger than 0.7 if .

Sources

• Wasserman, L. (2010). All of Statistics: A concise Course in Statistical Inference. Chapter 5.3.