Inequalities for Expectations

Cauchy-Schwartz Inequality

https://en.wikipedia.org/wiki/Cauchy%E2%80%93Schwarz_inequality

Theorem (Cauchy-Schwartz Inequality)

If and have finite variances, then

measures how much and align on average (positive when they tend to have the same sign and large together, negative when opposite). Cauchy-Schwartz essentially says that no matter how cleverly they align, the average product is bounded by the inner product of their root-mean-square.

If shows up and we don’t know the joint distribution well, Cauchy-Schwartz lets you reduce it to and which are often easier to control. It also shows up in covariance bounds

Jensen’s Inequality

https://en.wikipedia.org/wiki/Jensen%27s_inequality

A function is convex if for each and each ,

If is twice differentiable and for all , then is convex. It can be shown that if is convex, then lies above any line that touches at some point, called the tangent line. A function is concave if is convex.

Theorem (Jensen's Inequality)

If is convex, then

If is concave, then

This can be visualized pictorially below.

Proof

Let be a line, tangent to at the point . Since is convex, it lies above the line . So,

Sources

• Wasserman, L. (2010). All of Statistics: A concise Course in Statistical Inference. Chapter 4.2.