Expectation of Random Variables

Expectation and Moments

Definition

The expected value, or mean, or first moment of a random variable $X$ is defined to be

$$\mathbb{E}(X)=\int xdF(x)=\{\begin{array}{ll}{\sum }_{x}xf(x)& \text{if}X\text{is discrete}\\ \int xf(x)dx& \text{if}X\text{is continuous}\end{array}$$

Assuming that the sum (or integral) is well defined. We use the following notation to denote the expected value of $X$:

$$\mathbb{E}(X)=\mathbb{E}X=\int xdF(x)=\mu ={\mu }_X.$$

Theorem (Rule of the Lazy Statistician)

Let $Y=r(X)$. Then

$$\mathbb{E}(Y)=\mathbb{E}(r(X))=\int r(x)d{F}_X(x).$$

Let $A$ be an event, and let $r(x)=I_A(x)$ where $I_A(x)=1$ if $x\in A$ and $I_A(x)=0$ if $x\notin A$. Then,

$$\mathbb{E}(I_A(X))=\int I_A(x)f_X(x)dx={\int }_A{f}_X(x)dx=\mathbf{P}(X\in A).$$

In other words, probability is a special case of expectation.

Definition

The $k$th moment of $X$ is defined to be $\mathbb{E}(X^k)$ assuming that $\mathbb{E}(|X{|}^k)<\infty$.

Theorem

If the $k$th moment exists, and if $j<k$, then the $j$th moment exists.

Definition

The $k$th central moment is defined to be $\mathbb{E}((X-\mu {)}^k)$.

Properties of Expectation

Theorem

If $X_1,\dots ,X_n$ are random variables, and $a_1,\dots ,a_n$ are constants, then

$$\mathbb{E}\left(\sum _i{a}_i{X}_i\right)=\sum _i{a}_i\mathbb{E}(X_i).$$

Theorem

Let $X_1,\dots ,X_n$ be independent random variables. Then,

$$\mathbb{E}\left(\prod _{i=1}^n{X}_i\right)=\prod _i\mathbb{E}(X_i).$$

Notice that the summation rule does not require independence, but the multiplication rule does.