Conditional Expectation

Definition

The conditional expectation of $X$ given $Y=y$ is

$$\mathbb{E}(X|Y=y)=\{\begin{array}{ll}{\sum }_x{f}_{X|Y}(x|y)dx& \text{discrete case}\\ {\int }_x{f}_{X|Y}(x|y)dx& \text{continuous case.}\end{array}$$

If $r(x,y)$ is a function of $x$ and $y$, then

$$\mathbb{E}(r(X,Y)|Y=y)=\{\begin{array}{ll}\sum r(x,y)f_{X|Y}(x|y)dx& \text{discrete case}\\ \int r(x,y)f_{X|Y}(x|y)dx& \text{continuous case.}\end{array}$$

While $\mathbb{E}(X)$ is a number, $\mathbb{E}(X|Y=y)$ is a function of $y$. Before we observe $Y$, we do not know the value of $\mathbb{E}(X|Y=y)$, so it is a random variable which we denote $\mathbb{E}(X|Y)$.

Theorem (The Rule of Iterated Expectations)

For random variables $X$ and $Y$, assuming the expectations exists, we have that

$$\mathbb{E}[\mathbb{E}(Y|X)]=\mathbb{E}(Y),\text{and}\mathbb{E}[\mathbb{E}(X|Y)]=\mathbb{E}(X).$$

More generally, for any function $r(x,y)$, we have

$$\mathbb{E}[\mathbb{E}(r(X,Y)|X)]=\mathbb{E}(r(X,Y)).$$

Definition

The conditional variance is defined as

$$\mathbb{V}(Y|X=x)=\int (y-\mu (x){)}^{2}f(y|x)dy,$$

where $\mu (x)=\mathbb{E}(Y|X=x)$.

Theorem

For random variables $X$ and $Y$,

$$\mathbb{V}(Y)=\mathbb{E}[\mathbb{V}(Y|X)]+\mathbb{V}[\mathbb{E}(Y|X)].$$