Independent Random Variables

Definition

Two random variables $X$ and $Y$ are independent if, for every $A$ and $B$,

$$\mathbf{P}(X\in A,Y\in B)=\mathbf{P}(X\in A)\mathbf{P}(Y\in B),$$

and we write $X\perp Y$. Otherwise, we say that $X$ and $Y$ are dependent.

In principle, to check whether $X$ and $Y$ are independent, we must use the above to check for all subsets $A$ and $B$. Fortunately, we can use the following result.

Theorem

Let $X$ and $Y$ have joint PDF $f_{X,Y}$. Then $X\perp Y$ if and only if $f_{X,Y}(x,y)=f_X(x)f_Y(y)$ for all values $x$ and $y$.

The following result is helpful for verifying independence.

Theorem

Suppose that the range of $X$ and $Y$ is a (possibly infinite) rectangle. If $f(x,y)=g(x)h(y)$ for some functions $g$ and $h$ (not necessarily probability density functions), then $X$ and $Y$ are independent.

Example

Let $X$ and $Y$ have density

$$f(x,y)=\{\begin{array}{ll}2e^{-(x+2y)}& \text{if}x>0\text{and}y>0,\\ 0& \text{otherwise.}\end{array}$$

The range of $X$ and $Y$ is the rectangle $(0,\infty )\times (0,\infty )$. We can write $f(x,y)=g(x)h(y)$ where $g(x)=2e^{-x}$ and $h(y)=e^{-2y}$. Thus, $X\perp Y$.

Definition

If $X_1,\dots ,X_n$ are independent and each has the same marginal distribution with CDF $F$, we say that $X_1,\dots ,X_n$ are IID (independent and identically distributed) and we write

$$X_1,\dots ,X_n\sim F.$$

If $F$ has a density $f$, we also write $X_1,\dots ,X_N\sim f$. We also call $X_1,\dots ,X_n$ a random sample of size $n$ from $F$.