Transformations of Random Variables

Single Random Variable Transformations

Suppose that $X$ is a random variable with PDF $f_X$ and CDF $F_X$. Let $Y=r(X)$ be a function of $X$; for example, $Y=X^2$. We call $Y=r(X)$ a transformation of $X$. In the discrete case, the mass function of $Y$ is given by

$$\begin{array}{rl}{f}_Y(y)& =\mathbf{P}(Y=y)=\mathbf{P}(r(X)=y)\\ & =\mathbf{P}(\{x;r(x)=y\})=\mathbf{P}(X\in r^{-1}(y)).\end{array}$$

Example

Suppose that $\mathbf{P}(X=-1)=\mathbf{P}(X=1)=1/4$ and $\mathbf{P}(X=0)=1/2$. Let $Y=X^2$. Then, $\mathbf{P}(Y=0)=P(X=0)=1/2$, and $\mathbf{P}(Y=1)=\mathbf{P}(X=1)+\mathbf{P}(X=-1)=1/2$. Note that $Y$ takes fewer values than $X$ because the transformation is not one-to-one.

In the continuous case, there are three steps for finding $f_Y$:

1. For each $y$, find the set $A_y=\{x:r(x)\le y\}$
2. Find the CDF

$$\begin{array}{rl}{F}_Y(y)& =\mathbf{P}(Y\le y)=\mathbf{P}(r(X)\le y)\\ & =\mathbf{P}(\{x;r(x)\le y\})\\ & ={\int }_{A_y}{f}_X(x)dx\end{array}$$

3. The PDF is $f_Y(y)=F_Y^{\prime }(y)$

Example

Let $f_X(x)=e^{-x}$ for $x>0$. Hence,

$$F_X(x)={\int }_0^{\infty }{f}_X(s)ds=1-e^{-x}.$$

Let $Y=r(X)=\log (X)$. Then, $A_y=\{x:x\le e^y\}$ and

$$\begin{array}{rl}{F}_Y(y)& =\mathbf{P}(Y\le y)=\mathbf{P}(\log (X)\le y)\\ & =\mathbf{P}(X\le e^y)=F_X(e^y)=1-e^{-e^y}.\end{array}$$

Therefore, $f_Y(y)=e^y{e}^{-e^y}$ for $y\in \mathbb{R}$.

Theorem

When $r$ is strictly monotone increasing or strictly monotone decreasing, then $r$ has an inverse $s=r^{-1}$ and in this case one can show that

$$f_Y(y)=f_X(s(y))\left|\frac{ds(y)}{dy}\right|.$$

Multiple Random Variable Transformations

If $X$ and $Y$ are random variables, we might want to know the distribution of $X/Y$, $X+Y$, or $\text{max}\{X,Y\}$. Let $Z=r(X,Y)$ be the function of interest. The steps for finding $f_Z$ are as follows:

1. For each $z$, find the set $A_z=\{(x,y):r(x,y)\le z\}$
2. Find the CDF

$$\begin{array}{rl}{F}_Z(z)& =\mathbf{P}(Z\le z)=\mathbf{P}(r(X,Y)\le z)\\ & =\mathbf{P}(\{(x,y);r(x,y)\le z\})\\ & =\int {\int }_{A_z}{f}_{X,Y}(x,y)dxdy.\end{array}$$

3. The PDF is $f_Z(z)=F_Z^{\prime }(z)$

Example

let $X_1,X_2\sim \text{Uniform}(0,1)$ be independent. Find the density of $Y=X_1+X_2$. The joint density of $(X_1,X_2)$ is

$$f(x_1,x_2)=\{\begin{array}{ll}1& 0<x_1<1,0<x_2<1\\ 0& \text{otherwise}.\end{array}$$

let $r(x_1,x_2)=x_1+x_2$. Now,

$$\begin{array}{rl}{F}_Y(y)& =\mathbf{P}(Y\le y)=\mathbf{P}(r(X_1,X_2)\le y)\\ & =\mathbf{P}(\{(x_1,x_2):r(x_1,x_2)\le y\})\\ & =\int {\int }_{A_y}f(x_1,x_2)d{x}_{1}d{x}_2.\end{array}$$

Finding $A_y$ is non-trivial. First, suppose that $0<y\le 1$. Then $A_y$ is the triangle with vertices $(0,0)$, $(y,0)$, and $(0,y)$. In this case, the above integral is the area of this triangle which is $y^2/2$. If $1<y<2$, then $A_y$ is everything in the unique square except the triangle with vertices $(1,y-1)$, $(1,1)$, $(y-1,1)$. This set has area $1-(2-y{)}^2/2$. Therefore,

$$F_Y(y)=\{\begin{array}{ll}0& y<0\\ \frac{y^2}{2}& 0\le y<1\\ 1-\frac{(2-y{)}^2}{2}& 1\le y<2\\ 1& y\ge 2.\end{array}$$

By differentiation, the PDF is

$$f_Y(y)=\{\begin{array}{ll}y& 0\le y\le 1\\ 2-y& 1\le y\le 2\\ 0& \text{otherwise.}\end{array}$$