Distribution Functions

Cumulative Distribution Function

Definition (Cumulative distribution function)

The distribution ${\mu }_X$ is often given in terms of the distribution function, or cumulative distribution function (CDF), which is the function $F_X:\mathbb{R}\to [0,1]$, defined by

$$F_X(x)=\mathbf{P}(X\le x)={\mu }_X((-\infty ,x]).$$

The CDF effectively contains all the information about the random variable.

Theorem

Let $X$ have CDF $F$ and let $Y$ have CDF $G$. If $F(x)=G(x)$ for all $x$, then $\mathbf{P}(X\in A)=\mathbf{P}(Y\in A)$ for all $A$.

Note that $F=F_X$ satisfies the following:

• ${\lim }_{x\to -\infty }F(x)=0$
• ${\lim }_{x\to \infty }F(x)=1$
• $F$ is a non-decreasing function
• $F$ is right continuous, i.e, for every $x$, $F(a)={\lim }_{x\to a+}$ F(x)

Probability Mass Function

Definition (Probability mass function)

$X$ is discrete if it takes countably many values. The probability function or probability mass function (PMF) of a discrete random variable $X$ is the function $f_X(x)=\mathbf{P}(X=x)$.

The PMF must satisfy that $f_X(x)\ge 0$ for all $x\in \mathbb{R}$, and that ${\sum }_i{f}_X(x_i)=1$. The CDF of $X$ is related to $f_X$ by

$$F_X(x)=\mathbf{P}(X\le x)=\sum _{x_i\le x}{f}_X(x_i).$$

Probability Density Function

Definition (Probability density function)

A random variable $X$ is continuous if there exists a function $f_X$ such that $f_X(x)\ge 0$ for all $x$, ${\int }_{-\infty }^{\infty }{f}_X(x)dx=1$, and for every $a\le b$,

$$\mathbf{P}(a<X<b)={\int }_a^b{f}_X(x)dx.$$

The function $f_X$ is called the probability density function (PDF). If the density exists, then we have that

$$F_X(x)={\int }_{-\infty }^x{f}_X(t)dt,$$

and $f_X(x)=F_X^{\prime }(x)$ at all points $x$ at which $F_X$ is differentiable.

A density $f$ satisfies

$${\int }_{\infty }^{\infty }f(x)dx=1.$$

Note that if $X$ is continuous, then $\mathbf{P}(X=x)=0$ for every $x$. It is not valid to think of $f(x)$ as $\mathbf{P}(X=x)$; only for discrete random variables. We get probabilities from PDF by integrating. A PDF can be bigger than 1, unlike the mass function.

Lemma

Let $F$ be the CDF for a random variable $X$. Then:

• $\mathbf{P}(x<X\le y)=F(y)-F(x)$
• $\mathbf{P}(X>x)=1-F(x)$

Quantile Function

Definition (Quantile function)

Let $X$ be a random variable with CDF $F$. The inverse CDF or quantile function is defined by

$$F^{-1}(q)=\text{inf}\left\{x:F(x)>q\right\},$$

for $q\in [0,1]$. If $F$ is strictly increasing and continuous, then $F^{-1}(q)$ is the unique real number $x$ such that $F(x)=q$.