Continuous Random Variables

Uniform Real Distribution

https://en.wikipedia.org/wiki/Continuous_uniform_distribution

Definition

$X$ has a Uniform real distribution, denoted as $X\sim \text{Uniform}(a,b)$, if

$$f(x)=\{\begin{array}{ll}\frac{1}{b-a}& \text{for}x\in [a,b]\\ 0& \text{otherwise},\end{array}$$

where $a<b$. The distribution function is

$$F(x)=\{\begin{array}{ll}0& x<a\\ \frac{x-a}{b-a}& x\in [a,b]\\ 1& x>b.\end{array}$$

Normal (Gaussian) Distribution

https://en.wikipedia.org/wiki/Normal_distribution

Definition

$X$ has a Normal (or Gaussian) distribution with parameters $\mu$ and $\sigma$, denoted $X\sim \mathcal{N}(\mu ,{\sigma }^2)$, if

$$f(x)=\frac{1}{\sigma \sqrt{2\pi }}\text{exp}\left\{-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right\},x\in \mathbb{R}$$

where $\mu \in \mathbb{R}$ and $\sigma >0$. We say that $X$ has a standard Normal distribution if $\mu =0$ and $\sigma =1$. We also denote the PDF of a standard Normal as $\varphi (z)$ and the CDF as $\Phi (z)$. Note that there is no closed form for $\Phi (z)$.

Theorem

If $X\sim \mathcal{N}(\mu ,{\sigma }^2)$, then $Z=(X-\mu )/\sigma \sim \mathcal{N}(0,1)$. It also follows that

$$\begin{array}{rl}\mathbf{P}(a<X<b)& =\mathbf{P}\left(\frac{a-\mu }{\sigma }<Z<\frac{b-\mu }{\sigma }\right)\\ & =\Phi \left(\frac{b-\mu }{\sigma }\right)-\Phi \left(\frac{a-\mu }{\sigma }\right).\end{array}$$

Thus, we can compute any probability we want as long as we can compute the CDF $\Phi (z)$ for a standard Normal.

Theorem

if $Z\sim \mathcal{N}(0,1)$, then $X=\mu +\sigma Z\sim \mathcal{N}(\mu ,{\sigma }^2)$.

Theorem

If $X_i\sim \mathcal{N}({\mu }_i,{\sigma }_i^2)$ for $i=1,\dots ,n$ are independent, then

$$\sum _{i=1}^n{X}_i\sim \mathcal{N}\left(\sum _{i=1}^n{\mu }_i,\sum _{i=1}^n{\sigma }_i^2\right).$$

Generating Normal Samples

// Box-Muller transform
template <typename T>
auto normal(T mu, T std, Generator gen) -> T {
    const auto u1 = uniform_real_sample(0, 1, gen);
     const auto u2 = uniform_real_sample(0, 1, gen);
     // log(1 - u1) for u \in [0, 1) ensures log(> 0)
     const auto r = std::sqrt(static_cast<T>(-2) * std::log1p(-u1)); 
     const auto theta = static_cast<T>(2) * std::numbers::pi_v<T> * u2;
     const auto z1 = r * std::sin(theta);
     return std * z1 + mu;
}

Exponential Distribution

https://en.wikipedia.org/wiki/Exponential_distribution

Definition

The exponential distribution is the probability distribution of the distance (waiting times) between events in a Poisson point process. $X$ has an exponential distribution with parameter $\lambda$, denoted by $X\sim \text{Exp}(\lambda )$, if

$$f(x)=\frac{1}{\lambda }{e}^{-x/\lambda },x>0,$$

where $\lambda >0$.

Generating Exponential Samples

template <typename T>
auto normal(T lambda, Generator gen) -> T {
    const auto u = uniform_real_sample(0, 1, gen);
    return -std::log1p(-u) / lambda;
}

Gamma Distribution

https://en.wikipedia.org/wiki/Gamma_distribution

Distribution

The Gamma distribution is a versatile two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the gamma distribution. For $a>0$, the Gamma function is defined by

$$\Gamma (\alpha )={\int }_0^{\infty }{y}^{\alpha -1}{e}^{-y}dy$$

$X$ has a Gamma distribution with parameters $\alpha$ and $\beta$, denoted by $X\sim \text{Gamma}(\alpha ,\beta )$, if

$$f(x)=\frac{1}{{\beta }^{\alpha }\Gamma (\alpha )}{x}^{\alpha -1}{e}^{-x/\beta },x>0,$$

where $\alpha ,\beta >0$.

Theorem

If $X\sim \text{Gamma}(1,\beta )$, then $X$ has an exponential distribution with rate $\beta$.

Theorem

If $X\sim \text{Gamma}(v/2,v)$, then $X$ has a chi-squared distribution with rate $v$ degrees of freedom.

Theorem

If $X_i\sim \text{Gamma}({\alpha }_i,\beta )$ are independent, then

$$\sum _{i=1}^n{X}_i\sim \text{Gamma}\left(\sum _{i=1}^n{\alpha }_i,\beta \right).$$

Beta Distribution

https://en.wikipedia.org/wiki/Beta_distribution

Definition

The Beta distribution is a family of distributions defined on the interval $[0,1]$ in terms of two parameters $\alpha$ and $\beta$, and is often used to model random behavior of percentages and proportions. In Bayesian inference, the beta distribution is the conjugate prior probability distribution for the Bernoulli distribution, binomial distribution, and geometric distribution. $X$ has a Beta distribution with parameters $\alpha >0$ and $\beta >0$, denoted $X\sim \text{Beta}(\alpha ,\beta )$, if

$$f(x)=\frac{\Gamma (\alpha +\beta )}{\Gamma (\alpha )\Gamma (\beta )}{x}^{\alpha -1}(1-x{)}^{\beta -1},0<x<1.$$

Student’s t-Distribution

https://en.wikipedia.org/wiki/Student%27s_t-distribution

Definition

The $t$ distribution $t_{\nu }$ is a generalization of the standard normal distribution, which has heavier tails and the amount of mass in the tails is controlled by the $\nu$ parameter. $X$ has a $t$ distribution with $\nu$ degrees of freedom, written $X\sim t_{\nu }$, if

$$f(x)=\frac{\Gamma \left(\frac{\nu +1}{2}\right)}{\sqrt{\pi \nu }\Gamma \left(\frac{\nu }{2}\right)}{\left(1+\frac{x^2}{\nu }\right)}^{-(\nu +1)/2}.$$

The Normal distribution corresponds to $t$ with $\nu =\infty$, and the Cauchy distribution is a special case of the $t$ distribution corresponding to $\nu =1$.

Cauchy Distribution

https://en.wikipedia.org/wiki/Cauchy_distribution

Distribution

The Cauchy Distribution is a special case of the t-distribution with $\nu =1$. The density is given by

$$f(x)=\frac{1}{\pi (1+x^2)}.$$

Chi-Squared Distribution

https://en.wikipedia.org/wiki/Chi-squared_distribution

Definition

The ${\chi }^2$ distribution with $k$ degrees of freedom is the distribution of a sum of the squares of $k$ independent standard normal distributions. $X$ has a ${\chi }^2$ distribution with $k$ degrees of freedom, denoted $X\sim {\chi }_k^2$, if

$$f(x)=\frac{x^{(p/2)-1}{e}^{-x/2}}{\Gamma (p/2)2^{p/2}},x>0.$$

If $Z_1,\dots ,Z_k$ are independent standard Normal random variables, then ${\sum }_{i=1}^k{Z}_i^2\sim {\chi }_k^2$.