Parametric and Nonparametric Models

Definition

A statistical model $\mathfrak{F}$ is a set of distributions (or densities or regression functions). A parametric model is a set $\mathfrak{F}$ that can be parameterized by a finite number of parameters.

Example

If we assume that the data come from a Normal distribution, then the model is

$$\mathfrak{F}=\left\{f(x;\mu ,\sigma )=\frac{1}{\sigma \sqrt{2\pi }}\exp \left\{-\frac{1}{2{\sigma }^2}(x-\mu {)}^2\right\},\mu \in \mathbb{R},\sigma >0\right\}.$$

This is a two-parameter model.

We write the density as $f(x;\mu ,\sigma )$ to show that $x$ is the value of the rando variable, whereas $\mu$ and $\sigma$ are parameters.

Definition

In general, a parametric model takes the form

$$\mathfrak{F}=\left\{f(x;\theta ):\theta \in \Theta \right\}$$

where $\theta$ is an unknown parameter (or vector of parameters) that can take values in the parameter space $\Theta$.