Statistical Functionals

Definition (Statistical Functional)

A statistical functional $T(F)$ is any function of $F$, i.e., you input a distribution (CDF) and it returns a number.

Definition (Plug-in Estimator)

The plug-in estimator of $\theta =T(F)$ is defined by

$${\hat{\theta }}_n=T({\hat{F}}_n).$$

In other words, just plug in ${\hat{F}}_n$ for the unknown $F$.

Definition (Linear Functional)

If

$$T(F)=\int r(x)dF(x)$$

for some function $r(x)$, then $T$ is called a linear functional.

The reason $T(F)=\int r(x)dF(x)$ is called a linear functional is because $T$ satisfies

$$T(aF+bG)=aT(F)+bT(G),$$

hence $T$ is linear in its arguments.

Theorem

The plug-in estimator for linear functional $T=\int r(x)dF(x)$ is

$$T({\hat{F}}_n)=\int f(x)d{\hat{F}}_n(x)=\frac{1}{n}\sum _{i=1}^{n}r(X_i).$$

Definition (Normal-based Interval)

In many cases, it turns out that

$$T({\hat{F}}_n)\approx N(T(F),\widehat{{\mathrm{SE}}^2}).$$

By normal-based confidence intervals, an approximate $1-\alpha$ confidence interval for $T(F)$ is then

$$T({\hat{F}}_n)\pm z_{\alpha /2}\widehat{\mathrm{SE}}.$$

We call this the Normal-based interval.

Mean of a Distribution

The mean of a distribution is a statistical functional

$$\mu =T_{\mathrm{mean}}(F)=\int xdF(x).$$

When $F$ has a PDF $f(x)$, $dF(x)=f(x)dx$. The plug-in estimator is

$$\hat{\mu }=\int xd{\hat{F}}_n(x)={\bar{X}}_n.$$

The standard error is $\mathrm{SE}=\sqrt{\mathbb{V}({\bar{X}}_n)}=\sigma /\sqrt{n}$. If $\hat{\sigma }$ denotes an estimate of $\sigma$, then the estimated standard error is $\hat{\sigma }/\sqrt{n}$. A Normal-based confidence interval for $mu$ is ${\bar{X}}_n\pm z_{\alpha /2}\widehat{\mathrm{SE}}$.

Variance of a Distribution

The variance of a distribution is a statistical functional

$${\sigma }^2=T_{\mathrm{var}}(F)=\int (x-\mu {)}^{2}dF(x).$$

When $F$ has a PDF $f(x)$, $dF(x)=f(x)dx$. The plug-in estimator is

$$\begin{array}{rl}\hat{\sigma }& =\int x^{2}d/{\hat{F}}_n(x)-{\left(\int xd{\hat{F}}_n(x)\right)}^2\\ & =\frac{1}{n}\sum _{i=1}^n{X}_i^2-{\left(\frac{1}{n}\sum _{i=1}^n{X}_i\right)}^2\\ & =\frac{1}{n}\sum _{i=1}^n{\left(X_i-{\bar{X}}_n\right)}^2.\end{array}$$

Another reasonable estimator of ${\sigma }^2$ is the sample variance

$$S_{n-1}^2=\frac{1}{n-1}\sum _{i=1}^n{\left(X_i-{\bar{X}}_n\right)}^2.$$

In practice, there is little difference between ${\hat{\sigma }}^2$ and $S_{n-1}^2$.

Median of a Distribution

Using the concept of a statistical functional, median and any quantile can be easily defined. The median of a distribution $F$ is a point ${\theta }_{\mathrm{med}}$ such that $F({\theta }_{\mathrm{med}})=0.5$. Thus,

$$m=T_{\mathrm{med}}=F^{-1}(0.5).$$