Point Estimation

Point estimation refers to providing a single best guess of some quantity of interest. This could be a parameter in a parametric model, a CDF , a probability density function , a regression function , or a prediction for a future value of some random variable.

By convention, we denote a point estimate of by or . Remember that is a fixed unknown quantity. The estimate depends on the data, and so is a random variable.

Definition

Let be IID data points from some distribution . A point estimator of a parameter is some function of :

The bias of the estimator is defined by

We say that is unbiased if . Many of the estimators studied are biased. A reasonable requirement for an estimator is that it should converge to the true parameter value as we collect more and more data.

Definition (Consistent)

A point estimator of a parameter is consistent if .

Definition (Sampling Distribution)

The distribution of is called the sampling distribution.

Limiting Normal Distribution

Definition

An estimator is asymptotically Normal if

Sources

• Wasserman, L. (2010). All of Statistics: A concise Course in Statistical Inference. Chapter 6.3.