Chapter 4

Frequentist Inference

Frequentist inference is the process of determining properties of an underlying distribution via the observation of data.

Point Estimation

One of the main goals of statistics is to estimate unknown parameters. To approximate these parameters, we choose an estimator, which is simply any function of randomly sampled observations. To illustrate this idea, we will estimate the value of \( \pi \) by uniformly dropping samples on a square containing an inscribed circle. We define the estimator \( \hat{\pi} \) below, where \( m \) is the number of samples within our circle and \( n \) is the total number of samples dropped. It can be shown that this estimator has the desirable properties of being unbiased and consistent.

\(\hat{\pi} = 4\dfrac{m}{n}\) \( m= \) 0.00
\( n= \) 0.00
\( \hat{\pi}= \)
Drop 100 Samples
Drop 1000 Samples

Confidence Interval

In contrast to point estimators, confidence intervals estimate a parameter by specifying a range of possible values. Such an interval is associated with a confidence level, which is the probability that the procedure used to generate the interval will produce an interval containing the true parameter.

Choose a probability distribution to sample from.

Choose a sample size \((n)\) and confidence level \((1-\alpha)\).

Start sampling to generate confidence intervals.

Start Sampling

The Bootstrap

Much of frequentist inference centers on the use of "good" estimators. The precise distributions of these estimators, however, can often be difficult to derive analytically. The computational technique known as the Bootstrap provides a convenient way to estimate properties of an estimator via resampling. In this example, we resample with replacement from the empirical distribution function (which is itelf generated by sampling once from the population) in order to estimate the standard error of the sample mean.

Choose a probability distribution from which we will sample once to generate the empirical distribution function.

Choose a sample (and resampling) size \((n)\) and sample from your chosen distribution.


Resample to get an idea of the spread of the sample mean's distribution.

Resample 100 times