16 Null Hypothesis Significance Testing

In this chapter, we are going to look at basic notions and principles of frequentist statistics. Section 16.1 sets the scene by describing what frequentist statistics is and how it differs from Bayesian approaches. The key notion of a \(p\)-value is discussed in Section 16.2. Section 16.3 takes an excursion to introduce the Central Limit Theorem. Section 16.4 shortly surveys the main ideas of the Neyman-Pearson approach to statistical inference, including notions like \(\beta\)-error and statistical power. Section 16.5 covers confidence intervals. Finally, Section 16.6 introduces some of the most commonly used frequentist tests from a model-centric perspective.

The learning goals for this chapter are:

become familiar with frequentist hypothesis testing
- see the differences between the Bayesian and the frequentist approaches
understand key statistical notions such as:
- sampling distribution
- \(p\)-value
- confidence interval
- statistical significance
understand and become able to apply and interpret basic frequentist tests:
- binomial test, \(t\)-tests, ANOVA, linear regression, \(\chi^2\)-test, likelihood-ratio test