17.7 Evidence for the null

Frequentist analyses in the style of Neyman-Pearson do allow for a categorical decision to “accept the null-hypothesis”. This requires specification of a (point-valued) alternative hypothesis and it requires sufficient statistical power (see Section 16.4). Nonetheless, this approach still relies on using a \(p\)-value derived from the assumption that the null-hypothesis is true. This is still a measure of testing whether the null-model is a plausible model of the data. The frequentist approach does not offer a direct and intuitively interpretable measure of evidence in favor of the null-hypothesis.

Arguably, the most straightforward measure of evidence in favor of the null-hypothesis involves assigning some relative probability to it. This can only be achieved under a Bayesian approach. For example, using model comparison, a Bayesian approach to testing a null-hypothesis is able to conclude that there is evidence in favor of the null-hypothesis (when compared against some alternative) without this necessarily being tight to (i) a point-valued alternative hypothesis or (ii) a binary decision in favor of the null-hypothesis.