17.2 Approximation: in the model or through the computation

Standard frequentist methods often rely on assumptions, e.g., \(\chi^2\)-tests cash in the fact that, given enough data, it is safe to assume a normal distribution for data that is de facto not normally distributed. In this sense, frequentist statistics has approximation built into the models, but often uses clear-cut mathematical analysis to derive results based on these approximation assumptions.

Bayesian models, on the other hand, frequently do not make these approximations in their models. But since the posterior inference is hard, if not impossible to solve analytically, the Bayesian approach relies on approximating the Bayesian inference, e.g., via sampling techniques. In this way, the Bayesian approach shifts the approximation into the computation, not the researcher’s assumptions about the data-generating process as such.

Notice, however, that Bayesian models can incorporate the same (kind of) approximations the frequentist approach often critically relies on. At the same time, the frequentist approach can rely similarly on numerical approximation of its key quantitative notions. The next section shows an example of this, namely the approximate computation of \(p\)-values through Monte Carlo sampling.