Penalising model component complexity: A principled, practical approach to constructing priors

The authors introduce a new methodology to design priors for models with a nested structure, i.e. for which the model component is defined to be an extension of a base model (for instance the Negative Binomial Distribution is an extension of the Poisson distribution). Penalized Complexity (PC) Priors are designed to penalize the complexity for deviating from the base model.

For a model \(\phi(x|\zeta)\) with flexibility parameter \(\zeta\) the base model is the simplest model in the class and we asssume it corresponds to \(\zeta = 0\).

Gaussian Random Effects \(z | \zeta\) is a multivariate normal with precision matrix \(\tau \mathbb{1}\) where \(\tau = \zeta^{-2}\). The base model is the model that puts all the mass at \(\zeta =0\). For the multivariate case we can allow for correlations among components, and uncorrelated case is the base model.