Gibbs sampling with JAGS: behind the scenes

Coro G.

Gibbs sampling  JAGS  Bayesian Inference  Markov Chains  Markov Chain Monte Carlo 

Gibbs sampling is a Bayesian inference technique that is used in various scientific domains to generate samples from a certain posterior probability density function, given experimental data. Several software implementations of Gibbs sampling exist, which generally adopt very different approaches, because it is not easy to make a Gibbs sampling implementation exactly correspond to the theoretical approach. In particular, these implementations may use different approximation algorithms to and solutions to sub-steps of the Gibbs sampling process. Scientists working in different domains often use Gibbs sampling software without knowing the details of the implementation. Nevertheless, it is our experience that understanding the implementation can be crucial to enhance the performance of a model, because a software configuration conceived to help the underlying implementation may end in better approximation of the estimated probabilities functions. JAGS (Just Another Gibbs Sampler) is a widely used open-source implementation of Gibbs sampling. Its installation and user's guide are accurate, but do not indicate how the software really implements Gibbs sampling and it is not easy to infer this information from the source code. The aim of this paper is to give a high-level overview of the JAGS algorithms and its extensions that implement Gibbs sampling. Our target reader is a scientist who may want to understand the basic concepts underlying Bayesian inference and Gibbs sampling and who want to be aware of what happens behind the scenes when building a model.