Part III. Computational Bayesian methods

Part III

The numerical machinery that turns a posterior you can write down into one you can actually use: integration, Monte Carlo error, MCMC, Hamiltonian methods, variational approximation, and sequential updating.

Part overview

Outside conjugate families, a posterior is defined by an integral nobody can evaluate. Part III is about the ways round that, and about the fact that every one of them is an approximation with its own failure mode.

The order is deliberate. Deterministic approximation comes first, because it is cheap and because knowing when a Laplace approximation suffices saves a great deal of sampling. Independent Monte Carlo comes next, since its error analysis is the baseline against which dependent sampling is judged. Only then does MCMC arrive, followed by the gradient-based methods that exploit posterior geometry, the optimisation-based alternative in variational inference, and the sequential methods for data that arrive over time.

The part closes with a case where the difficulty is structural rather than numerical: a model whose parameter vector changes length as the sampler runs.

Part learning goals

By the end of Part III you should be able to:

  • say which posterior quantity is being approximated, by what mechanism, and with what error;
  • implement Monte Carlo, Metropolis-Hastings, and Gibbs samplers from scratch;
  • tune and diagnose random-walk and HMC-type samplers, and tell a geometric problem from a tuning problem;
  • choose between MCMC, variational, and sequential methods on the basis of what each one sacrifices;
  • handle a model in which the number of parameters is itself unknown.

Chapters

Chapter What it covers
Numerical integration and approximation Quadrature, Laplace approximation, importance sampling, bridge sampling
Monte Carlo methods Monte Carlo error, the CLT, variance reduction, weighted ESS
Markov chain Monte Carlo Detailed balance, Metropolis-Hastings, Gibbs, autocorrelation, diagnostics
Advanced MCMC Hamiltonian dynamics, leapfrog, NUTS, adaptation, divergences
Variational inference ELBO, mean-field factorisation, coordinate ascent, ADVI
Sequential and online Bayes Recursive filtering, particle methods, degeneracy, state-space models
The telescoping sampler Mixtures with unknown \(K\), transdimensional sampling without reversible jump
Table 1: Chapters in Part III

Reading path

Work through them in order. The first two are short and the payoff comes later: the error analysis in Monte Carlo methods is what makes the MCMC diagnostics interpretable rather than ritual.

  1. Numerical integration and approximation
  2. Monte Carlo methods
  3. Markov chain Monte Carlo
  4. Advanced MCMC: HMC and NUTS
  5. Variational inference
  6. Sequential and online Bayes
  7. The telescoping sampler

The last chapter depends on mixture models from Part IV. Read that first, or take the mixture model on trust and come back.

Chapter 7. Numerical integration and approximation

Core question

What can be done before MCMC, and when are deterministic or semi-deterministic approximations sufficient?

Section blueprint

  • Quadrature in low dimensions.
  • Laplace approximation.
  • Importance sampling.
  • Bridge sampling.
  • Thermodynamic integration overview.
  • Variational ideas preview.

Derivations that must appear

  • Laplace approximation from second-order Taylor expansion of the log posterior.
  • Importance sampling identity and variance formula.
  • Self-normalized importance sampling.
  • Bridge sampling intuition through ratio-of-normalizing-constants arguments.

Computational lab

  • Compare exact integration, Laplace approximation, and importance sampling on a one-dimensional posterior.
  • Use bridge sampling software or pseudocode to estimate a marginal likelihood.

Common pitfalls

  • Applying Laplace approximation to multi-modal or heavily skewed posteriors.
  • Ignoring weight degeneracy in importance sampling.
  • Using marginal likelihood approximations as black boxes.

Exercises

  • Derive the Laplace approximation for a scalar posterior around its mode.
  • Show how importance weights behave under tail mismatch.

Core references

  • Robert and Casella (2004)
  • MacKay (2003)
  • Gelman et al. (2013)

Chapter 8. Monte Carlo methods

Core question

How does simple sampling approximate posterior expectations, and how should Monte Carlo error be quantified?

Section blueprint

  • Monte Carlo integration.
  • Law of large numbers in simulation.
  • Monte Carlo standard error.
  • Variance reduction: antithetic variables, control variates, stratification.
  • Effective sample size ideas in independent sampling.

Derivations that must appear

  • Unbiasedness of Monte Carlo estimators.
  • Monte Carlo CLT and variance formula.
  • Control variate optimal coefficient.
  • ESS interpretation under weighted samples.

Computational lab

  • Estimate posterior means by direct simulation in a conjugate model.
  • Compare variance reduction methods empirically.
  • Plot Monte Carlo error decay on log-log axes.

Common pitfalls

  • Reporting Monte Carlo estimates without uncertainty.
  • Confusing independent-sample ESS with MCMC ESS.
  • Using a control variate with poorly estimated correlation.

Exercises

  • Derive the variance of a Monte Carlo estimator.
  • Construct an antithetic estimator for a monotone function of a uniform random variable.
  • Estimate ESS for an importance-sampling run.

Core references

  • Robert and Casella (2004)
  • Casella and Berger (2002)

Full chapter

Chapter 9. Markov chain Monte Carlo

Core question

How can a Markov chain be designed to have the posterior as its invariant distribution?

Section blueprint

  • Markov chain preliminaries.
  • Detailed balance and reversibility.
  • Metropolis and Metropolis-Hastings algorithms.
  • Gibbs sampling.
  • Blocked and collapsed Gibbs sampling.
  • Adaptive MCMC overview.
  • Mixing, autocorrelation, ESS, and convergence.

Derivations that must appear

  • Detailed balance proof for Metropolis-Hastings.
  • Acceptance ratio derivation.
  • Gibbs sampling as exact conditional replacement.
  • Autocorrelation-based ESS formula and interpretation.

Computational lab

  • Random-walk Metropolis in R.
  • Gibbs sampler for a conjugate normal model.
  • Tuning proposal scale and comparing acceptance rates.

Common pitfalls

  • Equating acceptance rate with overall sampler quality.
  • Using a single chain and overtrusting visual convergence.
  • Ignoring high autocorrelation when reporting Monte Carlo precision.

Exercises

  • Derive the MH ratio for an independence sampler.
  • Show that Gibbs sampling satisfies detailed balance.
  • Compare random-walk proposals with different scales on the same target.

Core references

  • Metropolis et al. (1953)
  • Hastings (1970)
  • Geman and Geman (1984)
  • Andrieu et al. (2003)
  • Brooks et al. (2011)

Full sample chapter

Chapter 10. Advanced MCMC

Core question

How do gradient-informed samplers exploit geometry to explore high-dimensional posteriors more efficiently than random walks?

Section blueprint

  • Hamiltonian Monte Carlo.
  • Hamiltonian dynamics and potential energy.
  • Leapfrog integration.
  • No-U-Turn Sampler.
  • Step size and mass matrix adaptation.
  • Warmup and adaptation schedules.
  • Centered versus non-centered parameterization.
  • Slice sampling and elliptical slice sampling.
  • Riemannian manifold ideas overview.
  • Reversible jump MCMC overview.

Derivations that must appear

  • Hamiltonian equations for position and momentum.
  • Leapfrog update map and symplectic intuition.
  • Metropolis correction for discretization error.
  • Why reparameterization changes geometry but not the posterior.

Computational lab

  • Visualize HMC trajectories on a correlated Gaussian target.
  • Compare centered and non-centered eight-schools parameterizations in Stan.
  • Diagnose divergences and E-BFMI.

Common pitfalls

  • Increasing adapt_delta without addressing geometric pathologies.
  • Misinterpreting divergences as merely numerical annoyances.
  • Forgetting that tuning improves exploration only when the model geometry is sensible.

Exercises

  • Derive leapfrog updates for a separable Hamiltonian.
  • Explain why HMC avoids random-walk behavior in smooth targets.
  • Analyze a funnel-shaped posterior and propose a reparameterization.

Core references

Chapter 11. Variational inference

Core question

How can posterior inference be reframed as an optimization problem over tractable approximating distributions?

Section blueprint

  • KL divergence and variational families.
  • ELBO.
  • Mean-field approximation.
  • Coordinate ascent VI.
  • Stochastic VI.
  • ADVI.
  • Amortized VI overview.
  • Comparison with MCMC.

Derivations that must appear

  • ELBO derivation from \(\log p(y)\).
  • Mean-field coordinate updates in a conjugate setting.
  • Gradient estimators and stochastic optimization overview.
  • Forward versus reverse KL geometry discussion.

Computational lab

  • Derive a simple coordinate ascent VI procedure in a conjugate model.
  • Compare ADVI-style approximations with MCMC on a skewed posterior.

Common pitfalls

  • Assuming VI error is negligible because optimization converged.
  • Underestimating posterior variance due to mean-field assumptions.
  • Using ELBO values to compare arbitrary models without care.

Exercises

  • Derive the ELBO identity.
  • Show how a mean-field factorization removes posterior dependence.
  • Compare reverse-KL behavior with a multi-modal target conceptually.

Core references

  • Blei et al. (2017)
  • Kucukelbir et al. (2017)

Chapter 12. Sequential and online Bayesian computation

Core question

How can posterior inference be updated over time when data arrive sequentially or latent states evolve dynamically?

Section blueprint

  • Sequential importance sampling and resampling.
  • Particle filters.
  • Sequential Monte Carlo.
  • Particle MCMC overview.
  • Online updating and streaming Bayes.
  • State-space models.
  • Hidden Markov models.
  • Dynamic linear models.

Derivations that must appear

  • Recursive Bayesian filtering identity.
  • Particle weight updates and normalization.
  • Effective sample size for particle systems.
  • Kalman filter equations in the linear Gaussian case.
  • Forward algorithm for hidden Markov models.

Computational lab

  • Implement a bootstrap particle filter for a simple state-space model.
  • Compare Kalman filtering with simulation in a local-level model.
  • Track weight degeneracy over time.

Common pitfalls

  • Ignoring particle degeneracy.
  • Using sequential methods without checking numerical stability.
  • Confusing filtering, smoothing, and prediction targets.

Exercises

  • Derive the recursive filtering equation.
  • Implement multinomial resampling and study its variability.
  • Write the forward recursion for a finite-state HMM.

Core references

  • Doucet et al. (2001)
  • Del Moral (2004)
  • Gelman et al. (2013)

Chapter 13. The telescoping sampler

Core question

How can a sampler explore a model whose parameter vector changes length, without the bespoke machinery that reversible jump demands?

Section blueprint

  • Components versus clusters: \(K\) against \(K_+\).
  • The generalised mixture of finite mixtures.
  • Static and dynamic specifications, and the Dirichlet process limit.
  • Priors on the number of components.
  • The partition marginal and the conditional posterior of \(K\).
  • The telescoping step and refilling from the prior.
  • Summarising posteriors over partitions.

Derivations that must appear

  • The allocation marginal, integrating the Dirichlet weights out.
  • The combinatorial factor \(K!/(K-K_+)!\) relating labelled allocations to partitions.
  • The conditional posterior \(p(K \mid \mathcal{C})\), and why the component parameters and the data drop out of it.
  • Why the dimension-changing move needs no Jacobian and no Metropolis correction.

Computational lab

  • Implement the telescoping sampler for a univariate normal mixture.
  • Compare the posteriors of \(K\) and \(K_+\) on simulated data with known structure.
  • Study prior sensitivity across Poisson and beta-negative-binomial priors on \(K\).
  • Compute a posterior similarity matrix and summarise the partition.

Common pitfalls

  • Reporting \(K\) when the clustering question calls for \(K_+\).
  • Cancelling \(\Gamma(\gamma_K)^{-K_+}\) as a constant under the dynamic specification, where it is not.
  • Treating the prior on \(K\) as innocuous when the data are weakly informative about it.
  • Averaging component-specific parameters across draws.

Exercises

  • Derive the partition marginal and verify that empty components contribute a factor of one.
  • Compute \(p(K \mid \mathcal{C})\) by hand for a small partition under both specifications.
  • Adapt the sampler to Poisson components and confirm that only the allocation and parameter updates change.

Core references

  • Frühwirth-Schnatter et al. (2021)
  • Frühwirth-Schnatter et al. (2024)
  • Miller and Harrison (2018)
  • Richardson and Green (1997)

Full chapter

Back to top

References

Andrieu, Christophe, Nando de Freitas, Arnaud Doucet, and Michael I. Jordan. 2003. “An Introduction to MCMC for Machine Learning.” Machine Learning 50 (1–2): 5–43.
Betancourt, Michael. 2017. “A Conceptual Introduction to Hamiltonian Monte Carlo.” arXiv Preprint arXiv:1701.02434.
Blei, David M., Alp Kucukelbir, and Jon D. McAuliffe. 2017. “Variational Inference: A Review for Statisticians.” Journal of the American Statistical Association 112 (518): 859–77.
Brooks, Steve, Andrew Gelman, Galin Jones, and Xiao-Li Meng. 2011. Handbook of Markov Chain Monte Carlo. CRC Press.
Casella, George, and Roger L. Berger. 2002. Statistical Inference. 2nd ed. Duxbury.
Del Moral, Pierre. 2004. Feynman–Kac Formulae: Genealogical and Interacting Particle Systems with Applications. Springer.
Doucet, Arnaud, Nando de Freitas, and Neil Gordon. 2001. Sequential Monte Carlo Methods in Practice. Springer.
Frühwirth-Schnatter, Sylvia, Bettina Grün, and Gertraud Malsiner-Walli. 2024. “Sampling Methods for Bayesian Finite Mixtures with an Unknown Number of Components.” Annual Review of Statistics and Its Application 11: 335–60.
Frühwirth-Schnatter, Sylvia, Gertraud Malsiner-Walli, and Bettina Grün. 2021. “Generalized Mixtures of Finite Mixtures and Telescoping Sampling.” Bayesian Analysis 16 (4): 1279–307.
Gelman, Andrew, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, and Donald B. Rubin. 2013. Bayesian Data Analysis. 3rd ed. CRC Press.
Geman, Stuart, and Donald Geman. 1984. “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images.” IEEE Transactions on Pattern Analysis and Machine Intelligence 6 (6): 721–41.
Hastings, W. K. 1970. “Monte Carlo Sampling Methods Using Markov Chains and Their Applications.” Biometrika 57 (1): 97–109.
Hoffman, Matthew D., and Andrew Gelman. 2014. “The No-u-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo.” Journal of Machine Learning Research 15 (47): 1593–623.
Kucukelbir, Alp, Dustin Tran, Rajesh Ranganath, Andrew Gelman, and David M. Blei. 2017. “Automatic Differentiation Variational Inference.” Journal of Machine Learning Research 18 (14): 1–45.
MacKay, David J. C. 2003. Information Theory, Inference, and Learning Algorithms. Cambridge University Press.
Metropolis, Nicholas, Arianna W. Rosenbluth, Marshall N. Rosenbluth, Augusta H. Teller, and Edward Teller. 1953. “Equation of State Calculations by Fast Computing Machines.” Journal of Chemical Physics 21 (6): 1087–92.
Miller, Jeffrey W., and Matthew T. Harrison. 2018. “Mixture Models with a Prior on the Number of Components.” Journal of the American Statistical Association 113 (521): 340–56.
Neal, Radford M. 2011. “MCMC Using Hamiltonian Dynamics.” Handbook of Markov Chain Monte Carlo, 113–62.
Richardson, Sylvia, and Peter J. Green. 1997. “On Bayesian Analysis of Mixtures with an Unknown Number of Components.” Journal of the Royal Statistical Society: Series B 59 (4): 731–92.
Robert, Christian P., and George Casella. 2004. Monte Carlo Statistical Methods. 2nd ed. Springer.