References and citation strategy

Citation strategy

The course cites sources for four distinct purposes:

Foundations for probability, Bayesian theory, and decision theory.
Methodological landmarks for Monte Carlo, MCMC, HMC, variational inference, and Bayesian workflow.
Modeling references for hierarchical models, Gaussian processes, nonparametrics, and predictive evaluation.
Software references for R, Stan, cmdstanr, posterior, bayesplot, loo, and JAX.

Linked reference guide

The bibliography below remains the formal citation record. This section is the practical reading guide: each item links to a publisher page, official documentation page, DOI landing page, or open-access version where possible.

Annotated reading guide by course part

Part I. Foundations

Foundations of the Theory of Probability
Formal probability foundations and the language behind measurable probability models.
Probability: Theory and Examples
A strong reference for conditioning, convergence, and rigorous probability arguments.
Bayesian Theory
Foundational Bayesian reasoning, decision structure, and posterior logic.

Part II. Core Bayesian inference

Bayesian Data Analysis
The main all-purpose reference for posterior inference, priors, prediction, and applied Bayesian workflow.
Statistical Decision Theory and Bayesian Analysis
Canonical text for loss, utility, Bayes rules, and decision-theoretic structure.
Prior Distributions for Variance Parameters in Hierarchical Models
Influential discussion of weakly informative prior design.

Part III. Computation

Monte Carlo Statistical Methods
Core reference for Monte Carlo, importance sampling, MCMC, and asymptotic simulation logic.
The No-U-Turn Sampler
Canonical NUTS paper, central for modern Stan computation.
A Conceptual Introduction to Hamiltonian Monte Carlo
Best geometry-first explanation of HMC pathologies and behavior.
Variational Inference: A Review for Statisticians
Clear overview of ELBOs, KL geometry, and variational approximations.

Part IV. Models

Gaussian Processes for Machine Learning
Standard reference for kernel-based Bayesian function modeling.
A Bayesian Analysis of Some Nonparametric Problems
Foundational Dirichlet process paper.
Probabilistic Machine Learning: Advanced Topics
Broad modern reference connecting hierarchical models, latent variables, graphical models, and advanced Bayesian machine learning.

Part V. Optimization, tuning, and diagnostics

Practical Bayesian Model Evaluation Using Leave-One-Out Cross-Validation and WAIC
Standard predictive model evaluation reference.
Rank-Normalization, Folding, and Localization: An Improved R-hat
Modern convergence diagnostic reference.
Visualization in Bayesian Workflow
Strong workflow paper connecting computation diagnostics and model criticism.

Part VI. Software workflows

R Project
Core language environment for the R side of the course.
Stan
Official Stan ecosystem site.
CmdStanR
Official R interface used in the course.
posterior
Posterior-draw manipulation and summary tools.
bayesplot
Diagnostic and posterior visualization package.
loo
Official package for PSIS-LOO and related diagnostics.
JAX
Autodiff and array programming system used for custom Bayesian computation.
NumPyro
Useful adjacent probabilistic programming system in the JAX ecosystem.
Quarto
Publishing system used to build this course.

Part VII. Advanced topics

The Bayesian Choice
Strong reference for Bayesian evidence, Bayes factors, and model comparison logic.
Using Stacking to Average Bayesian Predictive Distributions
Canonical stacking reference.
Projective Inference in High-Dimensional Problems
Projection predictive model selection reference.
Bayesian Experimental Design: A Review
Classic overview of Bayesian design criteria and methods.

Core texts

Canonical textbooks

The most important book-length references for the course are:

Bernardo and Smith (1994)
Berger (1985)
Gelman et al. (2013)
Hoff (2009)
Robert (2007)
Robert and Casella (2004)
Brooks et al. (2011)
McElreath (2020)
MacKay (2003)
Murphy (2023)

Landmark papers and articles

Core methodological papers repeatedly used in the course include:

Metropolis et al. (1953)
Hastings (1970)
Geman and Geman (1984)
Andrieu et al. (2003)
Neal (2011)
Hoffman and Gelman (2014)
Betancourt (2017)
Kucukelbir et al. (2017)
Blei et al. (2017)
Vehtari et al. (2017)
Vehtari et al. (2021)
Gabry et al. (2019)
Ferguson (1973)
Neal (2000)

Software references

When the course discusses tools or packages, the preferred citations are:

R Core Team (2025)
Carpenter et al. (2017)
Gabry et al. (2024)
Bürkner et al. (2024)
Gabry and Mahr (2024)
Vehtari et al. (2024)
Bradbury et al. (2018)
Phan et al. (2020)

Full bibliography

Andrieu, Christophe, Nando de Freitas, Arnaud Doucet, and Michael I. Jordan. 2003. “An Introduction to MCMC for Machine Learning.” Machine Learning 50 (1–2): 5–43.

Antoniak, Charles E. 1974. “Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems.” Annals of Statistics 2 (6): 1152–74.

Berger, James O. 1985. Statistical Decision Theory and Bayesian Analysis. 2nd ed. Springer.

Bernardo, José M., and Adrian F. M. Smith. 1994. Bayesian Theory. Wiley.

Betancourt, Michael. 2017. “A Conceptual Introduction to Hamiltonian Monte Carlo.” arXiv Preprint arXiv:1701.02434.

Betancourt, Michael, and Mark Girolami. 2015. “Hamiltonian Monte Carlo for Hierarchical Models.” Current Trends in Bayesian Methodology with Applications, 79–101.

Bishop, Christopher M. 2006. Pattern Recognition and Machine Learning. Springer.

Bissiri, Pier Giovanni, Chris Holmes, and Stephen G. Walker. 2016. “A General Framework for Updating Belief Distributions.” Journal of the Royal Statistical Society: Series B 78 (5): 1103–30.

Blei, David M., Alp Kucukelbir, and Jon D. McAuliffe. 2017. “Variational Inference: A Review for Statisticians.” Journal of the American Statistical Association 112 (518): 859–77.

Bradbury, James, Roy Frostig, Peter Hawkins, et al. 2018. “JAX: Composable Transformations of Python+NumPy Programs.” GitHub Repository.

Brooks, Steve, Andrew Gelman, Galin Jones, and Xiao-Li Meng. 2011. Handbook of Markov Chain Monte Carlo. CRC Press.

Bürkner, Paul-Christian, Jonah Gabry, Aki Vehtari, et al. 2024. Posterior: Tools for Working with Posterior Distributions.

Carpenter, Bob, Andrew Gelman, Matthew D. Hoffman, et al. 2017. “Stan: A Probabilistic Programming Language.” Journal of Statistical Software 76 (1): 1–32.

Casella, George, and Roger L. Berger. 2002. Statistical Inference. 2nd ed. Duxbury.

Chaloner, Kathryn, and Isabella Verdinelli. 1995. “Bayesian Experimental Design: A Review.” Statistical Science 10 (3): 273–304.

David, Claire. 2023. Introduction to Machine Learning. Online course notes.

Del Moral, Pierre. 2004. Feynman–Kac Formulae: Genealogical and Interacting Particle Systems with Applications. Springer.

Doucet, Arnaud, Nando de Freitas, and Neil Gordon. 2001. Sequential Monte Carlo Methods in Practice. Springer.

Durrett, Rick. 2019. Probability: Theory and Examples. 5th ed. Cambridge University Press.

Efron, Bradley, and Carl Morris. 1975. “Data Analysis Using Stein’s Estimator and Its Generalizations.” Journal of the American Statistical Association 70 (350): 311–19.

Ferguson, Thomas S. 1973. “A Bayesian Analysis of Some Nonparametric Problems.” The Annals of Statistics 1 (2): 209–30.

Finetti, Bruno de. 1974. “Theory of Probability: A Critical Introductory Treatment.” Wiley Series in Probability and Mathematical Statistics 1.

Frühwirth-Schnatter, Sylvia. 2006. Finite Mixture and Markov Switching Models. Springer.

Frühwirth-Schnatter, Sylvia, Bettina Grün, and Gertraud Malsiner-Walli. 2024. “Sampling Methods for Bayesian Finite Mixtures with an Unknown Number of Components.” Annual Review of Statistics and Its Application 11: 335–60.

Frühwirth-Schnatter, Sylvia, Gertraud Malsiner-Walli, and Bettina Grün. 2021. “Generalized Mixtures of Finite Mixtures and Telescoping Sampling.” Bayesian Analysis 16 (4): 1279–307.

Gabry, Jonah, Rok Cešnovar, Andrew Johnson, Sean Bronder, Ben Goodrich, et al. 2024. Cmdstanr: Interface to ’CmdStan’.

Gabry, Jonah, and Tristan Mahr. 2024. Bayesplot: Plotting for Bayesian Models.

Gabry, Jonah, Daniel Simpson, Aki Vehtari, Michael Betancourt, and Andrew Gelman. 2019. “Visualization in Bayesian Workflow.” Journal of the Royal Statistical Society: Series A 182 (2): 389–402.

Geisser, Seymour. 1993. “Predictive Inference.” Monographs on Statistics and Applied Probability.

Gelman, Andrew. 2006. “Prior Distributions for Variance Parameters in Hierarchical Models.” Bayesian Analysis 1 (3): 515–34.

Gelman, Andrew, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, and Donald B. Rubin. 2013. Bayesian Data Analysis. 3rd ed. CRC Press.

Gelman, Andrew, and Jennifer Hill. 2007. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press.

Gelman, Andrew, and Donald B. Rubin. 1992. “Inference from Iterative Simulation Using Multiple Sequences.” Statistical Science 7 (4): 457–72.

Gelman, Andrew, Aki Vehtari, Daniel Simpson, et al. 2020. “Bayesian Workflow.” arXiv Preprint arXiv:2011.01808.

Geman, Stuart, and Donald Geman. 1984. “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images.” IEEE Transactions on Pattern Analysis and Machine Intelligence 6 (6): 721–41.

Green, Peter J. 1995. “Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination.” Biometrika 82 (4): 711–32.

Hastings, W. K. 1970. “Monte Carlo Sampling Methods Using Markov Chains and Their Applications.” Biometrika 57 (1): 97–109.

Hoff, Peter D. 2009. A First Course in Bayesian Statistical Methods. Springer.

Hoffman, Matthew D., and Andrew Gelman. 2014. “The No-u-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo.” Journal of Machine Learning Research 15 (47): 1593–623.

Jain, Sonia, and Radford M. Neal. 2004. “A Split-Merge Markov Chain Monte Carlo Procedure for the Dirichlet Process Mixture Model.” Journal of Computational and Graphical Statistics 13 (1): 158–82.

Jaynes, E. T. 2003. Probability Theory: The Logic of Science. Cambridge University Press.

Jeffreys, Harold. 1961. Theory of Probability. 3rd ed. Oxford University Press.

Kass, Robert E., and Adrian E. Raftery. 1995. “Bayes Factors.” Journal of the American Statistical Association 90 (430): 773–95.

Kleijn, Bas J. K., and Aad W. van der Vaart. 2012. “The Bernstein–von Mises Theorem Under Misspecification.” Electronic Journal of Statistics 6: 354–81.

Kolmogorov, A. N. 1956. Foundations of the Theory of Probability. Chelsea Publishing Company.

Kosmidis, Ioannis. 2024. APTS Statistical Modelling: Lecture Notes. Academy for PhD Training in Statistics.

Kucukelbir, Alp, Dustin Tran, Rajesh Ranganath, Andrew Gelman, and David M. Blei. 2017. “Automatic Differentiation Variational Inference.” Journal of Machine Learning Research 18 (14): 1–45.

Lange, Kenneth L., Roderick J. A. Little, and Jeremy M. G. Taylor. 1989. “Robust Statistical Modeling Using the \(t\) Distribution.” Journal of the American Statistical Association 84 (408): 881–96.

Lindley, Dennis V., and Adrian F. M. Smith. 1972. “Bayes Estimates for the Linear Model.” Journal of the Royal Statistical Society: Series B 34 (1): 1–41.

MacKay, David J. C. 2003. Information Theory, Inference, and Learning Algorithms. Cambridge University Press.

Malsiner-Walli, Gertraud, Sylvia Frühwirth-Schnatter, and Bettina Grün. 2016. “Model-Based Clustering Based on Sparse Finite Gaussian Mixtures.” Statistics and Computing 26 (1): 303–24.

McElreath, Richard. 2020. Statistical Rethinking. 2nd ed. CRC Press.

Meng, Xiao-Li, and Wing Hung Wong. 1996. “Simulating Ratios of Normalizing Constants via a Simple Identity: A Theoretical Exploration.” Statistica Sinica 6 (4): 831–60.

Metropolis, Nicholas, Arianna W. Rosenbluth, Marshall N. Rosenbluth, Augusta H. Teller, and Edward Teller. 1953. “Equation of State Calculations by Fast Computing Machines.” Journal of Chemical Physics 21 (6): 1087–92.

Miller, Jeffrey W., and Matthew T. Harrison. 2018. “Mixture Models with a Prior on the Number of Components.” Journal of the American Statistical Association 113 (521): 340–56.

Murphy, Kevin P. 2023. Probabilistic Machine Learning: Advanced Topics. MIT Press.

Neal, Radford M. 2000. “Markov Chain Sampling Methods for Dirichlet Process Mixture Models.” Journal of Computational and Graphical Statistics 9 (2): 249–65.

Neal, Radford M. 2011. “MCMC Using Hamiltonian Dynamics.” Handbook of Markov Chain Monte Carlo, 113–62.

Papaspiliopoulos, Omiros, Gareth O. Roberts, and Martin Sköld. 2007. “A General Framework for the Parametrization of Hierarchical Models.” Statistical Science 22 (1): 59–73.

Phan, Du, Neeraj Pradhan, and Martin Jankowiak. 2020. “Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro.” arXiv Preprint arXiv:1912.11554.

Piironen, Juho, Markus Paasiniemi, and Aki Vehtari. 2020. “Projective Inference in High-Dimensional Problems: Prediction and Feature Selection.” Electronic Journal of Statistics 14 (1): 2155–97.

Piironen, Juho, and Aki Vehtari. 2017. “Sparsity Information and Regularization in the Horseshoe and Other Shrinkage Priors.” Electronic Journal of Statistics 11 (2): 5018–51.

R Core Team. 2025. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing.

Rasmussen, Carl Edward, and Christopher K. I. Williams. 2006. Gaussian Processes for Machine Learning. MIT Press.

Richardson, Sylvia, and Peter J. Green. 1997. “On Bayesian Analysis of Mixtures with an Unknown Number of Components.” Journal of the Royal Statistical Society: Series B 59 (4): 731–92.

Robert, Christian P. 2007. The Bayesian Choice. 2nd ed. Springer.

Robert, Christian P., and George Casella. 2004. Monte Carlo Statistical Methods. 2nd ed. Springer.

Roberts, Gareth O., Andrew Gelman, and Walter R. Gilks. 1997. “Weak Convergence and Optimal Scaling of Random Walk Metropolis Algorithms.” Annals of Applied Probability 7 (1): 110–20.

Rousseau, Judith, and Kerrie Mengersen. 2011. “Asymptotic Behaviour of the Posterior Distribution in Overfitted Mixture Models.” Journal of the Royal Statistical Society: Series B 73 (5): 689–710.

Rubin, Donald B. 1984. “Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician.” The Annals of Statistics 12 (4): 1151–72.

Sethuraman, Jayaram. 1994. “A Constructive Definition of Dirichlet Priors.” Statistica Sinica 4 (2): 639–50.

Simpson, Daniel, Håvard Rue, Andrea Riebler, Thiago G. Martins, and Sigrunn H. Sørbye. 2017. “Penalising Model Component Complexity: A Principled, Practical Approach to Constructing Priors.” Statistical Science 32 (1): 1–28.

Stein, Charles. 1956. “Inadmissibility of the Usual Estimator for the Mean of a Multivariate Normal Distribution.” In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, vol. 1. University of California Press.

Stephens, Matthew. 2000. “Dealing with Label Switching in Mixture Models.” Journal of the Royal Statistical Society: Series B 62 (4): 795–809.

Talts, Sean, Michael Betancourt, Daniel Simpson, Aki Vehtari, and Andrew Gelman. 2018. “Validating Bayesian Inference Algorithms with Simulation-Based Calibration.” arXiv Preprint arXiv:1804.06788.

Vaart, A. W. van der. 1998. Asymptotic Statistics. Cambridge University Press.

Vehtari, Aki, Jonah Gabry, Yuling Yao, and Andrew Gelman. 2024. Loo: Efficient Leave-One-Out Cross-Validation and WAIC for Bayesian Models.

Vehtari, Aki, Andrew Gelman, and Jonah Gabry. 2017. “Practical Bayesian Model Evaluation Using Leave-One-Out Cross-Validation and WAIC.” Statistics and Computing 27 (5): 1413–32.

Vehtari, Aki, Andrew Gelman, Daniel Simpson, Bob Carpenter, and Paul-Christian Bürkner. 2021. “Rank-Normalization, Folding, and Localization: An Improved \(\hat{R}\) for Assessing Convergence of MCMC.” Bayesian Analysis 16 (2): 667–718.

Wade, Sara, and Zoubin Ghahramani. 2018. “Bayesian Cluster Analysis: Point Estimation and Credible Balls.” Bayesian Analysis 13 (2): 559–626.

Yao, Yuling, Aki Vehtari, Daniel Simpson, and Andrew Gelman. 2018. “Using Stacking to Average Bayesian Predictive Distributions.” Bayesian Analysis 13 (3): 917–1007.