Detailed syllabus and recommended pacing

Course modes

This curriculum supports three teaching modes:

  • a 14-week semester course;
  • a 28-week self-study sequence;
  • a modular graduate seminar or reading course.

The chapter dependencies are strong enough that the intended order matters. Probability and likelihood precede priors and posteriors; priors and posteriors precede computational methods; computational methods precede realistic modeling and diagnostics.

Semester syllabus

Week Topics Core mathematical focus Computational focus Suggested deliverable
1 Probability spaces, random variables, conditioning Conditional expectation, tower property Simulation in R Short derivation set
2 Likelihood theory and asymptotics Score, Fisher information, MLE Likelihood surfaces in R Problem set 1
3 Bayes’ theorem, exchangeability, updating Posterior factorization, predictive distribution Conjugate computation in R Problem set 2
4 Conjugate models and marginal likelihoods Beta-Binomial, Gamma-Poisson, Normal-Normal Posterior predictive simulation Lab 1
5 Decision theory and prior design Posterior risk, Bayes estimators, Jeffreys priors Prior predictive checking Problem set 3
6 Numerical integration and Monte Carlo Laplace, importance sampling Variance reduction in R Lab 2
7 MCMC foundations Detailed balance, ergodicity, MH ratio Implement MH and Gibbs Problem set 4
8 HMC, NUTS, reparameterization Hamiltonian dynamics, leapfrog Stan diagnostics Lab 3
9 Variational inference and SMC ELBO, particle approximations ADVI and sequential update demos Short memo
10 Bayesian linear and generalized linear models Posterior regression structure cmdstanr regression workflow Problem set 5
11 Hierarchical models and mixtures Partial pooling, label switching Non-centered parameterization Lab 4
12 Time series, nonparametrics, spatial and causal models Filtering, GP covariance, DP intuition Specialized model workflow Reading response
13 Optimization, tuning, and diagnostics MAP, SBC, PPC, LOO R-hat, ESS, divergences, E-BFMI Problem set 6
14 R, Stan, JAX workflow integration; advanced topics and projects Model averaging, robustness, communication Reproducible final pipeline Final project

Self-study sequence

The self-study track is slower and more deliberate. A reasonable pacing plan is:

Block Weeks Chapters Goal
A 1-5 1-3 Solidify probability, likelihood, and posterior logic
B 6-10 4-6 Become fluent in conjugacy, loss, and priors
C 11-18 7-12 Build computational maturity
D 19-24 13-20 Work through model families
E 25-28 21-31 Diagnostics, workflow, advanced topics, project

For self-study, the minimum viable sequence is:

  1. Read the theory chapter or part overview.
  2. Reproduce one worked derivation on paper.
  3. Run or retype one code example.
  4. Complete two exercises: one analytic and one computational.
  5. Write a one-paragraph reflection on diagnostics or modeling assumptions.

Learning outcomes by assessment type

Derivation sets

Students should prove, derive, or verify:

  • conjugate posterior forms;
  • marginal likelihood calculations;
  • the Metropolis-Hastings acceptance ratio;
  • HMC equations and the leapfrog map;
  • the ELBO decomposition;
  • Kalman filtering recursions in linear Gaussian state-space models.

Computational labs

Students should implement:

  • conjugate updating and predictive simulation;
  • importance sampling and ESS calculations;
  • random-walk Metropolis and Gibbs samplers;
  • Stan models with posterior predictive checks;
  • JAX gradient and Hessian calculations for MAP and Laplace approximations.

Project work

Students should demonstrate:

  • principled prior design;
  • justified model structure;
  • convergence diagnostics;
  • posterior and prior predictive checks;
  • sensitivity analysis;
  • clearly written uncertainty communication.

Prerequisites checklist

Use the following checklist before starting the full course.

Skill Required level Where to review if weak
Single-variable calculus Comfortable Appendix and standard calculus text
Multivariable differentiation Comfortable Optimization chapters and JAX chapter
Linear algebra Comfortable Regression and HMC chapters
Probability Intermediate Measure-probability review
Programming Basic R or Python Software setup

Suggested weekly rhythm

For an instructor-led course, each week should ideally include:

  1. One lecture focused on mathematical structure and derivations.
  2. One lecture or lab focused on computation and diagnostics.
  3. A small problem set mixing analytic derivations, simulation, and interpretation.

What to emphasize pedagogically

Across the full syllabus, several misconceptions recur. The course should repeatedly correct them.

  • A posterior distribution is not a likelihood with axes relabeled.
  • A credible interval and a confidence interval solve different inferential problems.
  • Good convergence diagnostics do not certify a good model.
  • Fast optimization is not a substitute for posterior uncertainty quantification.
  • Posterior predictive fit does not imply causal validity.
  • Bayes factors are highly prior-sensitive and should not be used reflexively.

Suggested milestones

By the midpoint of the course, students should already be able to:

  • derive and implement at least three conjugate models;
  • explain and code a random-walk Metropolis sampler;
  • interpret acceptance rates, autocorrelation, ESS, and \(\hat{R}\);
  • run a simple Stan model and diagnose obvious pathologies.

By the end of the course, students should be able to:

  • fit and criticize multilevel and generalized linear Bayesian models;
  • explain why centered and non-centered parameterizations behave differently;
  • use predictive criteria and posterior checks to compare models;
  • integrate R, Stan, and JAX into a coherent reproducible workflow.
Back to top

References

Bernardo, José M., and Adrian F. M. Smith. 1994. Bayesian Theory. Wiley.
Betancourt, Michael. 2017. “A Conceptual Introduction to Hamiltonian Monte Carlo.” arXiv Preprint arXiv:1701.02434.
Blei, David M., Alp Kucukelbir, and Jon D. McAuliffe. 2017. “Variational Inference: A Review for Statisticians.” Journal of the American Statistical Association 112 (518): 859–77.
Brooks, Steve, Andrew Gelman, Galin Jones, and Xiao-Li Meng. 2011. Handbook of Markov Chain Monte Carlo. CRC Press.
Gelman, Andrew, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, and Donald B. Rubin. 2013. Bayesian Data Analysis. 3rd ed. CRC Press.
Hoff, Peter D. 2009. A First Course in Bayesian Statistical Methods. Springer.
Kucukelbir, Alp, Dustin Tran, Rajesh Ranganath, Andrew Gelman, and David M. Blei. 2017. “Automatic Differentiation Variational Inference.” Journal of Machine Learning Research 18 (14): 1–45.
McElreath, Richard. 2020. Statistical Rethinking. 2nd ed. CRC Press.
Murphy, Kevin P. 2023. Probabilistic Machine Learning: Advanced Topics. MIT Press.
Rasmussen, Carl Edward, and Christopher K. I. Williams. 2006. Gaussian Processes for Machine Learning. MIT Press.
Robert, Christian P., and George Casella. 2004. Monte Carlo Statistical Methods. 2nd ed. Springer.