Bayesian Model Selection

This page contains resources about Bayesian Model Selection and Bayesian Model Comparison.

Subfields and Concepts

 * Bayes Factor
 * Marginal likelihood / (Bayesian Model) evidence
 * Bayesian Occam's Razor
 * Minimum Message Length (MML)
 * Bayesian Model Averaging (in Ensemble Learning)
 * Reversible jump MCMC
 * Birth and death MCMC
 * Bayesian Model Combination
 * Independent Bayesian Classifier Combination (IBCC)
 * Type II Maximum likelihood / Empirical Bayes / Evidence Approximation / Maximum marginal likelihood estimator (MMLE)
 * Type II Maximum a posteriori (MAP)
 * Approximate Bayesian Computation (ABC)
 * Akaike Information Criterion (AIC)
 * Laplace Approximation
 * Bayesian Information Criterion (BIC)
 * Deviance Information Criterion (DIC)
 * Bayesian Predictive Information Criterion (BPIC)
 * Focused Information Criterion (FIC)
 * Cheeseman-Stutz Criterion

Books and Book Chapters

 * Theodoridis, S. (2015). "Chapter 12: Bayesian Learning" Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
 * Barber, D. (2012). "Chapter 12: Bayesian Model Selection". Bayesian Reasoning and Machine Learning. Cambridge University Press.
 * Murphy, K. P. (2012). "Section 5.3: Bayesian model selection". Machine Learning: A Probabilistic Perspective. MIT Press.
 * Claeskens, G., & Hjort, N. L. (2008). Model selection and model averaging. Cambridge: Cambridge University Press.
 * Bishop, C. M. (2006). "Section 3.4: Bayesian Model Comparison". Pattern Recognition and Machine Learning. Springer.
 * Williams, C. K., & Rasmussen, C. E. (2006). "Chapter 5: Model Selection and Adaptation of Hyperparameters". Gaussian Processes for Machine Learning. MIT Press.
 * MacKay, D. J. (2003). "Chapter 28: Model Comparison and Occam’s Razor". Information Theory, Inference and Learning Algorithms. Cambridge University Press.

Scholarly Articles

 * Wang, H., & Yeung, D. Y. (2016). Towards Bayesian deep learning: A survey. arXiv preprint arXiv:1604.01662.
 * Kim, H. C., & Ghahramani, Z. (2012). Bayesian classifier combination. In Artificial Intelligence and Statistics (pp. 619-627).
 * Alston, C., Kuhnert, P., Choy, S. L., McVinish, R., & Mengersen, K. (2005). Bayesian model comparison: Review and discussion. International Statistical Insitute, 55th session.
 * Wasserman, L. (2000). Bayesian model selection and model averaging. Journal of mathematical psychology, 44(1), 92-107.
 * Kass, R. E., & Raftery, A. E. (1993), Bayes factors and model uncertainty, Technical Report 254, Department of Statistics, University of Washington

Tuturials

 * Bayesian Model Selection by Tom Minka

Other resources

 * Bayesian model comparison - Metacademy
 * Evidence approximation - Metacademy