Bayesian Model Selection

This page contains resources about Bayesian Model Selection and Bayesian Model Comparison.

Subfields and Concepts[edit]

Bayes Factor
Marginal likelihood / (Bayesian Model) evidence
Bayesian Occam's Razor
Minimum Message Length (MML)
Bayesian Model Averaging (in Ensemble Learning)
- Reversible jump MCMC
- Birth and death MCMC
- Bayesian Model Combination
Independent Bayesian Classifier Combination (IBCC)
Type II Maximum likelihood / Empirical Bayes / Evidence Approximation / Maximum marginal likelihood estimator (MMLE)
Type II Maximum a posteriori (MAP)
Approximate Bayesian Computation (ABC)
Akaike Information Criterion (AIC)
Laplace Approximation
Bayesian Information Criterion (BIC)
Deviance Information Criterion (DIC)
Bayesian Predictive Information Criterion (BPIC)
Focused Information Criterion (FIC)
Cheeseman-Stutz Criterion

Theodoridis, S. (2015). "Chapter 12: Bayesian Learning" Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
Barber, D. (2012). "Chapter 12: Bayesian Model Selection". Bayesian Reasoning and Machine Learning. Cambridge University Press.
Murphy, K. P. (2012). "Section 5.3: Bayesian model selection". Machine Learning: A Probabilistic Perspective. MIT Press.
Claeskens, G., & Hjort, N. L. (2008). Model selection and model averaging. Cambridge: Cambridge University Press.
Bishop, C. M. (2006). "Section 3.4: Bayesian Model Comparison". Pattern Recognition and Machine Learning. Springer.
Williams, C. K., & Rasmussen, C. E. (2006). "Chapter 5: Model Selection and Adaptation of Hyperparameters". Gaussian Processes for Machine Learning. MIT Press.
MacKay, D. J. (2003). "Chapter 28: Model Comparison and Occam’s Razor". Information Theory, Inference and Learning Algorithms. Cambridge University Press.

Wang, H., & Yeung, D. Y. (2016). Towards Bayesian deep learning: A survey. arXiv preprint arXiv:1604.01662.
Kim, H. C., & Ghahramani, Z. (2012). Bayesian classifier combination. In Artificial Intelligence and Statistics (pp. 619-627).
Alston, C., Kuhnert, P., Choy, S. L., McVinish, R., & Mengersen, K. (2005). Bayesian model comparison: Review and discussion. International Statistical Insitute, 55th session.
Wasserman, L. (2000). Bayesian model selection and model averaging. Journal of mathematical psychology, 44(1), 92-107.
Kass, R. E., & Raftery, A. E. (1993), Bayes factors and model uncertainty, Technical Report 254, Department of Statistics, University of Washington