Bayesian Model Selection

From Ioannis Kourouklides

This page contains resources about Bayesian Model Selection and Bayesian Model Comparison.

Subfields and Concepts[edit]

  • Bayes Factor
  • Marginal likelihood / (Bayesian Model) evidence
  • Bayesian Occam's Razor
  • Minimum Message Length (MML)
  • Bayesian Model Averaging (in Ensemble Learning)
    • Reversible jump MCMC
    • Birth and death MCMC
    • Bayesian Model Combination
  • Independent Bayesian Classifier Combination (IBCC)
  • Type II Maximum likelihood / Empirical Bayes / Evidence Approximation / Maximum marginal likelihood estimator (MMLE)
  • Type II Maximum a posteriori (MAP)
  • Approximate Bayesian Computation (ABC)
  • Akaike Information Criterion (AIC)
  • Laplace Approximation
  • Bayesian Information Criterion (BIC)
  • Deviance Information Criterion (DIC)
  • Bayesian Predictive Information Criterion (BPIC)
  • Focused Information Criterion (FIC)
  • Cheeseman-Stutz Criterion

Books and Book Chapters[edit]

  • Theodoridis, S. (2015). "Chapter 12: Bayesian Learning" Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
  • Barber, D. (2012). "Chapter 12: Bayesian Model Selection". Bayesian Reasoning and Machine Learning. Cambridge University Press.
  • Murphy, K. P. (2012). "Section 5.3: Bayesian model selection". Machine Learning: A Probabilistic Perspective. MIT Press.
  • Claeskens, G., & Hjort, N. L. (2008). Model selection and model averaging. Cambridge: Cambridge University Press.
  • Bishop, C. M. (2006). "Section 3.4: Bayesian Model Comparison". Pattern Recognition and Machine Learning. Springer.
  • Williams, C. K., & Rasmussen, C. E. (2006). "Chapter 5: Model Selection and Adaptation of Hyperparameters". Gaussian Processes for Machine Learning. MIT Press.
  • MacKay, D. J. (2003). "Chapter 28: Model Comparison and Occam’s Razor". Information Theory, Inference and Learning Algorithms. Cambridge University Press.

Scholarly Articles[edit]

  • Wang, H., & Yeung, D. Y. (2016). Towards Bayesian deep learning: A survey. arXiv preprint arXiv:1604.01662.
  • Kim, H. C., & Ghahramani, Z. (2012). Bayesian classifier combination. In Artificial Intelligence and Statistics (pp. 619-627).
  • Alston, C., Kuhnert, P., Choy, S. L., McVinish, R., & Mengersen, K. (2005). Bayesian model comparison: Review and discussion. International Statistical Insitute, 55th session.
  • Wasserman, L. (2000). Bayesian model selection and model averaging. Journal of mathematical psychology44(1), 92-107.
  • Kass, R. E., & Raftery, A. E. (1993), Bayes factors and model uncertainty, Technical Report 254, Department of Statistics, University of Washington


See also[edit]

Other resources[edit]