Ensemble Learning

This page contains resources about Ensemble Learning, including Committee Machines.

Subfields and Concepts

 * Voting
 * Ensemble Averaging
 * Bayesian Averaging
 * Reversible jump MCMC
 * Birth and death MCMC
 * Bayesian Model Combination
 * Bagging
 * Boosting
 * Gradient boosting
 * Bayes Optimal Classifier
 * Decision Forests / Random Forests
 * Stacking
 * Committee Machines
 * Mixture Models
 * Mixture of Experts
 * Hierarchical Mixture of Experts
 * k-Nearest Neighbour (k-NN)
 * Mixture of Factor Analyzers
 * Gaussian Mixture Model (GMM)
 * Snapshot Ensembling (in Artificial Neural Networks)

Books and Book Chapters

 * Zhou, Z. H. (2015). "Ensemble Learning". Encyclopedia of biometrics. Springer.
 * Zhou, Z. H. (2012). Ensemble methods: foundations and algorithms. CRC press.
 * Alpaydin, E. (2010). "Chapter 17: Combining Multiple Learners". Introduction to machine learning. MIT Press.
 * Russell, S. J., & Norvig, P. (2010). "Section 18.10: Ensemble Learning". Artificial Intelligence: A Modern Approach. Prentice Hall.
 * Bishop, C. M. (2006). "Chapter 9: Mixture Models and EM". Pattern Recognition and Machine Learning. Springer.
 * Bishop, C. M. (2006). "Chapter 14: Combining Models". Pattern Recognition and Machine Learning. Springer.
 * Kuncheva, L. I. (2004). Combining pattern classifiers: methods and algorithms. John Wiley & Sons.
 * Dietterich, T. G. (2002). "Ensemble Learning". The handbook of brain theory and neural networks. MIT Press.

Scholarly Articles

 * Dzeroski, S., & Zenko, B. (2004). Is combining classifiers with stacking better than selecting the best one?. Machine learning, 54(3), 255-273.

Software

 * Classification Ensembles (Statistics and Machine Learning Toolbox) - MATLAB
 * Regression Tree Ensembles (Statistics and Machine Learning) - MATLAB
 * Ensemble Methods (scikit-learn) - Python
 * BMS - R
 * BAS - R
 * BMA - R

Other Resources

 * Ensemble Methods in Machine Learning - Notebook
 * Ensemble Learning to Improve Machine Learning Results - blog post
 * Kaggle Ensembling Guide - MLWave
 * Kaggle Winning Ensemble - blog post
 * Ensemble Learning to Improve Machine Learning Results - blog post
 * what is the difference between the stacking grading, and voting algorithms? - Stack Overflow
 * Ensemble Methods: Elegant Techniques to Produce Improved Machine Learning Results - blog post