Bayesian Nonparametrics

This page contains resources about Bayesian Nonparametrics and Bayesian Nonparametric Models.

A distinction should be made between Models and Methods (which might be applied on or using these Models).

Subfields
See Category:Bayesian Nonparametrics for some of its subfields.
 * Nonparametric Bayesian Models
 * Gamma Process Nonnegative Matrix Factorization (GaP-NMF) Model
 * Gaussian Process
 * Dirichlet Process
 * Chinese Restaurant Process (CRP)
 * Chinese Restaurant Franchise (CRF)
 * Indian Buffet Process (IBP)
 * Pitman–Yor Process
 * Hierarchical Dirichlet Process (HDP)
 * Mixture of Dirichlet Processes (MDP)
 * Dirichlet Processes Mixture Model (DPMM)
 * CRP Mixture Model
 * IBP Latent Factor Model
 * Latent Dirichlet Allocation (LDA)
 * Levy Process
 * Bernoulli Process
 * Completely Random Measures
 * Poisson Random Measure / Poisson Point Process
 * Gamma Process
 * Beta Process / Beta-Bernoulli Process
 * Stable Process
 * Polya Trees
 * Polya's Urn Process
 * Hoppe's Urn Process
 * Stick Breaking Process
 * Nonparametric Bayesian Methods
 * Consistency
 * Contraction rates

Video Lectures

 * Bayesian Nonparametrics by Yee Whye Teh - VideoLectures.Net
 * Nonparametric Bayesian Models by Yee Whye Teh - VideoLectures.Net
 * Dirichlet Processes, Chinese Restaurant Processes, and all that - VideoLectures.Net
 * Foundations of Nonparametric Bayesian Models by Peter Orbanz - VideoLectures.Net

Lecture Notes

 * Bayesian Nonparametrics by Peter Orbanz
 * Bayesian Nonparametrics by David M. Blei
 * Advanced Methods in Probabilistic Modeling BY David M. Blei
 * Topics in Probability: Lévy Processes by Davar Khoshnevisan
 * Nonparametric Bayesian Methods (Dirichlet Process Mixtures) by Jun Zhu
 * Nonparametric Bayesian methods (Dirichlet processes) by Kurt Miller
 * CRiSM Master Class Non-Parametric Bayes by David Dunson and Harry van Zanten

Books and Book Chapters

 * Küchler, U., & Sorensen, M. (1997). Exponential families of stochastic processes. Springer Science & Business Media.
 * Dey, D. D., MuIler, P., & Sinha, D. (Eds.). (1998). Practical nonparametric and semiparametric Bayesian statistics (Vol. 133). Springer Science & Business Media.
 * Ghosh, J. K., & Ramamoorthi, R. V. (2003). Bayesian Nonparametrics.Springer Series in Statistics. Springer-Verlag, New York, 16, 37.
 * Gorur, D. (2007). Nonparametric Bayesian Discrete Latent Variable Models for Unsupervised Learning. PhD Diss. Technical University of Berlin.
 * Koller, D., & Friedman, N. (2009). "Section 19.5: Learning Models with Hidden Variables ". Probabilistic Graphical Models. MIT Press.
 * Hjort, N. L., Holmes, C., Muller, P., & Walker, S. G. (Eds.). (2010). Bayesian Nonparametrics. Cambridge University Press.
 * Orbanz, P., & Teh, Y. W. (2011). Bayesian Nonparametric Models. In Encyclopedia of Machine Learning (pp. 81-89). Springer US.
 * Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. Chapter 25: Clustering. MIT Press.
 * Jordan, M. I. (2013). Hierarchical models, nested models and completely random measures. Frontiers of Statistical Decision Making and Bayesian Analysis: in Honor of James O. Berger. New York: Springer, 207-218.
 * Theodoridis, S. (2015). "Section 13.12: Nonparametric Bayesian Modeling". Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
 * Muller, P., Quintana, F. A., Jara, A., & Hanson, T. (2015). Nonparametric Bayesian Data Analysis. New York: Springer.
 * Phadia, E. G. (2015). Prior Processes and Their Applications: Nonparametric Bayesian Estimation. Springer.
 * Mitra, R., & Muller, P. (Eds.). (2015). Nonparametric Bayesian Inference in Biostatistics. Springer.
 * Goodman, N. D., & Tenenbaum, J. B. (2016). "Chapter 12: Non-parametric models".  Probabilistic Models of Cognition. 2nd Ed. (link)
 * Ghosal, S., & Van der Vaart, A. (2017). Fundamentals of nonparametric Bayesian inference. Cambridge University Press.

Scholarly Articles
See also NPBayes 2008 for more references.
 * Damien, P. (2005). Some Bayesian Nonparametric Models. Handbook of statistics, 25, 279-314.
 * Hanson, T. E., Branscum, A. J., & Johnson, W. O. (2005). Bayesian nonparametric modeling and data analysis: an introduction. Handbook of statistics, 25, 245-278.
 * Walker, S. (2005). Bayesian Nonparametric Inference. Handbook of statistics, 25, 339-371.
 * Sudderth, E.B. (2006). Graphical Models for Visual Object Recognition and Tracking. Ph.D. dissertation, Massachusetts Institute of Technology.
 * Görür, D. (2007). Nonparametric Bayesian Discrete Latent Variable Models for Unsupervised Learning. Ph.D. dissertation, Max Planck Institute for Biological Cybernetics.
 * Thibaux, R. J. (2008). Nonparametric Bayesian Models for Machine Learning. Ph.D. dissertation, Department of Statistics, University of California, Berkeley.
 * Frigyik, B. A., Kapila, A., & Gupta, M. R. (2010). Introduction to the dirichlet distribution and related processes. Department of Electrical Engineering, University of Washington. UWEETR-2010-0006.
 * Gershman, S. J., & Blei, D. M. (2012). A tutorial on Bayesian Nonparametric Models. Journal of Mathematical Psychology, 56(1), 1-12.
 * Ghahramani, Z. (2013). Bayesian non-parametrics and the probabilistic approach to modelling. Phil. Trans. R. Soc. A, 371, 20110553.

Tutorials

 * Infinite Mixture Models with Nonparametric Bayes and the Dirichlet Process by Edwin Chen
 * Dirichlet Processes by Emin Orhan
 * Dirichlet Processes: A Gentle Tutorial by Khalid El-Arini
 * Dirichlet processes, Chinese restaurant processes and all that by Michael Jordan - NIPS 2005
 * Non-parametric Bayesian Methods by Zoubin Ghahramani - UAI 2005
 * Machine Learning from a Nonparametric Bayesian Point of View by Michael Jordan (Youtube) - Rutgers 2008
 * An Introduction to Bayesian Nonparametric Modelling by Yee Whye Teh - Toronto 2009
 * An Introduction to Bayesian Nonparametric Modelling by Yee Whye Teh - MSR 2009
 * An Introduction to Bayesian Nonparametrics by Yee Whye Teh - 2009
 * An Introduction to Bayesian Nonparametric Modelling by Yee Whye Teh - MLSS 2009
 * Bayesian Nonparametrics by Yee Whye Teh - CIMAT 2010
 * Bayesian Nonparametrics by Yee Whye Teh - KAIST 2010
 * Introduction to Bayesian Nonparametrics by Yee Whye Teh - MLSS September 2011
 * An Introduction to Bayesian Nonparametric Modelling by Yee Whye Teh - MLSS June 2011
 * Modern Bayesian nonparametrics by P Orbanz and YW Teh (Youtube) - NIPS 2011
 * Probabilistic Modelling, Machine Learning, and the Information Revolution by Zoubin Ghahramani - 2012
 * Bayesian Nonparametrics: Models Based on the Dirichlet Process by Alessandro Panella - 2013

Software

 * Edward: A library for probabilistic modeling, inference, and criticism - Python with TensorFlow
 * Edward2 - Python with TensorFlow
 * Pyro - Python with PyTorch
 * NPBayesHMM - Nonparametric Bayesian Inference for Sequential Data in MATLAB
 * DPackage - R
 * DIRECT - R
 * profdpm - R
 * bnpy - Python (Bitbucket)
 * bnpy - Python (Github)
 * PyMC3 - Python with Theano
 * datamicroscopes - Python
 * Nonparametric Bayesian Mixture - MATLAB and C
 * Hierarchical Bayesian compiler
 * Adaptor grammars
 * The MIT-Church project
 * Infer.NET - Developed by Microsoft Research
 * OpenBUGS - Bayesian Inference Using Gibbs Sampling
 * DPMM - MATLAB. Sampling (MCMC) and variational.
 * Variational Gaussian DPMM - MATLAB
 * DPVC - MATLAB
 * IDP - MATLAB and R
 * BNPgraph - MATLAB

Other Resources

 * Tutorials on Bayesian Nonparametrics
 * Dirichlet Process: Practical course - MATLAB
 * notes-on-dirichlet-processes - IPython notebooks explaining DPs, HDPs, and LDA
 * Clustering with Dirichlet process mixtures - MATLAB practical
 * List of papers on Nonparametric Bayes by Yee Whye Teh
 * List of papers on Bayesian Nonparametrics by Michael Jordan
 * List of papers in Bayesian Nonparametric by Dan Roy
 * NPBayes 2008 - Nonparametric Bayes Workshop at ICML/UAI/COLT 2008
 * NPBayes 2009
 * Course notes on Bayesian Nonparametrics by Athanasios Kottas
 * Bayesian machine learning - Metacademy
 * Dirichlet Process, Infinite Mixture Models, and Clustering - Python and R
 * Introduction to Bayesian Nonparametrics - blog post
 * Non-parametric Bayesian Models - Zoubin Ghahramani
 * All of Bayesian Nonparametrics (Especially the Useful Bits) - NIPS 2018 Workshop
 * Nonparametric Bayes - Collection of review articles and tutorials