Bayesian Nonparametrics

This page contains resources about Bayesian Nonparametrics and Bayesian Nonparametric Models.

A distinction should be made between Models and Methods (which might be applied on or using these Models).

Subfields

See Category:Bayesian Nonparametrics for some of its subfields.

• Nonparametric Bayesian Models
  • Gamma Process Nonnegative Matrix Factorization (GaP-NMF) Model
  • Gaussian Process
  • Dirichlet Process
  • Chinese Restaurant Process (CRP)
  • Chinese Restaurant Franchise (CRF)
  • Indian Buffet Process (IBP)
  • Pitman–Yor Process
  • Hierarchical Dirichlet Process (HDP)
  • Mixture of Dirichlet Processes (MDP)
  • Dirichlet Processes Mixture Model (DPMM)
  • CRP Mixture Model
  • IBP Latent Factor Model
  • Latent Dirichlet Allocation (LDA)
  • Levy Process
  • Bernoulli Process
  • Completely Random Measures
    • Poisson Random Measure / Poisson Point Process
    • Gamma Process
    • Beta Process / Beta-Bernoulli Process
    • Stable Process 
  • Polya Trees
  • Polya's Urn Process
  • Hoppe's Urn Process
  • Stick Breaking Process
• Nonparametric Bayesian Methods
  • Consistency
  • Contraction rates

Online Courses

Video Lectures

• Bayesian Nonparametrics by Yee Whye Teh - VideoLectures.Net
• Nonparametric Bayesian Models by Yee Whye Teh - VideoLectures.Net
• Dirichlet Processes, Chinese Restaurant Processes, and all that - VideoLectures.Net
• Foundations of Nonparametric Bayesian Models by Peter Orbanz - VideoLectures.Net

Lecture Notes

• Bayesian Nonparametrics by Peter Orbanz
• Bayesian Nonparametrics by David M. Blei
• Advanced Methods in Probabilistic Modeling BY David M. Blei
• Topics in Probability: Lévy Processes by Davar Khoshnevisan
• Nonparametric Bayesian Methods (Dirichlet Process Mixtures) by Jun Zhu
• Nonparametric Bayesian methods (Dirichlet processes) by Kurt Miller
• CRiSM Master Class Non-Parametric Bayes by David Dunson and Harry van Zanten

Books and Book Chapters

• Küchler, U., & Sorensen, M. (1997). Exponential families of stochastic processes. Springer Science & Business Media.
• Dey, D. D., MuIler, P., & Sinha, D. (Eds.). (1998). Practical nonparametric and semiparametric Bayesian statistics (Vol. 133). Springer Science & Business Media.
• Ghosh, J. K., & Ramamoorthi, R. V. (2003). Bayesian Nonparametrics.Springer Series in Statistics. Springer-Verlag, New York, 16, 37.
• Gorur, D. (2007). Nonparametric Bayesian Discrete Latent Variable Models for Unsupervised Learning. PhD Diss. Technical University of Berlin.
• Koller, D., & Friedman, N. (2009). "Section 19.5: Learning Models with Hidden Variables ". Probabilistic Graphical Models. MIT Press.
• Hjort, N. L., Holmes, C., Muller, P., & Walker, S. G. (Eds.). (2010). Bayesian Nonparametrics. Cambridge University Press.
• Orbanz, P., & Teh, Y. W. (2011). Bayesian Nonparametric Models. In Encyclopedia of Machine Learning (pp. 81-89). Springer US.
• Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. Chapter 25: Clustering. MIT Press.
• Jordan, M. I. (2013). Hierarchical models, nested models and completely random measures. Frontiers of Statistical Decision Making and Bayesian Analysis: in Honor of James O. Berger. New York: Springer, 207-218.
• Theodoridis, S. (2015). "Section 13.12: Nonparametric Bayesian Modeling". Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
• Muller, P., Quintana, F. A., Jara, A., & Hanson, T. (2015). Nonparametric Bayesian Data Analysis. New York: Springer.
• Phadia, E. G. (2015). Prior Processes and Their Applications: Nonparametric Bayesian Estimation. Springer.
• Mitra, R., & Muller, P. (Eds.). (2015). Nonparametric Bayesian Inference in Biostatistics. Springer.
• Goodman, N. D., & Tenenbaum, J. B. (2016). "Chapter 12: Non-parametric models".  Probabilistic Models of Cognition. 2nd Ed. (link)
• Ghosal, S., & Van der Vaart, A. (2017). Fundamentals of nonparametric Bayesian inference. Cambridge University Press.

Scholarly Articles

See also NPBayes 2008 for more references.

• Damien, P. (2005). Some Bayesian Nonparametric Models. Handbook of statistics, 25, 279-314.
• Hanson, T. E., Branscum, A. J., & Johnson, W. O. (2005). Bayesian nonparametric modeling and data analysis: an introduction. Handbook of statistics, 25, 245-278.
• Walker, S. (2005). Bayesian Nonparametric Inference. Handbook of statistics, 25, 339-371.
• Sudderth, E.B. (2006). Graphical Models for Visual Object Recognition and Tracking. Ph.D. dissertation, Massachusetts Institute of Technology.
• Görür, D. (2007). Nonparametric Bayesian Discrete Latent Variable Models for Unsupervised Learning. Ph.D. dissertation, Max Planck Institute for Biological Cybernetics.
• Thibaux, R. J. (2008). Nonparametric Bayesian Models for Machine Learning. Ph.D. dissertation, Department of Statistics, University of California, Berkeley.
• Frigyik, B. A., Kapila, A., & Gupta, M. R. (2010). Introduction to the dirichlet distribution and related processes. Department of Electrical Engineering, University of Washington. UWEETR-2010-0006.
• Gershman, S. J., & Blei, D. M. (2012). A tutorial on Bayesian Nonparametric Models. Journal of Mathematical Psychology, 56(1), 1-12.
• Ghahramani, Z. (2013). Bayesian non-parametrics and the probabilistic approach to modelling. Phil. Trans. R. Soc. A, 371, 20110553.

Tutorials

• Infinite Mixture Models with Nonparametric Bayes and the Dirichlet Process by Edwin Chen
• Dirichlet Processes by Emin Orhan
• Dirichlet Processes: A Gentle Tutorial by Khalid El-Arini
• Dirichlet processes, Chinese restaurant processes and all that by Michael Jordan - NIPS 2005
• Non-parametric Bayesian Methods by Zoubin Ghahramani - UAI 2005
• Machine Learning from a Nonparametric Bayesian Point of View by Michael Jordan (Youtube) - Rutgers 2008
• An Introduction to Bayesian Nonparametric Modelling by Yee Whye Teh - Toronto 2009
• An Introduction to Bayesian Nonparametric Modelling by Yee Whye Teh - MSR 2009
• An Introduction to Bayesian Nonparametrics by Yee Whye Teh - 2009
• An Introduction to Bayesian Nonparametric Modelling by Yee Whye Teh - MLSS 2009
• Bayesian Nonparametrics by Yee Whye Teh - CIMAT 2010
• Bayesian Nonparametrics by Yee Whye Teh - KAIST 2010
• Introduction to Bayesian Nonparametrics by Yee Whye Teh - MLSS September 2011
• An Introduction to Bayesian Nonparametric Modelling by Yee Whye Teh - MLSS June 2011
• Modern Bayesian nonparametrics by P Orbanz and YW Teh (Youtube) - NIPS 2011
• Probabilistic Modelling, Machine Learning, and the Information Revolution by Zoubin Ghahramani - 2012
• Bayesian Nonparametrics: Models Based on the Dirichlet Process by Alessandro Panella - 2013

Software

• Edward: A library for probabilistic modeling, inference, and criticism - Python with TensorFlow
• Edward2 - Python with TensorFlow
• Pyro - Python with PyTorch
• NPBayesHMM - Nonparametric Bayesian Inference for Sequential Data in MATLAB
• DPackage - R
• DIRECT - R
• profdpm - R
• bnpy - Python (Bitbucket)
• bnpy - Python (Github)
• PyMC3 - Python with Theano
• datamicroscopes - Python
• Nonparametric Bayesian Mixture - MATLAB and C
• Hierarchical Bayesian compiler
• Adaptor grammars
• The MIT-Church project
• Infer.NET - Developed by Microsoft Research
• OpenBUGS - Bayesian Inference Using Gibbs Sampling
• DPMM - MATLAB. Sampling (MCMC) and variational.
• Variational Gaussian DPMM - MATLAB
• DPVC - MATLAB
• IDP - MATLAB and R
• BNPgraph - MATLAB

See also

• Discrete Latent Variable Models
• Stochastic Processes and Random Fields
• Kernel Methods

Other Resources

• Tutorials on Bayesian Nonparametrics
• Dirichlet Process: Practical course - MATLAB
• notes-on-dirichlet-processes - IPython notebooks explaining DPs, HDPs, and LDA
• Clustering with Dirichlet process mixtures - MATLAB practical
• List of papers on Nonparametric Bayes by Yee Whye Teh
• List of papers on Bayesian Nonparametrics by Michael Jordan
• List of papers in Bayesian Nonparametric by Dan Roy
• NPBayes 2008 - Nonparametric Bayes Workshop at ICML/UAI/COLT 2008
• NPBayes 2009
• Course notes on Bayesian Nonparametrics by Athanasios Kottas
• Bayesian machine learning - Metacademy
• Dirichlet Process, Infinite Mixture Models, and Clustering - Python and R
• Introduction to Bayesian Nonparametrics - blog post
• Non-parametric Bayesian Models - Zoubin Ghahramani
• All of Bayesian Nonparametrics (Especially the Useful Bits) - NIPS 2018 Workshop
• Nonparametric Bayes - Collection of review articles and tutorials