Bayesian Network

This page contains resources about Belief Networks and Bayesian Networks (directed graphical models), also called Bayes Networks.

Bayesian Networks do not necessarily follow Bayesian Methods, but they are named after Bayes' Rule. Bayesian and Non-Bayesian (Frequentist) Methods can either be used.

A distinction should be made between Models and Methods (which might be applied on or using these Models).

Subfields and Concepts

• Naive Bayes classifier
  • Bayesian Naive Bayes
  • Tree Augmented Naive Bayes
• Gaussian Bayes Network / Gaussian Belief Net / Directed Gaussian Graphical Model
• Dynamic Bayesian Network
  • Hidden Markov Model
    • Coupled HMM
    • Factorial HMM
    • Autoregressive HMM / Regime Switching Markov Model
    • Hierarchical HMM
    • Hidden Markov Random Field
  • Linear Dynamical System / State Space Model
    • Kalman filter / Linear Gaussian State Space Model
    • Time Series Model
    • SSM with Regime Switching / Jump Markov Linear Systems / Switching LDS / Switching SSM
• Bayesian Nonparametrics
• Deep Belief Network
• Stochastic Computation Graph
• Factor Analyzer
• Auto-Regressive Network / Fully-visible Bayes Network (FVBN)
• Variational Autoencoder (VAE)

Online Courses

Video Lectures

• Learning Bayesian Networks by Richard E. Neapolitan - VideoLectures.Net

Books and Book Chapters

• Davidson-Pilon, C. (2015). Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference. Addison-Wesley Professional.
• Conrady, S., & Jouffe, L. (2015). Bayesian Networks and BayesiaLab: A Practical Introduction for Researchers. BayesiaLab USA.
• Koduvely, H. M. (2015). Learning Bayesian Models with R. Packt Publishing.
• Theodoridis, S. (2015). "Section 15.3: Bayesian Networks and the Markov Condition". Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
• Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2014). Bayesian data analysis. Boca Raton, FL, USA: Chapman & Hall/CRC.
• Nagarajan, R., Scutari, M., & Lèbre, S. (2013). Bayesian Networks in R. Springer, 122, 125-127.
• Barber, D. (2012). "Chapter 3: Belief Networks". Bayesian Reasoning and Machine Learning. Cambridge University Press.
• Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern Classification. John Wiley & Sons.
• Murphy, K. P. (2012). "Chapter 10: Directed graphical models (Bayes nets) ". Machine Learning: A Probabilistic Perspective. MIT Press.
• Russell, S. J., & Norvig, P. (2010). "Part IV: Uncertain knowledge and reasoning". Artificial Intelligence: A Modern Approach. Prentice Hall.
• Koller, D., & Friedman, N. (2009). "Chapter 3: The Bayesian Network Representation". Probabilistic Graphical Models. MIT Press.
• Darwiche, A. (2009). Modeling and Reasoning with Bayesian Networks. Cambridge University Press.
• Nielsen, T. D., & Jensen, F. V. (2007). Bayesian Networks and Decision Graphs. Springer Science & Business Media.
• Bishop, C. M. (2006). "Section 8.1: Bayesian Networks". Pattern Recognition and Machine Learning. Springer.
• Murphy, K. P. (2002). "Chapter: Dynamic Bayesian Networks". In M. Jordan, Probabilistic Graphical Models.
• Murphy, K. P. (2002). "Dynamic Bayesian Networks: Representation, Inference and Learning". PhD Diss. University of California, Berkeley.
• Friedman, N., Murphy, K., & Russell, S. (1998). Learning the structure of dynamic probabilistic networks. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence (pp. 139-147). Morgan Kaufmann.
• Ghahramani, Z. (1998). Learning dynamic Bayesian networks. In Giles, C. L., & Gori, M. (Eds.). Lecture Notes in Computer Science 1387. Adaptive processing of sequences and data structures, (pp. 168-197). Springer.
• Mitchell, T. M. (1997). "Chapter 6: Bayesian Learning". Machine Learning. McGraw Hill.
• Jensen, F. (1996). An Introduction to Bayesian Networks. Springer.

Scholarly Articles

• Mnih, A., & Gregor, K. (2014). Neural variational inference and learning in belief networks. arXiv preprint arXiv:1402.0030.
• Heckerman, D., Geiger, D., & Chickering, D. M. (1995). Learning Bayesian networks: The combination of knowledge and statistical data. Machine learning, 20(3), 197-243.

Tutorials

• Heckerman's Bayes Net Learning Tutorial
• A Brief Introduction to Graphical Models and Bayesian Networks by K. Murphy
• A brief introduction to Bayes' Rule by K. Murphin
• An Introduction to Graphical Models by M. Jordan
• Bayesian Modelling in Machine Learning: A Tutorial Review
• Bayesian Methods for Machine Learning - NIPS 2004
• Bayesian Machine Learning by Ian Murray
• Bayesian Machine Learning by Zoubin Ghahramani
• A Tutorial on Dynamic Bayesian Networks by Kevin P. Murphy

Software

See Software and Software for complete lists.

• Edward: A library for probabilistic modeling, inference, and criticism - Python with TensorFlow
• pgmpy - Python
• PyJAGS - Python
• Mocapy++ - A Dynamic Bayesian Network toolkit, implemented in C++ (It supports discrete, multinomial, Gaussian, Kent, Von Mises and Poisson nodes. Inference and Learning is done by Gibbs Sampling/Stochastic-EM.)
• Stan - Python (PyStan) and R (RStan) interfaces
• PyMC3 - Python
• Bayesian Probabilistic Matrix Factorization - MATLAB
• Bayesian Modeling and Monte Carlo Methods - MATLAB
• Bayesian Methods for Hackers - Python
• Infer.NET - Developed by Microsoft Research
• OpenBUGS - Bayesian Inference Using Gibbs Sampling
• gRain: Graphical Independence Networks - R
• Naive Bayes (Statistics and Machine Learning Toolbox) - MATLAB

See also

• Markov Random Fields / Markov Networks - undirected graphical models
• Stochastic Processes
• Bayesian Machine Learning

Other Resources

• Are "Bayesian networks" Bayesian? - No, Bayesian and Frequentist approaches can both be used.
• Probabilistic Graphical Models for Fraud Detection (Part 1, Part 2, Part 3) - R