# Probabilistic Graphical Model

This page contains resources about Probabilistic Graphical Models, Probabilistic Machine Learning and Probabilistic Models, including Latent Variable Models.

Graphical Models do not necessarily follow Bayesian Methods, but they are named after Bayes' Rule. Bayesian and Non-Bayesian (Frequentist) Methods can either be used.

A distinction should be made between Models and Methods (which might be applied on or using these Models).

## Subfields and Concepts

See Category:Probabilistic Graphical Models for some of its subfields.

• Bayesian Networks (directed graphical models) - not  necessarily following a "Bayesian" approach
• Artificial Neural Network
• Feedforward Nerual Network (Directed Acyclic Graph)
• Recurrent Neural Network (Directed Cyclic Graph)
• Naive Bayes classifier (generative model)
• Bayesian Naive Bayes
• Tree Augmented Naive Bayes
• Logistic Regression (discriminative model)
• Gaussian Bayes Network / Gaussian Belief Net / Directed Gaussian Graphical Model
• Dynamic Bayesian Network (used for Sequential Data / Time Series)
• Deep Belief Network
• Hierarchical Bayesian Model
• Stochastic Computation Graph
• Factor Analyzer
• Auto-Regressive Network / Fully-visible Bayes Network (FVBN)
• Variational Autoencoder (VAE)
• Markov Random Fields (undirected graphical models)
• Gibbs Random Field
• Gaussian MRF / Undirected Gaussian Graphical Model
• Lattice Model
• Potts Model
• Ising Model
• Hopfield Network
• Boltzmann Machine
• Restricted Boltzmann Machine
• Conditional Random Field
• Structural Support Vector Machine
• Deep Boltzmann Machine
• Associative Markov Network
• Maximum Entropy (Maxent) Model
• Structural Support Vector Machine (SSVM) / Max Margin Markov Network (M3net)
• Factor Graph
• Stochastic Models (Stochastic Processes, Random Fields, ...)
• Latent Variable Models (i.e. Partially Observed Probabilistic Models)
• Mixed Networks (i.e. both deterministic and probabilistic)
• Chain Graph / Mixed Graph (i.e. both directed and undirected edges)
• Structure Learning
• PC Algorithm
• Network Scoring
• Chow-Liu Trees
• Minimal I-Map
• Bayesian Model Selection
• Annealed Importance Sampling
• Sparsity promoting priors / Sparsity inducing priors
• L2-regularization / Bayesian Ridge Regression / Gaussian prior
• L1-regularization / Bayesian LASSO / Laplace prior
• Spike and Slab / Bernoulli-Gaussian prior
• Inference in graphical models / Probabilistic Inference
• Exact Inference / Exact Marginalization
• Enumeration
• Variable Elimination Algorithm / Bucket Elimination
• Sum-Product Algorithm / Belief Propagation / Sum-Product Message Passing / Factor Graph propagation
• Max-Product Algorithm / Max-Product Belief Propagation / Max-Sum Algorithm
• Conditioning
• Junction Tree Algorithm / Clique Tree Propagation
• Forward-Backward Algorithm (used for HMM)
• Baum-Welch Algorithm (used for HMM)
• Viterbi Algorithm (used for HMM)
• Approximate Inference

## Books and Book Chapters

• Jordan, M. I. (TBA) An Introduction to Probabilistic Graphical Models. (draft)
• Bellot, D. (2016). Learning Probabilistic Graphical Models in R. Packt Publishing.
• Pfeffer, A. (2016). Practical probabilistic programming. Manning Publications Co.
• Koduvely, H. M. (2015). Learning Bayesian Models with R. Packt Publishing.
• Theodoridis, S. (2015). Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
• Hastie, T., Tibshirani, R., & Wainwright, M. (2015). "Chapter 9: Graphs and Model Selection". Statistical learning with sparsity: the lasso and generalizations. CRC Press.
• Davidson-Pilon, C. (2015). Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference. Addison-Wesley Professional.
• Ankan, A., & Panda, A. (2015). Mastering Probabilistic Graphical Models Using Python. Packt Publishing Ltd.
• Nagarajan, R., Scutari, M., & Lèbre, S. (2013). Bayesian Networks in R. Springer122, 125-127.
• Barber, D. (2012). Bayesian Reasoning and Machine Learning. Cambridge University Press.
• Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press.
• Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern Classification. John Wiley & Sons.
• Neal, R. M. (2012). Bayesian learning for neural networks. Springer Science & Business Media.
• Russell, S. J., & Norvig, P. (2010). "Part IV: Uncertain knowledge and reasoning". Artificial Intelligence: A Modern Approach. Prentice Hall.
• Alpaydin, E. (2010). "Chapter 16: Graphical Models". Introduction to machine learning. MIT Press.
• Koller, D., & Friedman, N. (2009). Probabilistic Graphical Models. MIT Press.
• Darwiche, A. (2009). Modeling and reasoning with Bayesian networks. Cambridge University Press.
• Borgelt, C., Steinbrecher, M., & Kruse, R. R. (2009). Graphical Models - Representations for Learning, Reasoning and Data Mining. John Wiley & Sons.
• Theodoridis, S., Pikrakis, A., Koutroumbas, K., & Cavouras, D. (2008). "Chapter 9: Context-dependent Classification". Pattern Recognition. 4th Ed. Academic Press.
• Wainwright, M. J., & Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning1(1-2), 1-305.
• Bishop, C. M. (2006). "Chapter 8. Graphical Models". Pattern Recognition and Machine Learning. Springer. pp. 359–422.
• Jordan, M. I. (2003). An Introduction to Probabilistic Graphical Models.
• Jordan, M. I., & Sejnowski, T. J. (Ed.). (2001). Graphical models: Foundations of neural computation. MIT Press.
• Cowell, R. G., D., A. Philip, L., Steffen L., & Spiegelhalter, D. J. (1999). Probabilistic Networks and Expert Systems. Springer.
• Lauritzen, S. L. (1996). Graphical Models. Oxford University Press.
• Jensen, F. (1996). An Introduction to Bayesian Networks. Springer.
• Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann.
• Jordan, M. I. (Ed.). (1998). Learning in graphical models. Kluwer Academic Publishers.

## Scholarly Articles

• Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature521(7553), 452-459.
• Larranaga, P., & Moral, S. (2011). Probabilistic graphical models in artificial intelligence. Applied soft computing11(2), 1511-1528.
• Airoldi, E. M. (2007). Getting Started in Probabilistic Graphical Models. PLoS Computational Biology, 3(12), e252.
• Wainwright, M. J., & Jordan, M. I. (2008). Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends® in Machine Learning, 1(1-2), 1-305.
• Koller, D., Friedman, N., Getoor, L., & Taskar, B. (2007). 2 Graphical Models in a Nutshell. Statistical Relational Learning, 13.
• Silva, R., Scheine, R., Glymour, C., & Spirtes, P. (2006). Learning the structure of linear latent variable models. Journal of Machine Learning Research7(Feb), 191-246.
• Frey, B. J., & Jojic, N. (2005). A comparison of algorithms for inference and learning in probabilistic graphical models. IEEE Transactions on pattern analysis and machine intelligence27(9), 1392-1416.
• Ghahramani, Z. (2004). Unsupervised learning. In Advanced lectures on machine learning (pp. 72-112). Springer.
• Jordan, M. I. (2004). Graphical Models. Statistical Science, 140-155.
• Jordan, M. I., & Weiss, Y. (2002). Graphical models: Probabilistic inference.The handbook of brain theory and neural networks, 490-496.