Probabilistic Graphical Model

From Ioannis Kourouklides
Jump to navigation Jump to search

This page contains resources about Probabilistic Graphical Models, Probabilistic Machine Learning and Probabilistic Models, including Latent Variable Models.

Graphical Models do not necessarily follow Bayesian Methods, but they are named after Bayes' Rule. Bayesian and Non-Bayesian (Frequentist) Methods can either be used.

A distinction should be made between Models and Methods (which might be applied on or using these Models).

Subfields and Concepts[edit]

See Category:Probabilistic Graphical Models for some of its subfields.

  • Bayesian Networks (directed graphical models) - not  necessarily following a "Bayesian" approach
  • Markov Random Fields (undirected graphical models)
    • Gibbs Random Field
    • Gaussian MRF / Undirected Gaussian Graphical Model
    • Lattice Model
      • Potts Model
      • Ising Model
    • Hopfield Network
    • Boltzmann Machine
      • Restricted Boltzmann Machine
    • Conditional Random Field
    • Structural Support Vector Machine
    • Deep Boltzmann Machine
    • Associative Markov Network
    • Maximum Entropy (Maxent) Model
    • Structural Support Vector Machine (SSVM) / Max Margin Markov Network (M3net)
    • Factor Graph
  • Stochastic Models (Stochastic Processes, Random Fields, ...)
  • Latent Variable Models (i.e. Partially Observed Probabilistic Models)
  • Mixed Networks (i.e. both deterministic and probabilistic)
  • Chain Graph / Mixed Graph (i.e. both directed and undirected edges)
  • Structure Learning
    • PC Algorithm
    • Network Scoring
    • Chow-Liu Trees
    • Minimal I-Map
    • Bayesian Model Selection
    • Annealed Importance Sampling
    • Sparsity promoting priors / Sparsity inducing priors
      • L2-regularization / Bayesian Ridge Regression / Gaussian prior
      • L1-regularization / Bayesian LASSO / Laplace prior
      • Spike and Slab / Bernoulli-Gaussian prior
  • Inference in graphical models / Probabilistic Inference
    • Exact Inference / Exact Marginalization
      • Enumeration
      • Variable Elimination Algorithm / Bucket Elimination
      • Sum-Product Algorithm / Belief Propagation / Sum-Product Message Passing / Factor Graph propagation
      • Max-Product Algorithm / Max-Product Belief Propagation / Max-Sum Algorithm
      • Conditioning
      • Junction Tree Algorithm / Clique Tree Propagation
      • Forward-Backward Algorithm (used for HMM)
      • Baum-Welch Algorithm (used for HMM)
      • Viterbi Algorithm (used for HMM)
    • Approximate Inference

Online Courses[edit]

Video Lectures[edit]


Lecture Notes[edit]

Books and Book Chapters[edit]

  • Jordan, M. I. (TBA) An Introduction to Probabilistic Graphical Models. (draft)
  • Bellot, D. (2016). Learning Probabilistic Graphical Models in R. Packt Publishing.
  • Pfeffer, A. (2016). Practical probabilistic programming. Manning Publications Co.
  • Koduvely, H. M. (2015). Learning Bayesian Models with R. Packt Publishing.
  • Theodoridis, S. (2015). Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
  • Hastie, T., Tibshirani, R., & Wainwright, M. (2015). "Chapter 9: Graphs and Model Selection". Statistical learning with sparsity: the lasso and generalizations. CRC Press.
  • Davidson-Pilon, C. (2015). Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference. Addison-Wesley Professional.
  • Ankan, A., & Panda, A. (2015). Mastering Probabilistic Graphical Models Using Python. Packt Publishing Ltd.
  • Nagarajan, R., Scutari, M., & Lèbre, S. (2013). Bayesian Networks in R. Springer, 122, 125-127.
  • Barber, D. (2012). Bayesian Reasoning and Machine Learning. Cambridge University Press.
  • Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press.
  • Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern Classification. John Wiley & Sons.
  • Neal, R. M. (2012). Bayesian learning for neural networks. Springer Science & Business Media.
  • Russell, S. J., & Norvig, P. (2010). "Part IV: Uncertain knowledge and reasoning". Artificial Intelligence: A Modern Approach. Prentice Hall.
  • Alpaydin, E. (2010). "Chapter 16: Graphical Models". Introduction to machine learning. MIT Press.
  • Koller, D., & Friedman, N. (2009). Probabilistic Graphical Models. MIT Press.
  • Darwiche, A. (2009). Modeling and reasoning with Bayesian networks. Cambridge University Press.
  • Borgelt, C., Steinbrecher, M., & Kruse, R. R. (2009). Graphical Models - Representations for Learning, Reasoning and Data Mining. John Wiley & Sons.
  • Theodoridis, S., Pikrakis, A., Koutroumbas, K., & Cavouras, D. (2008). "Chapter 9: Context-dependent Classification". Pattern Recognition. 4th Ed. Academic Press.
  • Wainwright, M. J., & Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning, 1(1-2), 1-305.
  • Bishop, C. M. (2006). "Chapter 8. Graphical Models". Pattern Recognition and Machine Learning. Springer. pp. 359–422.
  • Jordan, M. I. (2003). An Introduction to Probabilistic Graphical Models.
  • Jordan, M. I., & Sejnowski, T. J. (Ed.). (2001). Graphical models: Foundations of neural computation. MIT Press.
  • Cowell, R. G., D., A. Philip, L., Steffen L., & Spiegelhalter, D. J. (1999). Probabilistic Networks and Expert Systems. Springer.
  • Lauritzen, S. L. (1996). Graphical Models. Oxford University Press.
  • Jensen, F. (1996). An Introduction to Bayesian Networks. Springer.
  • Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann.
  • Jordan, M. I. (Ed.). (1998). Learning in graphical models. Kluwer Academic Publishers.

Scholarly Articles[edit]

  • Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature, 521(7553), 452-459.
  • Larranaga, P., & Moral, S. (2011). Probabilistic graphical models in artificial intelligence. Applied soft computing, 11(2), 1511-1528.
  • Airoldi, E. M. (2007). Getting Started in Probabilistic Graphical Models. PLoS Computational Biology, 3(12), e252.
  • Wainwright, M. J., & Jordan, M. I. (2008). Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends® in Machine Learning, 1(1-2), 1-305.
  • Koller, D., Friedman, N., Getoor, L., & Taskar, B. (2007). 2 Graphical Models in a Nutshell. Statistical Relational Learning, 13.
  • Silva, R., Scheine, R., Glymour, C., & Spirtes, P. (2006). Learning the structure of linear latent variable models. Journal of Machine Learning Research, 7(Feb), 191-246.
  • Frey, B. J., & Jojic, N. (2005). A comparison of algorithms for inference and learning in probabilistic graphical models. IEEE Transactions on pattern analysis and machine intelligence, 27(9), 1392-1416.
  • Ghahramani, Z. (2004). Unsupervised learning. In Advanced lectures on machine learning (pp. 72-112). Springer.
  • Jordan, M. I. (2004). Graphical Models. Statistical Science, 140-155.
  • Jordan, M. I., & Weiss, Y. (2002). Graphical models: Probabilistic inference.The handbook of brain theory and neural networks, 490-496.

Tutorials[edit]

Software[edit]

See also[edit]

Other Resources[edit]

Retrieved from "https://wiki.kourouklides.com/w/index.php?title=Probabilistic_Graphical_Model&oldid=1902"