Hidden Markov Model

This page contains resources about Hidden Markov Models and Markov Models in general.

For temporal (Time Series) and atemporal Sequential Data, please check Linear Dynamical Systems.

Subfields and Concepts

 * Markov Models
 * Discrete-time Markov Chain (Discrete-time and Discrete State Space)
 * Discrete-time Harris Chain (Discrete-time and Continuous State Space)
 * Continuous-time Markov Chain / Continuous-time Markov Process / Markov Jump Process
 * Continuous-time Stochastic Process with the Markov property (e.g. Wiener Process)
 * Hidden Markov Model (HMM)
 * Coupled HMM
 * Factorial HMM
 * Autoregressive HMM / Regime Switching Markov Model
 * Hierarchical HMM
 * Hidden Markov Random Field
 * Markov Decision Process
 * Partially Observable Markov Decision Process
 * Hierarchical Markov Models
 * Inference in HMM
 * Baum-Welch Algorithm (Expectation-Maximization)
 * Forward-Backward Algorithm
 * Viterbi Algorithm

Book and Book Chapters

 * Puterman, M. L. (2014). Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons.
 * Murphy, K. P. (2012). "Chapter 17: Markov and hidden Markov models". Machine Learning: A Probabilistic Perspective. MIT Press.
 * Barber, D. (2012). "Chapter 23: Discrete-State Markov Models". Bayesian Reasoning and Machine Learning. Cambridge University Press.
 * Barber, D. (2012). "Chapter 7.5: Markov Decision Process ". Bayesian Reasoning and Machine Learning. Cambridge University Press.
 * Alpaydin, E. (2010). "Chapter 15: Hidden Markov Models". Introduction to machine learning. MIT Press.
 * Koller, D., & Friedman, N. (2009). "Section 6.2.3.1: Hidden Markov Models". Probabilistic Graphical Models. MIT Press.
 * Bishop, C. M. (2006). "Chapter 13: Sequential Data". Pattern Recognition and Machine Learning. Springer.

Software

 * hmmlearn - Python
 * markov - C++

Other Resources

 * Markov Models - Notebook
 * Statistical Inference for Markov and Hidden Markov Models - Notebook