Linear Dynamical System

This page contains resources about Linear Dynamical Systems, Linear Systems Theory, Dynamic Linear Models, Linear State Space Models and State-Space Representation, including temporal (Time Series) and atemporal Sequential Data.

Subfields and Concepts

 * Linear SSM
 * Discrete-time LDS
 * Continuous-time LDS
 * Linear Time-Invariant (LTI) system
 * Linear Time-Variant System
 * Parametric models / Time Series models
 * Autoregressive (AR) model / All-Pole model
 * Moving Average (MA) model / All-Zero model
 * ARMA model / Pole-Zero model
 * ARIMA and ARIMAX
 * Seasonal ARIMA (SARIMA) and SARIMAX
 * Autoregressive Conditional Heteroskedasticity (ARCH) model
 * Generalized ARCH (GARCH) model
 * Vector Autoregressive (VAR) model
 * Vector ARMA (VARMA) model
 * Martin Distance (for comparing ARMA processes)
 * Kalman filter / Linear Gaussian SSM
 * Stochastic LDS
 * Structured LDS
 * Bayesian SSM
 * Bayesian Time Series
 * Bayesian LDS
 * SSM with Regime Switching / Jump Markov Linear Systems / Switching LDS / Switching SSM
 * Kernels on Dynamical Systems
 * Computer Vision
 * Linear Dynamic Texture
 * Kernel Dynamic Texture
 * Time Series
 * Univariate Time Series
 * Multivariate Time Series
 * Time Series Forecasting
 * One-step ahead Forecasting
 * Multi-step ahead Forecasting
 * Dynamic Forecasting

Video Lectures

 * Introduction to Linear Dynamical Systems by Stephen Boyd
 * Topics in Mathematics with Applications in Finance by Peter Kempthorne, Choongbum Lee, Vasily Strela and Jake Xia

Lecture Notes

 * Dynamic Systems and Control by Emilio Frazzoli & Munther Dahleh
 * Linear Systems Theory by John Lygeros and Federico A. Ramponi
 * Linear System Theory by Claire Tomlin
 * Time Series Econometrics by Peter C. B. Phillips
 * Time Series Econometrics by Eric Zivot
 * Econometrics II by Rauli Susmel
 * Applied Econometrics by Baum
 * Dynamical Systems and Stochastic Processes by Pierre Collet
 * Linear Dynamical Systems by Stephen Boyd
 * Applied Time Series Analysis
 * Time Series Analysis I by Suhasini Subba Rao
 * Applied Forecasting for Business and Economics by Rob J Hyndman
 * Lecture 10: Sequential Data Models by Geoffrey Hinton

Books and Book Chapters
See also Further Reading.
 * Hyndman, R. J., & Athanasopoulos, G. (2018). Forecasting: Principles and Practice. 2nd Ed. OTexts. (link)
 * Brockett, R. W. (2015). Finite dimensional linear systems. SIAM.
 * Murphy, K. P. (2012). "Chapter 18: State space models". Machine Learning: A Probabilistic Perspective. MIT Press.
 * Barber, D. (2012). "Chapter 24: Continuous-State Markov Models". Bayesian Reasoning and Machine Learning. Cambridge University Press.
 * Barber, D. (2012). "Chapter 25: Switching Linear Dynamical Systems". Bayesian Reasoning and Machine Learning. Cambridge University Press.
 * Durbin, J., & Koopman, S. J. (2012). Time series analysis by state space methods. Oxford University Press.
 * Casti, J. L. (2012). Linear dynamical systems. Academic Press Professional.
 * Prado, R., & West, M. (2010). Time series: modeling, computation, and inference. CRC Press.
 * Tsay, R. S. (2010). Analysis of Financial Time Series. 3rd Ed. John Wiley & Sons.
 * Petris, G., Petrone, S., & Campagnoli, P. (2009). Dynamic Linear Models with R. Springer New York.
 * Hespanha, J. P. (2009). Linear systems theory. Princeton university press.
 * Zadeh, L. A., & Desoer, C. A. (2008). Linear System Theory: The State Space Approach. Dover.
 * Commandeur, J. J., & Koopman, S. J. (2007). An introduction to state space time series analysis. Oxford University Press.
 * Antsaklis, P. J., & Michel, A. N. (2007). A Linear Systems Primer. Springer Science & Business Media.
 * Antsaklis, P. J., & Michel, A. N. (2006). Linear systems. Springer Science & Business Media.
 * Bishop, C. M. (2006). "Chapter 13: Sequential Data". Pattern Recognition and Machine Learning. Springer.
 * Gajic, Z. (2003). Linear dynamic systems and signals. Prentice Hall/Pearson Education.
 * Chatfield, C. (2003). The analysis of time series: an introduction. 6th Ed. CRC press.
 * Harrison, J., & West, M. (1999). Bayesian Forecasting & Dynamic Models. Springer.
 * Chen, C. T. (1998). Linear system theory and design. Oxford University Press.
 * Rugh, W. J. (1996). Linear system theory. Prentice Hall.
 * Hamilton, J. D. (1994). Time series analysis. Princeton University Press.
 * Callier, F. M., & Desoer, C. A. (1991). Linear System Theory. Springer New York.
 * Harvey, A. C. (1990). Forecasting, structural time series models and the Kalman filter. Cambridge university press.
 * Harvey, A. C. (1993). Time series models. 2nd Ed. The MIT Press.
 * Delchamps, D. F. (1988). State space and input-output linear systems. Springer Science & Business Media.
 * Cryer, J. D. (1986). Time series analysis. Duxbury Press.
 * Kailath, T. (1980). Linear systems. Prentice-Hall.
 * Luenberger, D. G. (1979). Introduction to dynamic systems. John Wiley & Sons.

Scholarly Articles

 * Archer, E., Park, I. M., Buesing, L., Cunningham, J., & Paninski, L. (2015). Black box variational inference for state space models. arXiv preprint arXiv:1511.07367.
 * Taieb, S. B., Bontempi, G., Atiya, A. F., & Sorjamaa, A. (2012). A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition. Expert systems with applications, 39(8), 7067-7083.
 * Petris, G., & Petrone, S. (2011). State space models in R. Journal of Statistical Software, 41(4), 1-25.
 * Taieb, S. B., Sorjamaa, A., & Bontempi, G. (2010). Multiple-output modeling for multi-step-ahead time series forecasting. Neurocomputing, 73(10-12), 1950-1957.
 * Vishwanathan, S. V. N., Smola, A. J., & Vidal, R. (2007). Binet-Cauchy kernels on dynamical systems and its application to the analysis of dynamic scenes. International Journal of Computer Vision, 73(1), 95-119.
 * Chan, A. B., & Vasconcelos, N. (2007). Classifying video with kernel dynamic textures. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1-6).
 * Cheng, H., Tan, P. N., Gao, J., & Scripps, J. (2006). Multistep-ahead time series prediction. In Pacific-Asia Conference on Knowledge Discovery and Data Mining (pp. 765-774). Springer.
 * Rudary, M., Singh, S., & Wingate, D. (2005). Predictive linear-Gaussian models of stochastic dynamical systems. Conference on Uncertainty in Artificial Intelligence.
 * Doretto, G., Chiuso, A., Wu, Y. N., & Soatto, S. (2003). Dynamic textures. International Journal of Computer Vision, 51(2), 91-109.
 * Martin, R. J. (2000). A metric for ARMA processes. IEEE Transactions on Signal Processing, 48(4), 1164-1170.
 * Minka, T. (1999). From hidden markov models to linear dynamical systems. Technical Report, MIT.
 * Kim, C. J. (1994). Dynamic linear models with Markov-switching. Journal of Econometrics, 60(1-2), 1-22.
 * Ghahramani, Z., & Hinton, G. E. (1996). Parameter estimation for linear dynamical systems. Technical Report CRG-TR-96-2, University of Toronto, Dept. of Computer Science.
 * Kalman, R. E. (1963). Mathematical description of linear dynamical systems. Journal of the Society for Industrial and Applied Mathematics, Series A: Control, 1(2), 152-192.

Software

 * Control Systems Toolbox - MATLAB
 * System Identification Toolbox - MATLAB
 * Econometrics Toolbox - MATLAB
 * Kalman filter and Linear Dynamical System -MATLAB
 * DLM Matlab Toolbox
 * Python Control Systems Toolbox
 * prophet - Python, R
 * dynpy - Python
 * PyDSTool - Python
 * Statsmodels - Python
 * arch - Python
 * PyFlux - Python
 * DLM - R
 * dynr - R
 * timeSeries - R
 * zoo - R
 * tsDyn - R

Other Resources

 * State Space Models in Python
 * Time Series Analysis in R
 * Time Series - Notebook
 * State-Space Reconstruction  - Notebook
 * Time Series Analysis (TSA) in Python - Linear Models to GARCH
 * Time Series and Sequential Data - Zoubin Ghahramani
 * A comprehensive beginner’s guide to create a Time Series Forecast (with Codes in Python) - Blog post
 * Cross-validation for time series - Blog post
 * Time Series Analysis - Blog post
 * Simple Time Series Forecasting Models to Test So That You Don’t Fool Yourself - Blog post
 * How To Backtest Machine Learning Models for Time Series Forecasting - Blog post
 * How to Make Baseline Predictions for Time Series Forecasting with Python - blog post
 * Making Predictions with Sequences - blog post
 * 11 Classical Time Series Forecasting Methods in Python (Cheat Sheet) - blog post
 * How to Create an ARIMA Model for Time Series Forecasting in Python - blog post
 * Autoregression Models for Time Series Forecasting With Python - blog post
 * 4 Strategies for Multi-Step Time Series Forecasting - blog post
 * How to Develop Machine Learning Models for Multivariate Multi-Step Air Pollution Time Series Forecasting - blog post
 * Multi-step Time Series Forecasting with Machine Learning for Household Electricity Consumption - blog post
 * A Gentle Introduction to SARIMA for Time Series Forecasting in Python - blog post
 * How to Model Volatility with ARCH and GARCH for Time Series Forecasting in Python - blog post
 * How to Tune ARIMA Parameters in Python - blog post
 * 11 Classical Time Series Forecasting Methods in Python (Cheat Sheet) - blog post
 * 7 Ways Time-Series Forecasting Differs from Machine Learning - blog post
 * Open Machine Learning Course. Topic 9. Part 1. Time series analysis in Python - blog post
 * Trend, Seasonality, Moving Average, Auto Regressive Model : My Journey to Time Series Data with Interactive Code - blog post
 * Time Series Forecast : A basic introduction using Python. - blog post
 * Using Python and Auto ARIMA to Forecast Seasonal Time Series - blog post
 * Fitting an AR model: algorithm module interface
 * Implementing and estimating an ARMA(1, 1) state space mode - blog post
 * Time Series for scikit-learn People (Part I): Where's the X Matrix? - blog post
 * Time Series for scikit-learn People (Part II): Autoregressive Forecasting Pipelines - blog post
 * Modern Pandas (Part 7): Timeseries - blog post
 * How To Predict Multiple Time Series With Scikit-Learn (With a Sales Forecasting Example) - blog post
 * statsmodels - Documentation
 * pymar (GitHub) - code
 * tsa-notebooks (GitHub) - code
 * NN5 dataset