Sparse Coding

This page contains resources about Sparse Coding and Dictionary Learning.

Subfields and Concepts

 * Method of Optimal Directions
 * K-SVD
 * Orthogonal Matching Pursuit
 * Stochastic Gradient Descent
 * Lagrange Dual Method
 * Online Dictionary Learning
 * Sparse Dictionary Learning
 * Sparse Autoencoder

Video Lectures

 * Online Dictionary Learning for Sparse Coding by Julien Mairal - VideoLectures.Net
 * Compressive Sensing and Sparse Recovery by Justin Romberg (Youtube )
 * Sparse Methods for Machine Learning: Theory and Algorithms by Francis R. Bach
 * Lecture: Introduction to Sparse Modeling

Lecture Notes

 * Compressive sensing and Sparse optimization by Aswin C Sankaranarayanan
 * Sparse Coding by Ibrahim Almosallam
 * Lecture: Sparsity and Sparse Coding
 * Lecture: Dictionaries for Sparse Recovery
 * Lecture: Higher Order Statistics
 * Lecture: Why Sparse Coding Works
 * Lecture: Visual Recognition, Learning Features

Books and Book Chapters

 * Bengio, Y., Goodfellow, I. J., & Courville, A. (2016). "Section 13.4: Sparse Coding". Deep Learning. MIT Press.
 * Hastie, T., Tibshirani, R., & Wainwright, M. (2015). "Chapter 8: Sparse Multivariate Methods". Statistical learning with sparsity: the lasso and generalizations. CRC Press.
 * Rish, I., & Grabarnik, G. (2014). Sparse modeling: theory, algorithms, and applications. CRC Press.
 * Northoff, G. (2013). Unlocking the Brain: Volume 1: Coding (Vol. 1). Oxford University Press.
 * Murphy, K. P. (2012). "Chapter 13: Sparse linear models". Machine Learning: A Probabilistic Perspective. MIT Press.

Scholarly Articles
See also References .
 * Zhang, H., & Xu, J. (2014). Sparse Coding on Multiple Manifold Data. In International Conference on Neural Information Processing (pp. 515-523). Springer International Publishing.
 * Tosic, I., & Frossard, P. (2011). Dictionary Learning. IEEE Signal Processing Magazine, 28(2), 27-38.
 * Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2010). Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research,11(Jan), 19-60.
 * Rubinstein, R., Bruckstein, A. M., & Elad, M. (2010). Dictionaries for sparse representation modeling. Proceedings of the IEEE, 98(6), 1045-1057.
 * Rubinstein, R., Zibulevsky, M., & Elad, M. (2010). Double sparsity: Learning sparse dictionaries for sparse signal approximation. IEEE Transactions on signal processing, 58(3), 1553-1564.
 * Gregor, K., & LeCun, Y. (2010). Learning fast approximations of sparse coding. In Proceedings of the 27th International Conference on Machine Learning (pp. 399-406).
 * Mairal, J., Ponce, J., Sapiro, G., Zisserman, A., & Bach, F. R. (2009). Supervised dictionary learning. In Advances in Neural Information Processing Systems (pp. 1033-1040).
 * Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2009). Online dictionary learning for sparse coding. In Proceedings of the 26th International Conference on Machine Learning (pp. 689-696).
 * Rubinstein, R., Zibulevsky, M., & Elad, M. (2008). Efficient implementation of the K-SVD algorithm using batch orthogonal matching pursuit. Technical Report.
 * Aharon, M., Elad, M., & Bruckstein, A. (2006). K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. IEEE Transactions on Signal Processing, 54(11), 4311-4322.
 * Hoyer, P. O. (2004). Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research, 5(Nov), 1457-1469.

Tutorials

 * UFLDL Tutorial on Sparse Autoencoder
 * Sparse Coding for Feature Learning
 * Feature Encoding for Image Representation and Recognition (ICPR 2014)
 * BASes for Images and Surfaces (CVPR 2014)
 * Dictionary Learning in Games (GDC 2014)
 * Dictionary Learning for Sparse Representations (ICASSP 2013)
 * Sparse Coding (ICCV 2013)
 * Sparse Coding (EECV 2012)
 * Multiview Feature Learning (CVPR 2012)
 * Sparse Coding (CVPR 2012)
 * Topic Models, Latent Space Models, Sparse Coding, and All That (ACL 2012)
 * Higher-order Feature Learning (DAGM 2011)
 * Feature Learning for Image Classification (ECCV 2010)
 * Sparse Coding and Dictionary Learning for Image Analysis (CVPR 2010)
 * Sparse Coding and Dictionary Learning for Image Analysis (ICCV 2009)
 * A Friendly Guide To Sparse Coding (2009)

Software

 * Efficient sparse coding algorithms - MATLAB
 * Dictionary Learning Tools - MATLAB
 * KSVD/OMP- MATLAB
 * Matlab codes for sparse coding
 * SPArse Modeling Software (SPAMS) -  Python and R
 * Sparse Coding using the package scikit-learn - Python
 * Sparse Coding using the package sciPy - Python
 * Predictive Sparse Decomposition - Lush
 * CAUSEWAY - using the CVX toolbox

Other Resources

 * Google, Baidu, Dwave Systems focused on Sparse Coding for more accurate image classification and unsupervised feature learning
 * SMALL project - A EU-funded project on sparse models, algorithms and dictionary learning
 * Sparse Coding - Metacademy
 * Sparce Coding - Scholarpedia
 * Sparse Dictionary Learning - Wikipedia
 * Sparse Coding - Wikipedia
 * Sparse Approximation - Wikipedia