Deep Learning
Jump to navigation
Jump to search
This page contains resources about Deep Learning and Representation Learning .
Subfields and Concepts[edit]
- Deep Generative Models
- Deep Directed Networks (directed graphical models)
- Sigmoid Belief Net
- Differentiable Generator Net
- Variational Autoencoder (VAE)
- Generative Adversarial Network (GAN)
- Generative Moment Matching Network
- Convolutional Generative Network
- Auto-Regressive Network / Fully-visible Bayes Network (FVBN)
- Deep Latent Gaussian Model (DLGM)
- Deep AutoRegressive Network (DARN)
- Deep Boltzmann Machines (undirected graphical models)
- Deep Belief Networks (mixed graphs)
- Deep Directed Networks (directed graphical models)
- Deep Neural Networks (i.e. more than two hidden layers)
- Deep Multi-Layer Perceptron (i.e. Stacked RBMs)
- Deep Autoencoders (i.e. two symmetrical DBN)
- DARN
- Deep Neural Decision Forests
- Convolutional Deep Belief Network (i.e. Stacked CRBMs)
- Sparse Coding / Dictionary Learning
- Sparse Autoencoders
- Stacked Denoising Autoencoders
- Bayesian Deep Learning
- Bayesian Neural Networks
Online Courses[edit]
Video Lectures[edit]
- Introduction to Deep Learning - Coursera
- Neural Networks for Machine Learning by Geoffrey Hinton - Coursera
- Deep Learning Specialization by Andrew Ng - Coursera
- Deep Learning by Google - Udacity
- Neural networks class by Hugo Larochelle (Youtube )
- Deep Learning and Neural Networks by Kevin Duh
- Computer Perception with Deep Learning by Yann LeCun (Part 1 , Part 2 )
- Computational Neuroscience and Learning by Eugenio Culurciello (Youtube)
- A tutorial on Deep Learning by Geoffrey Hinton - VideoLectures.Net
- Deep Learning with TensorFlow: Applications of Deep Neural Networks to Machine Learning Tasks by Jon Krohn
Lecture Notes[edit]
- Practical Deep Learning for coders by Jeremy Howard
- Deep Learning by Yann LeCun
- Unsupervised Feature Learning and Deep Learning (UFLDL) by Andrew Ng
- Deep Learning by Bhiksha Raj
- Representation Learning by Yoshua Bengio
- Convolutional Neural Networks for Visual Recognition by Fei-Fei Li & Andrej Karpathy
- Deep Learning by Sargur Srihari
- DataLab Cup 5: Deep Reinforcement Learning
- Deep Learning by Shan-Hung Wu
- Cutting-Edge Trends in Deep Learning and Recognition by Svetlana Lazebnik
Books and Book Chapters[edit]
- Santana, E. (2018). Eder Santana's Deep Learning with Python. Packt Publishing.
- Shukla, N. (2018). Machine learning with TensorFlow. Manning.
- Zaccone, G., Karim, Md. R., & Menshawy, A. (2017). Deep Learning with TensorFlow. Packt Publishing.
- McClure, N. (2017). TensorFlow Machine Learning Cookbook. Packt Publishing.
- Gulli, A., & Pal, S. (2017). Deep Learning with Keras. Packt Publishing.
- Chollet, F. (2017). Deep Learning with Python. Manning Publications.
- Gulli, A., & Kapoor, A. (2017). TensorFlow 1.x Deep Learning Cookbook. Packt Publishing.
- Bengio, Y., Goodfellow, I. J., & Courville, A. (2016). Deep Learning. MIT Press. (link)
- Gibson, A., & Patterson J. (2016). Deep Learning: A Practitioner's Approach. O'Reilly Media.
- Nielsen, M. A. (2015). Neural Networks and Deep Learning. Determination Press.
- Theodoridis, S. (2015). "Chapter 18: Neural Networks and Deep Learning". Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
- Odense, S. (2015). Universal approximation theory of neural networks. MSc Diss. University of Victoria.
- Du, K. L., & Swamy, M. N. (2014). Neural networks and statistical learning. Springer Science & Business Media.
- Deng, L., & Yu, D. (2014). Deep Learning. Foundations and Trends in Signal Processing, 7, 3-4.
- Bengio, Y., & Courville, A. (2013). Deep Learning of Representations. Springer.
- Barber, D. (2012). "Chapter 26: Distributed Computation". Bayesian Reasoning and Machine Learning. Cambridge University Press.
- Neal, R. M. (2012). Bayesian learning for neural networks. Springer Science & Business Media.
- Orr, G. B., & Muller, K. R. (2012). Neural Networks: Tricks of the Trade. Springer.
- Murphy, K. P. (2012). "Chapter 28: Deep Learning". Machine Learning: A Probabilistic Perspective. MIT Press.
- Alpaydin, E. (2010). "Chapter 11: Multilayer Perceptrons". Introduction to Machine Learning. MIT Press.
- Haykin, S. S., Haykin, S. S., Haykin, S. S., & Haykin, S. S. (2009). Neural Networks and Learning Machines. 3rd Ed. Pearson.
- Bengio, Y. (2009). Learning Deep Architectures for AI. Foundations and Trends® in Machine Learning, 2(1), 1-127. Now Publishers.
- LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., & Huang, F. (2006). "A Tutorial on Energy-Based Learning". Predicting Structured Data. MIT Press.
- Bishop, C. M. (2006). "Chapter 5: Neural Networks". Pattern Recognition and Machine Learning. Springer.
- MacKay, D. J. (2003). "Chapter 38: Introduction to Neural Networks" Information Theory, Inference and Learning Algorithms. Cambridge University Press.
- Mandic, D. P., & Chambers, J. (2001). Recurrent neural networks for prediction: learning algorithms, architectures and stability. John Wiley & Sons.
- Rojas, R. (1996). Neural networks: a systematic introduction. Springer Science & Business Media. (link)
- Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford University Press.
- Hassoun, M. H. (1995). Fundamentals of artificial neural networks. MIT press.
Scholarly Articles[edit]
See Reading List and Recommended Readings for the complete list.
- Shwartz-Ziv, R., & Tishby, N. (2017). Opening the Black Box of Deep Neural Networks via Information. arXiv preprint arXiv:1703.00810.
- Chakraborty, S., Tomsett, R., Raghavendra, R., Harborne, D., Alzantot, M., Cerutti, F., ... & Kelley, T. D. (2017). Interpretability of Deep Learning Models: A Survey of Results.
- Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural networks, Volume 61, 85-117.
- Paul, A., & Venkatasubramanian, S. (2014). Why does Deep Learning work?-A perspective from Group Theory. arXiv preprint arXiv:1412.6621.
- Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798-1828.
Tutorials[edit]
- Information Theory of Deep Learning by Prof Naftali Tishby
- UFLDL Tutorial
- A Deep Learning Tutorial: From Perceptrons to Deep Networks - with Java examples
- Neural Networks, Manifolds, and Topology - advanced
- Deep Learning Tutorials by Olver Dürr - Lasagne and TensorFlow
- Efficient Deep Learning with Humans in the Loop by Zachary Chase Lipton
- Deep Generative Models by Durk Kingma (2014)
- Learning Invariant Feature Hierarchies (2013) and its Panel Discussion
- Deep Learning Tutorial (ICML 2013)
- Deep Learning for Computer Vision (NIPS 2013)
- Deep Learning for NLP (NAACL 2013)
- Deep Support Vector Machines (ROKS 2013)
- Deep Learning of Representaions (SSTiC 2013)
- Deep Learning for Machine Vision (BMVC 2013)
- Deep Learning for Computer Vision (NIPS 2013) (Video )
- Deep Learning Methods for Vision (CVPR 2012)
- Deep Learning for NLP (ACL 2012)
- Representation Learning (ICML 2012)
- Classification with Deep Invariant Scattering Networks (NIPS 2012)
- A tutorial on deep and unsupervised feature learning for activity recognition (2011)
- Tutorial on Deep Learning and Applications (NIPS 2010)
- A tutorial on Deep Learning (2009)
- Tutorial on Learning Deep Architectures (ICML 2009)
- Learning Deep Hierarchies of Representations (2009)
- Deep Learning with Multiplicative Interactions (NIPS 2009)
- Learning Feature Hierarchies (MLSS 2009)
- Deep Belief Networks (MLSS 2009)
Software[edit]
See Software Links for the complete list.
- TensorFlow - Python
- Theano - Python (discontinued; no longer supported or developed)
- Keras - Python library for TensorFlow and Theano
- Lasagne - Lightweight library to build and train neural networks in Theano
- Caffe - C/C++, Python, MATLAB, command line
- Torch - Lua
- PyTorch - Python and C++
- fastai - Python
- pydlt - PyTorch based Deep Learning Toolbox
- CNTK - The Microsoft Cognitive Toolkit
- ONNX
- cuDNN - CUDA GPU library supporting TensorFlow, Theano, Torch, Caffe, Keras, CNTK and others
- TFLean - Deep Learning Python library featuring a higher-level API for TensorFlow
- Blocks - A Theano framework for training neural networks
- Pylearn2 - Python
- deeplearning4j - Java
- MXNet - MATLAB, Python, C++, R, Julia, Scala, Go, Javascript and more
- Chainer - Python
- CudaCnn - MATLAB
- hebel - Python
- ConvNetJS - Deep Learning models (mainly Neural Networks) entirely in your browser
- OpenNN - C++
- visual-rbm
- MatCovNet - MATLAB
- Learning Deep Boltzmann Machines - MATLAB
- Estimating Partition Functions of RBM's - MATLAB
- Deep Belief Networks - MATLAB
- DeepLearnToolbox - MATLAB/Octave
- Netlab neural network software - MATLAB
- Neural Network Toolbox - MATLAB
- PyBrain - Python
- DyNet - Python and C++
- handson-ml - Python
- DLL - C++
- DeepRosetta - Python
- torchgeometry - Python
- distiller - Python
See also[edit]
Other Resources[edit]
- Deep Learning with Python by Jason Brownlee - practical book
- Deep Learning Reading List
- DeepLearning.Net - Tutorials and a general point of reference
- Toronto Deep Learning Demos - source code
- Deep learning from the bottom up - Metacademy
- Neural Networks and Deep Learning - free online book
- Awesome-Deep-Vision (Github) - A curated list of Deep Learning resources for Computer Vision
- Awesome-Deep-Learning (Github) - A curated list of resources
- Awesome-Deep-Learning-papers (Github)
- Awesome-TensoFflow (Github)
- Deep Learning Libraries by Language
- Deep Learning (Building Intelligent Probabilistic Systems) - Blog by Harvard University
- Benchmark of Deep Learning Representations for Visual Recognition
- Deep Learning Playlist - Youtube collection of video lectures and tutorials
- Deep Learning on Google+ - online community
- A Short History of and Introduction to Deep Learning - Presentation by John Kaufhold
- An Introduction to Deep Learning: From Perceptrons to Deep Networks - tutorial with Java examples
- Graduate Summer School: Deep Learning, Feature Learning by IPAM, UCLA
- What Does a Neural Network Actually Do? - Neural Networks and Deep Learning
- Ersatz - Deep Neural Networks in the cloud
- Bibliography in Deep Learning - collection of papers categorized according to type of application
- Deep Learning Papers Reading Roadmap
- What My Deep Model Doesn't Know - Blog post on Bayesian Neural Networks
- New Theory Cracks Open the Black Box of Deep Learning
- Why does Deep Learning work? by Charles H Martin - blog post
- Why Deep Learning Works II: The Renormalization Group by Charles H Martin - blog post
- Practical_DL - Github
- deep-learning-book - Github
- Deep Learning: Theory & Practice - blog post
- From Topological Data Analysis to Deep Learning: No Pain No Gain - blog post
- Deep learning: the final frontier for signal processing and time series analysis? - blog post