Open main menu

Deep Learning

This page contains resources about Deep Learning and Representation Learning .


Subfields and ConceptsEdit

  • Deep Generative Models
    • Deep Directed Networks (directed graphical models)
      • Sigmoid Belief Net
      • Differentiable Generator Net
      • Variational Autoencoder (VAE)
      • Generative Adversarial Network (GAN)
      • Generative Moment Matching Network
      • Convolutional Generative Network
      • Auto-Regressive Network / Fully-visible Bayes Network (FVBN)
      • Deep Latent Gaussian Model (DLGM)
      • Deep AutoRegressive Network (DARN)
    • Deep Boltzmann Machines (undirected graphical models)
    • Deep Belief Networks (mixed graphs)
  • Deep Neural Networks (i.e. more than two hidden layers)
    • Deep Multi-Layer Perceptron (i.e. Stacked RBMs) 
    • Deep Autoencoders (i.e. two symmetrical DBN)
      • DARN 
    • Deep Neural Decision Forests 
    • Convolutional Deep Belief Network (i.e. Stacked CRBMs)  
  • Sparse Coding / Dictionary Learning
    • Sparse Autoencoders
    • Stacked Denoising Autoencoders
  • Bayesian Deep Learning
    • Bayesian Neural Networks

Online CoursesEdit

Books and Book ChaptersEdit

  • Santana, E. (2018). Eder Santana's Deep Learning with Python. Packt Publishing.
  • Shukla, N. (2018). Machine learning with TensorFlow. Manning.
  • Zaccone, G., Karim, Md. R., & Menshawy, A. (2017). Deep Learning with TensorFlow. Packt Publishing.
  • McClure, N. (2017). TensorFlow Machine Learning Cookbook. Packt Publishing.
  • Gulli, A., & Pal, S. (2017). Deep Learning with Keras. Packt Publishing.
  • Chollet, F. (2017). Deep Learning with Python. Manning Publications.
  • Gulli, A., & Kapoor, A. (2017). TensorFlow 1.x Deep Learning Cookbook. Packt Publishing.
  • Bengio, Y., Goodfellow, I. J., & Courville, A. (2016). Deep Learning. MIT Press. (link)
  • Gibson, A., & Patterson J. (2016). Deep Learning: A Practitioner's Approach. O'Reilly Media.
  • Nielsen, M. A. (2015). Neural Networks and Deep Learning. Determination Press.
  • Theodoridis, S. (2015). "Chapter 18: Neural Networks and Deep Learning". Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
  • Odense, S. (2015). Universal approximation theory of neural networks. MSc Diss. University of Victoria.
  • Du, K. L., & Swamy, M. N. (2014). Neural networks and statistical learning. Springer Science & Business Media.
  • Deng, L., & Yu, D. (2014). Deep Learning. Foundations and Trends in Signal Processing7, 3-4.
  • Bengio, Y.,  & Courville, A. (2013). Deep Learning of Representations. Springer.
  • Barber, D. (2012). "Chapter 26: Distributed Computation". Bayesian Reasoning and Machine Learning. Cambridge University Press.
  • Neal, R. M. (2012). Bayesian learning for neural networks. Springer Science & Business Media.
  • Orr, G. B., & Muller, K. R. (2012). Neural Networks: Tricks of the Trade. Springer.
  • Murphy, K. P. (2012). "Chapter 28: Deep Learning". Machine Learning: A Probabilistic Perspective. MIT Press.
  • Alpaydin, E. (2010). "Chapter 11: Multilayer Perceptrons". Introduction to Machine Learning. MIT Press.
  • Haykin, S. S., Haykin, S. S., Haykin, S. S., & Haykin, S. S. (2009). Neural Networks and Learning Machines. 3rd Ed. Pearson.
  • Bengio, Y. (2009). Learning Deep Architectures for AI. Foundations and Trends® in Machine Learning, 2(1), 1-127. Now Publishers.
  • LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., & Huang, F. (2006). "A Tutorial on Energy-Based Learning". Predicting Structured Data. MIT Press.
  • Bishop, C. M. (2006). "Chapter 5: Neural Networks". Pattern Recognition and Machine Learning. Springer.
  • MacKay, D. J. (2003). "Chapter 38: Introduction to Neural Networks" Information Theory, Inference and Learning Algorithms. Cambridge University Press.
  • Mandic, D. P., & Chambers, J. (2001). Recurrent neural networks for prediction: learning algorithms, architectures and stability. John Wiley & Sons.
  • Rojas, R. (1996). Neural networks: a systematic introduction. Springer Science & Business Media. (link)
  • Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford University Press.
  • Hassoun, M. H. (1995). Fundamentals of artificial neural networks. MIT press.

Scholarly ArticlesEdit

See Reading List and Recommended Readings for the complete list.

  • Shwartz-Ziv, R., & Tishby, N. (2017). Opening the Black Box of Deep Neural Networks via Information. arXiv preprint arXiv:1703.00810.
  • Chakraborty, S., Tomsett, R., Raghavendra, R., Harborne, D., Alzantot, M., Cerutti, F., ... & Kelley, T. D. (2017). Interpretability of Deep Learning Models: A Survey of Results.
  • Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural networks, Volume 61, 85-117.
  • Paul, A., & Venkatasubramanian, S. (2014). Why does Deep Learning work?-A perspective from Group Theory. arXiv preprint arXiv:1412.6621.
  • Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798-1828.



See Software Links for the complete list.

See alsoEdit

Other ResourcesEdit