This wiki has no edits or logs made within the last 45 days, therefore it is marked as inactive. If you would like to prevent this wiki from being closed, please start showing signs of activity here. If there are no signs of this wiki being used within the next 15 days, this wiki may be closed per the Dormancy Policy. This wiki will then be eligible for adoption by another user. If not adopted and still inactive 135 days from now, this wiki will become eligible for deletion. Please be sure to familiarize yourself with Miraheze's Dormancy Policy. If you are a bureaucrat, you can go to Special:ManageWiki and uncheck "inactive" yourself. If you have any other questions or concerns, please don't hesitate to ask at Stewards' noticeboard.

Deep Learning

From Ioannis Kourouklides
Jump to navigation Jump to search

This page contains resources about Deep Learning and Representation Learning .

Subfields and Concepts[edit]

  • Deep Generative Models
    • Deep Directed Networks (directed graphical models)
      • Sigmoid Belief Net
      • Differentiable Generator Net
      • Variational Autoencoder (VAE)
      • Generative Adversarial Network (GAN)
      • Generative Moment Matching Network
      • Convolutional Generative Network
      • Auto-Regressive Network / Fully-visible Bayes Network (FVBN)
      • Deep Latent Gaussian Model (DLGM)
      • Deep AutoRegressive Network (DARN)
    • Deep Boltzmann Machines (undirected graphical models)
    • Deep Belief Networks (mixed graphs)
  • Deep Neural Networks (i.e. more than two hidden layers)
    • Deep Multi-Layer Perceptron (i.e. Stacked RBMs) 
    • Deep Autoencoders (i.e. two symmetrical DBN)
      • DARN 
    • Deep Neural Decision Forests 
    • Convolutional Deep Belief Network (i.e. Stacked CRBMs)  
  • Sparse Coding / Dictionary Learning
    • Sparse Autoencoders
    • Stacked Denoising Autoencoders
  • Bayesian Deep Learning
    • Bayesian Neural Networks

Online Courses[edit]

Video Lectures[edit]

Lecture Notes[edit]

Books and Book Chapters[edit]

  • Santana, E. (2018). Eder Santana's Deep Learning with Python. Packt Publishing.
  • Shukla, N. (2018). Machine learning with TensorFlow. Manning.
  • Zaccone, G., Karim, Md. R., & Menshawy, A. (2017). Deep Learning with TensorFlow. Packt Publishing.
  • McClure, N. (2017). TensorFlow Machine Learning Cookbook. Packt Publishing.
  • Gulli, A., & Pal, S. (2017). Deep Learning with Keras. Packt Publishing.
  • Chollet, F. (2017). Deep Learning with Python. Manning Publications.
  • Gulli, A., & Kapoor, A. (2017). TensorFlow 1.x Deep Learning Cookbook. Packt Publishing.
  • Bengio, Y., Goodfellow, I. J., & Courville, A. (2016). Deep Learning. MIT Press. (link)
  • Gibson, A., & Patterson J. (2016). Deep Learning: A Practitioner's Approach. O'Reilly Media.
  • Nielsen, M. A. (2015). Neural Networks and Deep Learning. Determination Press.
  • Theodoridis, S. (2015). "Chapter 18: Neural Networks and Deep Learning". Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
  • Odense, S. (2015). Universal approximation theory of neural networks. MSc Diss. University of Victoria.
  • Du, K. L., & Swamy, M. N. (2014). Neural networks and statistical learning. Springer Science & Business Media.
  • Deng, L., & Yu, D. (2014). Deep Learning. Foundations and Trends in Signal Processing7, 3-4.
  • Bengio, Y.,  & Courville, A. (2013). Deep Learning of Representations. Springer.
  • Barber, D. (2012). "Chapter 26: Distributed Computation". Bayesian Reasoning and Machine Learning. Cambridge University Press.
  • Neal, R. M. (2012). Bayesian learning for neural networks. Springer Science & Business Media.
  • Orr, G. B., & Muller, K. R. (2012). Neural Networks: Tricks of the Trade. Springer.
  • Murphy, K. P. (2012). "Chapter 28: Deep Learning". Machine Learning: A Probabilistic Perspective. MIT Press.
  • Alpaydin, E. (2010). "Chapter 11: Multilayer Perceptrons". Introduction to Machine Learning. MIT Press.
  • Haykin, S. S., Haykin, S. S., Haykin, S. S., & Haykin, S. S. (2009). Neural Networks and Learning Machines. 3rd Ed. Pearson.
  • Bengio, Y. (2009). Learning Deep Architectures for AI. Foundations and Trends® in Machine Learning, 2(1), 1-127. Now Publishers.
  • LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., & Huang, F. (2006). "A Tutorial on Energy-Based Learning". Predicting Structured Data. MIT Press.
  • Bishop, C. M. (2006). "Chapter 5: Neural Networks". Pattern Recognition and Machine Learning. Springer.
  • MacKay, D. J. (2003). "Chapter 38: Introduction to Neural Networks" Information Theory, Inference and Learning Algorithms. Cambridge University Press.
  • Mandic, D. P., & Chambers, J. (2001). Recurrent neural networks for prediction: learning algorithms, architectures and stability. John Wiley & Sons.
  • Rojas, R. (1996). Neural networks: a systematic introduction. Springer Science & Business Media. (link)
  • Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford University Press.
  • Hassoun, M. H. (1995). Fundamentals of artificial neural networks. MIT press.

Scholarly Articles[edit]

See Reading List and Recommended Readings for the complete list.

  • Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., & Tang, J. (2021). Self-supervised Learning: Generative or Contrastive. IEEE Transactions on Knowledge and Data Engineering.
  • Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., ... & Sun, M. (2020). Graph Neural Networks: A Review of Methods and Applications. AI Open, 1, 57-81.
  • Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Philip, S. Y. (2020). A Comprehensive Survey on Graph Neural Networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1), 4-24.
  • Shwartz-Ziv, R., & Tishby, N. (2017). Opening the Black Box of Deep Neural Networks via Information. arXiv preprint arXiv:1703.00810.
  • Chakraborty, S., Tomsett, R., Raghavendra, R., Harborne, D., Alzantot, M., Cerutti, F., ... & Kelley, T. D. (2017). Interpretability of Deep Learning Models: A Survey of Results.
  • Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural networks, Volume 61, 85-117.
  • Paul, A., & Venkatasubramanian, S. (2014). Why does Deep Learning work?-A perspective from Group Theory. arXiv preprint arXiv:1412.6621.
  • Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798-1828.



See Software Links for the complete list.

See also[edit]

Other Resources[edit]