Regularization

From Ioannis Kourouklides

This page contains resources about Regularization, Overfitting and Bias-Variance tradeoff.

Subfields and Concepts[edit]

  • Shrinkage Penalty / Regularization Term
  • Regularization
    • Regularized least squares
    • L0 penalization / Spike-and-slab prior
    • L0-regularization
    • L1-regularization / Least absolute shrinkage and selection operator (LASSO) / Laplace prior
    • L2-regularization / Ridge Regression / Tikhonov Regularization / Gaussian prior
    • Lp-regularization (where p is a positive real number)
    • Max norm constraints
    • Early Stopping (in epochs during training of Artificial Neural Networks)
    • Mini-Batches (in the training of Artificial Neural Networks)
    • Total Variation (TV) Regularization (i.e. L1-norm of the gradient)
    • Dropout
    • Matrix Regularization
    • Elastic Nets

Online courses[edit]

Video Lectures[edit]


Lecture Notes[edit]

Books[edit]

  • Ito, K., & Jin, B. (2014). Inverse Problems: Tikhonov Theory and Algorithms. World Scientific.
  • Engl, H. W., Hanke, M., & Neubauer, A. (1996). Regularization of Inverse Problems. Springer Science & Business Media.

Scholarly Articles[edit]

  • Starck, J. L., & Fadili, M. J. (2009). An overview of inverse problem regularization using sparsity. In Image Processing (ICIP), 16th IEEE International Conference on, 1453-1456.

Software[edit]

See also[edit]

Other Resources[edit]

Retrieved from "https://wiki.kourouklides.com/w/index.php?title=Regularization&oldid=1017"