Clustering

This page contains resources about Clustering, Clustering Analysis, Data Clustering and Discrete Latent Variable Models.

Subfields and Concepts

• Hierarchical clustering / Connectivity-based clustering
  • Agglomerative (i.e. bottom-up approach)
    • Minimum Spanning Tree (MST) Clustering Algorithm
    • Single-Linkage Clustering
    • Complete-Linkage Clustering
    • Average-Linkage Clustering
    • Mean or Average Linkage Clustering / UPGMA
    • Centroid Linkage Clustering / UPGMC
    • Minimum Energy Clustering
    • Nearest Neighbour Clustering / Mutual Neighbourhood Clustering
    • Ward's Method
  • Divisive (i.e. top-down approach)
    • Divisive Analysis Clustering (DIANA)
    • Monothetic Analysis (MONA)
• Partitional Clustering
  • Centroid-based Clustering
    • K-means Algorithm / Lloyd's Algorithm
    • Soft K-means Algorithm 
    • Fuzzy c-means
    • K-SVD (used in Dictionary Learning)
  • Distribution-based Clustering
    • Mixture Models and EM Algorithm / Clustering by Mixture Decomposition
  • Density-based Clustering
    • DBSCAN
    • OPTICS
    • Mean-shift
  • Graph Theory-based Clustering
  • Search Techniques-based Clustering Algorithms
• Kernel-based Clustering
• Sequential Data Clustering
• Discrete Latent Variable Models
• Deterministic Annealing
  • Latent Dirichlet Allocation
  • Hidden Markov Models
  • Mixture Models
    • Bayesian Model
    • Non-Bayesian Model

Books and Book Chapters

• Duda, R. O., Hart, P. E., & Stork, D. G. (2012). "Chapter 10: Unsupervised Learning and Clustering". Pattern Classification. John Wiley & Sons.
• Everitt, B. S., Landau, S., Leese, M., & Stahl, D. (2011). Cluster Analysis. 5th Ed. John Wiley & Sons.
• Alpaydin, E. (2010). "Chapter 7: Clustering". Introduction to machine learning. MIT Press.
• Xu, R., & Wunsch, D. (2008). Clustering. John Wiley & Sons.
• Theodoridis, S., Pikrakis, A., Koutroumbas, K., & Cavouras, D. (2008). Pattern Recognition. 4th Ed. Academic Press.
• Kaufman, L., & Rousseeuw, P. J. (2005). Finding groups in data: an introduction to cluster analysis. John Wiley & Sons.
• MacKay, D. J. (2003). "Chapter 20: Example Inference Task: Clustering". Information Theory, Inference and Learning Algorithms. Cambridge University Press.

Scholarly Articles

• Xu, R., & Wunsch, D. (2005). Survey of clustering algorithms. IEEE Transactions on neural networks, 16(3), 645-678.
• Jain, A. K., Duin, R. P. W., & Mao, J. (2000). Statistical pattern recognition: A review. IEEE Transactions on pattern analysis and machine intelligence, 22(1), 4-37.
• Roberts, S. J. (1997). Parametric and non-parametric unsupervised cluster analysis. Pattern Recognition, 30(2), 261-272.

Software

• Clustering Analysis (Statistics and Machine Learning Toolbox) - MATLAB
• Clustering (scikit-learn) - Python

See also

• Continuous Latent Variable Models / Dimensionality Reduction, an other set of methods for Unsupervised Learning
• Bayesian Nonparametrics
• Probabilistic Graphical Models

Other Resources

• Clustering - Notebook
• Clustering - Zoubin Ghahramani
• The 5 Clustering Algorithms Data Scientists Need to Know - blog post
• DP_means - GitHub