Lecture 11: K-Means Clustering
Finding Groups in Unsupervised Learning
Overview
K-means finds compact groups in a chosen representation. We trace its assignment and mean updates, compare starting points, and ask what cohesion, separation, silhouette and ARI actually measure. A real document example shows why a convincing cluster can reflect repeated wording rather than the topic we hoped to discover.
Learning Objectives
By the end of this lecture, you will:
- Understand the fundamental concepts of clustering and unsupervised learning
- Master the K-Means algorithm, including its objective function (WCSS) and iterative optimization
- Implement K-Means from scratch to understand the assignment and update steps
- Apply K-Means to text data using TF-IDF transformation
- Distinguish geometric quality (silhouette) from agreement with reference labels (ARI)
- Use cost, silhouette and inspection to evaluate choices of K without assuming a decisive elbow
- Explain and test K-Means++ initialization
- Recognize K-Means limitations (spherical clusters, fixed K) and when to use alternative methods
Materials
TipQuick Access
Datasets & Acknowledgments
- 20 Newsgroups (Ken Lang): Text documents from 20 different newsgroups used to demonstrate document clustering
- Libraries: scikit-learn (K-Means implementation, metrics), NumPy (numerical operations), Matplotlib/Seaborn (visualization)
Previous: ← Lecture 10: Kernel Methods & Gaussian Processes | Next: Lecture 12: PCA and Dimensionality Reduction →