K-Means Clustering in Python

K-Means is the most famous unsupervised clustering algorithm in scikit-learn. Unlike classification or regression, clustering has no labeled y to learn from — you only give it feature data X, and the algorithm groups similar rows together on its own. You tell it how many groups you want with the n_clusters parameter (K), and it finds them.

Creating a K-Means model looks just like any other scikit-learn estimator: KMeans(n_clusters=3) builds an unfitted model, and calling .fit(X) runs the clustering procedure. There's no y_train in this call, which is the biggest structural difference from the supervised algorithms covered earlier in this course — clustering discovers structure in the data rather than predicting a known target.

Common real-world uses include customer segmentation (grouping shoppers by purchasing behavior), image compression (grouping similar pixel colors), and anomaly detection (points that don't fit cleanly into any cluster). The rest of this lesson covers how K-Means actually finds those groups and where it can go wrong.

K-Means works through a simple, iterative loop. First, it drops K random 'centroids' — candidate center points — into the feature space. Every data point is then assigned to whichever centroid is closest to it, forming K temporary clusters.

Next, each centroid is recalculated as the mean position of all the points currently assigned to it, so it physically moves toward the middle of its cluster. Points are then reassigned based on the new centroid positions, which can shift some points into a different cluster than before.

This assign-then-recompute cycle repeats — model.fit(X) runs it automatically — until the centroids stop moving significantly between iterations, meaning the clusters have stabilized. This is why K-Means is sometimes called Lloyd's algorithm, and why the final result can depend on where the centroids happened to start.

In the K-Means algorithm, a centroid is the physical center point of a specific cluster — not a data point that necessarily exists in the original dataset, but a computed average position of every point currently assigned to that cluster. Each of the K clusters has exactly one centroid.

Membership works in one direction: a data point belongs to whichever centroid is closest to it, measured by Euclidean distance. After training, model.cluster_centers_ holds the final coordinates of all K centroids, and model.labels_ holds the cluster assignment (0 through K-1) for every row in the training data.

Understanding centroids as 'the average location of a group' rather than 'a real data point' matters when interpreting results — a centroid representing 'average customer spending' might land on a value no actual customer has, and that's expected behavior, not a bug.

The biggest flaw of K-Means is that you have to choose the value of K before the algorithm runs — it can't figure out the 'right' number of clusters on its own. If your data naturally has 5 groups but you set n_clusters=2, K-Means will not tell you it's wrong; it will silently force the data into 2 clusters that don't reflect the real structure.

This matters because, unlike a classification target, there's usually no ground truth for 'the correct number of clusters' — customer segments, for instance, don't come pre-labeled. Picking K carelessly (or just defaulting to K=3) can produce clusters that are meaningless for the business question you're trying to answer.

This is a fundamentally different kind of decision from anything in supervised learning: instead of the model learning a parameter from data, K is a hyperparameter the practitioner must choose deliberately, typically with the help of a technique like the Elbow Method covered next.

The primary limitation of K-Means is that you must manually specify the number of clusters (K) before running the algorithm, and in real-world data the true number of natural groups is usually unknown ahead of time. Guess too low and distinct groups get merged together; guess too high and a single real group gets artificially split in two.

K-Means has other well-known limitations worth knowing too: it assumes clusters are roughly spherical and similarly sized, so it struggles with elongated or unevenly-sized groups; it's sensitive to the initial random centroid placement, which is why scikit-learn's default n_init runs the algorithm multiple times with different starting points and keeps the best result; and it requires numeric, scaled features because it relies entirely on distance calculations.

None of these limitations make K-Means a bad algorithm — it's fast, simple, and works well on many real datasets — but they explain why it's paired with diagnostic techniques like the Elbow Method rather than trusted blindly.

To find a reasonable K, we run K-Means multiple times with increasing values (K=1, K=2, K=3...) and plot each model's 'inertia' — the sum of squared distances from every point to its assigned centroid. Inertia measures how tightly packed the clusters are, and it's available directly as model.inertia_ after fitting.

Inertia always decreases as K increases (more clusters means points are, on average, closer to their nearest centroid — taken to the extreme, K equal to the number of data points gives zero inertia). So the goal isn't to minimize inertia outright; it's to find where adding more clusters stops producing a meaningful improvement.

Plotted against K, inertia typically forms a curve that drops steeply at first and then flattens out, resembling a bent arm — which is exactly where the technique gets its name.

The Elbow Method is a visual technique for choosing K: you plot inertia against K for a range of values (say K=1 through K=10) and look for the 'elbow' — the point where the curve bends and adding more clusters starts producing only marginal drops in inertia instead of steep ones.

That bend represents diminishing returns: below the elbow, each additional cluster meaningfully tightens the groupings; past it, you're mostly just splitting already-coherent clusters into smaller, less interpretable pieces without gaining much. The elbow is a judgment call read off a chart, not an exact computed value, which is why it's called a heuristic rather than an optimization.

In practice, teams often combine the Elbow Method with domain knowledge (e.g. 'we know we want 4 customer tiers') or a complementary metric like the silhouette score, which measures how well-separated clusters are rather than just how tight they are internally.

Choosing K well doesn't help if the distance calculations underneath K-Means are themselves distorted. This section shifts focus from 'how many clusters' to a separate, equally important issue: how K-Means measures 'closeness' between points, and what happens when your features aren't on comparable scales.

Because K-Means assigns every point to its nearest centroid using raw Euclidean distance, any feature with a naturally larger numeric range will dominate that distance calculation — regardless of whether it's actually the most important feature for the clustering you're trying to achieve.

This is one of the most common ways K-Means silently produces nonsensical clusters in practice, and it's fixed with a single preprocessing step you should treat as mandatory whenever your features have different units or ranges.

K-Means calculates literal geometric distance — Euclidean distance — between every point and every centroid. In two dimensions this is the straight-line distance formula from geometry class, generalized to however many features your dataset has: sqrt((x1-c1)^2 + (x2-c2)^2 + ...).

The critical consequence is that this formula treats one unit of any feature as equally significant to one unit of any other feature. If one column is 'Age' (ranging roughly 0-100) and another is 'Salary' (ranging roughly 0-150,000), a $10,000 difference in salary moves the calculated distance far more than a 10-year difference in age — not because salary matters more to the clustering, but purely because its numbers are bigger.

The result is that K-Means effectively ignores small-range features and clusters almost entirely on whichever feature happens to have the largest raw numeric spread, producing groupings that have nothing to do with the actual similarity you were trying to capture.

Consider a dataset with 'Age' (0 to 100) and 'Salary' (0 to 150,000). Run K-Means directly on the raw values and the clusters make no sense — they group almost entirely by salary, with age barely influencing the result. The missing step is StandardScaler.

StandardScaler transforms every feature to have a mean of 0 and a standard deviation of 1, putting Age and Salary on the exact same numeric footing before distance is calculated. After scaling, a meaningful shift in age contributes to the distance calculation just as much as a meaningful shift in salary, so both features actually influence which cluster a point ends up in.

The fix is a Pipeline step, not an afterthought: Pipeline([('scaler', StandardScaler()), ('kmeans', KMeans(n_clusters=3))]). This same rule — scale your features first — applies to every distance-based algorithm in scikit-learn, including KNN, SVM, and PCA.

This wraps up K-Means clustering: how the centroid assign-and-recompute loop finds groups, why K must be chosen manually (with the Elbow Method as a practical guide), and why scaling features with StandardScaler before fitting is non-negotiable since the algorithm relies entirely on distance.

K-Means is a strong default for clustering because it's fast and easy to reason about, but it isn't the only option — it struggles with clusters that aren't roughly round or similarly sized, in which case algorithms like DBSCAN or hierarchical clustering are often a better fit.

With clustering covered, the natural next step is dimensionality reduction: real datasets often have far more features than can be visualized or clustered meaningfully, and techniques like PCA compress that feature space down to its most informative dimensions before feeding it into algorithms like K-Means.

K-Means is the most famous Unsupervised Clustering algorithm. You tell it how many groups you want (K), and it finds them.

It works by dropping K random "Centroids" (center points) into the data. Points snap to the closest Centroid. Then, the Centroids move to the middle of their points.

In the K-Means algorithm, what exactly does a "Centroid" represent?

• →A deep learning neural network node.
• →The physical center point of a specific cluster. All data points belong to whichever Centroid they are closest to.
• →A row of data that has been deleted.

The biggest flaw of K-Means is that YOU have to guess the value of K before it runs. If your data naturally has 5 groups, but you say K=2, it will force the data into 2 groups.

What is the primary limitation or drawback of the K-Means algorithm?

• →It takes weeks to train on small datasets.
• →You must manually specify the number of clusters (K) before running the algorithm, which is often unknown in real-world data.
• →It only works on images, not text.

To find the optimal K, we run K-Means multiple times (K=1, K=2, K=3...) and plot the "Inertia" (how tightly packed the clusters are). The graph looks like an arm. The "Elbow" is the best K.

What is the "Elbow Method" used for in K-Means clustering?

• →To increase the training speed of the model.
• →It is a visual technique used to determine the optimal number of clusters (K) by looking for the point where adding more clusters yields diminishing returns.
• →To curve straight regression lines into elbows.

Now, prepare yourself. We are about to enter the ADA Defense Protocol. Ensure you understand distance sensitivity.

K-Means calculates literal geometric distance (Euclidean distance) between points and Centroids.

ADA DEFENSE: Your dataset has "Age" (0 to 100) and "Salary" (0 to 150,000). You run K-Means and the clusters make zero sense; they only group by Salary. What critical step did you forget?

• →You forgot to set K to 100.
• →You forgot to use StandardScaler. K-Means relies entirely on distance. If Salary is unscaled, it mathematically overpowers Age, ruining the clusters.
• →You forgot to import the testing data.

Threat neutralized. Scaling confirmed. Proceeding to Dimensionality Reduction.

Cluster Real Unlabeled Data. Finish cluster_and_count(): KMeans always produces exactly k clusters.