Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 36 ยท Applications

Fuzzy C-Means Clustering from Scratch

Learn fuzzy c-means clustering: soft membership, the fuzzifier m, the alternating update rules, choosing the number of clusters, and a numpy implementation you can run.

K-means clustering gives every point to exactly one cluster. That is a poor fit when groups overlap: a customer who buys like both a "bargain hunter" and a "brand loyalist" must be forced into one of the two. Fuzzy c-means (FCM) uses the idea of this whole course, partial membership, to cluster data: every point belongs to every cluster, with a degree between 0 and 1, and the degrees of one point sum to 1.

This is one of the most common uses of fuzzy logic in data analysis, in image segmentation, customer grouping and signal processing. We will implement it from scratch with numpy in about twenty lines.

The idea

FCM looks for c cluster centers and a membership matrix U, where U[i, k] is how strongly point k belongs to cluster i. It minimizes

TEXT
J = sum over i, k of  U[i,k]^m * distance(x_k, v_i)^2

where v_i is the center of cluster i and m > 1 is the fuzzifier. Points far from a center get small membership in it, which keeps J small.

The algorithm alternates two steps until the centers stop moving:

  1. Update the centers as weighted averages: v_i = sum_k (U[i,k]^m * x_k) / sum_k U[i,k]^m.
  2. Update the memberships from distances: U[i,k] = 1 / sum over j of (d_ik / d_jk)^(2/(m-1)), where d_ik is the distance from point k to center i.

Intuition for step 2: the closer a point is to center i compared with the other centers, the larger its membership in i. If a point is exactly halfway between two centers, it is 0.5 and 0.5.

Fuzzy c-means

Start Random memberships
Centers Weighted averages
Memberships From distances
Repeat Until stable

Each point belongs to all clusters

1 / 2

Implementation

We generate two overlapping groups of points with a fixed random seed, run FCM and print the centers and the memberships of a few points. The data are made up, so the numbers are deterministic.

Python Fuzzy c-means in numpy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The centers land near (2, 2) and (5, 4), where the groups were created. Points deep inside a group have memberships close to 1 and 0. A point between the groups has a split membership, for example 0.6 and 0.4, and that is the information k-means throws away.

The fuzzifier m

The exponent m controls how soft the clustering is:

  • m close to 1: almost hard clustering (memberships near 0 or 1).
  • m = 2: the common default.
  • Large m: memberships flatten out toward 1/c for every point.
Python Effect of the fuzzifier on one borderline point

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The point is nearer to the first center, so its membership there is always larger. With m = 1.1 it is nearly 1, with m = 6 it sinks toward 0.5.

Choosing the number of clusters

FCM needs c in advance. A common check is the partition coefficient, PC = (1/N) * sum(U^2). It equals 1 for completely crisp partitions and 1/c for completely fuzzy ones. Higher is "crisper". You compute it for several c and look for a good value, but treat it as a hint rather than an answer, because it tends to favor small c.

Python Partition coefficient for c = 2, 3, 4

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Three well-separated groups were generated, and c=3 has the highest score (0.866), as it should. The score is not meaningful by itself: compare it with the lower bound 1/c and with the plot or domain knowledge.

Where FCM is used

  • Image segmentation: pixels in a brain scan on the boundary between two tissues get partial membership instead of an arbitrary label.
  • Customer segmentation: a customer can be 60% "frequent small buyer" and 40% "occasional big buyer".
  • Fuzzy rule generation: cluster centers become the centers of membership functions, a first step toward neuro-fuzzy systems.

Pitfalls

  • FCM finds a local minimum; run it with several random starts and keep the lowest J.
  • It assumes roughly round clusters (Euclidean distance); scale your features first.
  • It is sensitive to outliers, which get memberships too. Variants (possibilistic c-means) address this.
  • Memberships summing to 1 means a point equally far from everything cannot be flagged as "none of the above".

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What do the memberships of one data point in fuzzy c-means sum to?

    FCM constrains each point's memberships across the clusters to sum to 1.
  2. 2. What does a larger fuzzifier m do?

    As m grows, memberships move toward 1/c for every point.
  3. 3. How is a cluster center updated in FCM?

    Each center is the average of all points weighted by U raised to the power m.
  4. 4. Why run FCM from several random starts?

    Different initializations can end in different local minima; keep the best.

Summary

  • Fuzzy c-means gives each point a membership in every cluster, summing to 1.
  • It alternates weighted-average centers and distance-based memberships until stable.
  • The fuzzifier m sets the softness; m = 2 is the usual default.
  • Use partition coefficients and domain knowledge to choose c; restart several times and scale features.

Next: Neuro-Fuzzy Systems and ANFIS