Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 38 ยท Sparse, Spatial and Image Data

Special Functions and Clustering

Use scipy.special for gamma, beta, erf, Bessel, comb, expit and logsumexp, avoid overflow with stable formulas, and cluster data with kmeans and vq.

Some functions show up in so many formulas that mathematicians gave them names: the gamma function extends factorials to non-integers, the error function is the heart of the normal distribution, Bessel functions describe vibrating drums. scipy.special provides fast, carefully tested implementations of hundreds of them, all as ufuncs that work on arrays just like np.sin.

This lesson has three parts. First a tour of the functions you will meet most often. Then the numerical stability tricks that keep calculations from overflowing or losing precision, which is the practical reason most programmers reach for scipy.special. Finally the k-means clustering tools in scipy.cluster.vq, closing the section on spatial data after Spatial Algorithms.

A tour of scipy.special

The functions fall into a few families. You rarely need the formulas; you need to know what each one is for.

Function What it computes Typical use
gamma(x), gammaln(x) Gamma function, log of its absolute value factorials of non-integers, distributions
beta(a, b), betaln Beta function Beta distribution, Bayesian updates
erf(x), erfc(x), erfinv Error function, its complement and inverse normal CDF, measurement errors
jv(v, x), j0, j1 Bessel functions of the first kind waves, vibrations, signal processing
comb(n, k), perm(n, k) Combinations and permutations counting, probability
expit(x), logit(p) Logistic sigmoid and its inverse logistic regression, neural networks
logsumexp(a) log of the sum of exponentials probabilities in log space
softmax(x), log_softmax(x) Normalized exponentials classification outputs
Python Gamma, beta, erf and friends

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the exact=True option of comb: it returns a Python integer with no rounding, which is what you want for comb(50, 25). Without it you get a float, which is fine for moderate sizes and loses digits for huge ones.

scipy.special functions are vectorized: pass arrays and get arrays. The scipy.stats normal distribution (see Distributions and Descriptive Statistics) is built on exactly these functions, for instance ndtr for the CDF and ndtri for the inverse.

Small helpers: cbrt, exp10, exprel and friends

Besides the famous functions, scipy.special has a handful of tiny, vectorized helpers that fill gaps in NumPy or fix its rounding problems. They all accept scalars and arrays.

Function Computes Why use it
cbrt(x) the real cube root np.cbrt exists too; x ** (1/3) fails for negative numbers
exp10(x) 10 to the power x exact for whole numbers (exp10(3) is exactly 1000)
exp2(x) 2 to the power x exact powers of two
exprel(x) (exp(x) - 1) / x stays accurate when x is tiny, where the plain formula loses all digits
lambertw(z) the Lambert W function: the solution w of w * exp(w) = z solves equations where the unknown is both inside and outside an exponential
comb(n, k), perm(n, k) "n choose k" and the number of ordered selections exact big-integer results with exact=True

exprel deserves a closer look. For x = 1e-10 the expression (np.exp(x) - 1) / x subtracts two numbers that agree in 10 digits, so most of the precision vanishes. exprel uses a series for small x and returns the right answer. Anything in your code of the form (exp(x) - 1) / x should call it instead.

The Lambert W function has several branches. lambertw(z) returns a complex number (the real part is the answer when z >= -1/e on the main branch k=0; take .real). A classic use: solving x * 2**x = 10 without any iteration of your own. Write x * ln2 * exp(x * ln2) = 10 * ln2, so x * ln2 = W(10 * ln2).

Python cbrt, exp10, exprel and lambertw

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The naive formula returns 1.000000082740371 (only 8 correct digits), while exprel returns 1.00000000005. For z = -0.2, two real solutions exist: the main branch k=0 gives about -0.2592 and the lower branch k=-1 gives about -2.5426; both satisfy w * exp(w) = -0.2.

Why numerical stability matters

Computers store floating-point numbers with a limited range (about 1e-308 to 1e308) and about 16 digits of precision. A formula that is perfectly correct on paper can still fail: exp(1000) overflows to inf, log(0) is -inf, and 1 - cos(x) for tiny x cancels almost all its digits.

flowchart LR
    A["Large input<br/>x = 1000"] --> B["np.exp(x)"]
    B --> C["inf<br/>overflow"]
    C --> D["inf / inf = nan"]
    A --> E["subtract the max first"]
    E --> F["exp(x - max) is at most 1"]
    F --> G["correct result"]

Overflow turns a valid formula into nan; shifting by the maximum keeps every exponential in range.

The fix is usually algebra before arithmetic: rewrite the formula so no intermediate result is huge, tiny or the difference of two almost-equal numbers. SciPy and NumPy ship stable versions of the most common cases:

  • logsumexp(a) computes log(sum(exp(a))) by factoring out the largest term first.
  • softmax and log_softmax use the same trick, so class scores like [1000, 1001, 1002] work.
  • expit(x) is a stable sigmoid, correct for large positive or negative x.
  • np.log1p(x) and np.expm1(x) compute log(1 + x) and exp(x) - 1 accurately for tiny x.
  • gammaln, betaln work with the logarithm of huge values; compare gamma(200) (overflows) with gammaln(200) (a modest number).
  • xlogy(x, y) computes x * log(y) and returns 0 when x = 0, which is what entropy formulas need.
Python Naive formulas versus stable ones

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The pattern in every case is the same: the naive version is the textbook formula, the stable version is algebraically identical but ordered so that no intermediate result leaves the representable range. Whenever you see exp, log, a ratio of factorials or 1 - something_close_to_1 in your code, ask whether a stable function exists.

Work in log space whenever you multiply many probabilities. A product of a thousand numbers below 0.1 underflows to zero, but the sum of their logarithms is a perfectly ordinary number.

k-means clustering with scipy.cluster.vq

k-means splits n observations into k groups by alternating two steps: assign every point to its nearest centroid, then move each centroid to the mean of its points. The module scipy.cluster.vq offers a small, fast version:

  • whiten(data) divides each column by its standard deviation so that features with large units do not dominate the distance. Always do this when columns have different scales.
  • kmeans(data, k) returns the centroids and the average distance of points to their nearest centroid (the distortion). It uses random starts, so pass a seed for reproducibility.
  • kmeans2(data, k, minit="++") also returns a label for each point and supports the k-means++ initialization.
  • vq(data, centroids) ("vector quantization") assigns each observation to its nearest centroid and returns (labels, distances).
Python kmeans, vq and choosing k

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The distortion always decreases as k grows (with k equal to the number of points it reaches zero), so the number is only useful for spotting the elbow: the k after which extra clusters stop helping much. In this data the drop flattens after k=3, matching the three groups we generated. Results can vary with the random start; seed= fixes it. For anything heavier than a quick grouping, scikit-learn offers a richer KMeans, but the SciPy version needs no extra dependency and is perfect for vector quantization.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does np.log(np.sum(np.exp(logp))) fail for log-probabilities like -1000, while special.logsumexp(logp) works?

    exp(-1000) is smaller than the smallest float, so it becomes 0 and log(0) is -inf. logsumexp subtracts the maximum first, which keeps the exponentials in range.
  2. 2. What is special.gamma(5)?

    gamma(n) equals (n - 1)!, so gamma(5) = 4! = 24.
  3. 3. When should you call whiten before kmeans?

    k-means uses Euclidean distance, so a column measured in the hundreds would swamp one measured in single digits. whiten rescales each column to unit variance.
  4. 4. What does vq(data, centroids) return?

    vq assigns each observation to its closest centroid and returns (labels, distances). It is how you classify new points with fitted centroids.
  5. 5. Which pair avoids precision loss for very small x?

    Adding 1 to a tiny number throws away its digits. log1p and expm1 compute the same quantities without forming 1 + x or subtracting two nearly equal values.

Summary

  • scipy.special supplies vectorized gamma, beta, erf, Bessel, comb/perm, expit/logit and many more; comb(..., exact=True) returns exact integers.
  • Stability beats elegance: use logsumexp, softmax, expit, log1p, expm1, gammaln and xlogy instead of the textbook formula whenever huge, tiny or nearly equal numbers can appear.
  • Multiply probabilities by adding logs.
  • scipy.cluster.vq offers whiten, kmeans, kmeans2 and vq; whiten first, fix the seed, and use the distortion curve to find the elbow.

Next up: put everything together on a realistic task in Project: Denoise and Analyze a Sensor Signal.