Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 49 ยท Working with Real Data

Signals, Convolution and the FFT

Learn to build sine waves, find frequency peaks with np.fft.rfft, filter noise, smooth data with np.convolve and avoid windowing and aliasing mistakes.

A signal is just an array of measurements taken over time: audio samples, a temperature log, the voltage from a sensor, daily sales. NumPy is a natural tool for signals because a signal is a one-dimensional array, and most signal processing is arithmetic on arrays. Two ideas cover a surprising amount of ground: convolution, which slides a small pattern along the signal to smooth it or detect changes, and the Fourier transform, which tells you which frequencies the signal contains.

In this lesson you will generate test signals with linspace and arange, use np.fft.rfft to find the frequencies hiding in them, remove noise by editing the spectrum, smooth data with np.convolve, and learn the two classic traps: spectral leakage and aliasing. You do not need any signal-processing background. If a few array operations look unfamiliar, review vectorized operations first.

Making a signal

To work with frequencies you need to decide on a sampling rate fs, the number of samples per second, and then build a time axis. A sine wave with frequency f hertz (cycles per second) is np.sin(2 * np.pi * f * t).

Use np.arange(0, 1, 1 / fs) for the time axis when you want exactly one sample per tick and want to exclude the end point. np.linspace(0, 1, n, endpoint=False) gives the same thing. Leaving out the end point matters: a signal that is meant to repeat should not contain the first sample twice.

The Fourier transform answers the question "how much of each frequency is in this array?". For real-valued data you should call np.fft.rfft, the "real" FFT. It returns only the non-negative frequencies, because the negative ones are mirror images and carry no extra information. np.fft.rfftfreq(n, d) gives the frequency, in hertz, of each output bin, where d is the time between samples.

flowchart LR
    A["Samples<br/>(time domain)"] -->|"np.fft.rfft"| B["Complex spectrum<br/>(frequency domain)"]
    B --> C["np.abs gives amplitude"]
    B -->|"edit bins"| D["Modified spectrum"]
    D -->|"np.fft.irfft"| E["New signal"]

The forward FFT moves a signal into the frequency domain, where you can inspect or edit it; the inverse FFT brings it back.

Python Find the frequencies in a signal

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The signal is a 5 Hz wave of amplitude 1 plus a 20 Hz wave of amplitude 0.5, and the loop recovers exactly that. Three details are worth remembering:

  • With 100 samples, rfft returns 100 // 2 + 1 = 51 complex numbers, for frequencies 0, 1, 2, ... 50 Hz.
  • The raw magnitude np.abs(spectrum) grows with the number of samples. Multiplying by 2 / n turns it into the amplitude of a sine wave (the DC bin at 0 Hz and, for even n, the Nyquist bin should use 1 / n, but for peak-hunting the factor 2 is fine).
  • The frequency resolution is fs / n. Longer recordings give finer resolution: 1 second gives 1 Hz bins, 10 seconds gives 0.1 Hz bins.

Filtering noise in the frequency domain

Once you can see the spectrum, you can edit it. The simplest filter is a low-pass filter: set every bin above a cut-off frequency to zero and transform back with np.fft.irfft. Pass n= to irfft so the output has the original length. Without it, an odd-length signal comes back one sample short.

Python Low-pass filter with rfft and irfft

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The random noise spreads its energy over every frequency, while the real signal sits in a single tall spike at 7 Hz. Zeroing everything above 15 Hz throws away most of the noise and keeps the spike, so the error drops from about 0.70 to about 0.26 (the remaining error is the noise that lives below the cut-off). The same trick, with other masks, gives a high-pass filter (zero the low bins) and a band-pass filter (keep a range).

Cutting the spectrum with a hard edge is simple but not ideal: it can produce small ripples near sharp changes in the signal. Production filters (in SciPy's scipy.signal) use smoother designs. For learning, and for quick data cleaning, a hard cut is a perfectly good start.

Convolution: sliding a pattern along a signal

Convolution combines two arrays by flipping one of them and sliding it along the other, multiplying and summing at every position. The array you slide is called the kernel. With a kernel of all equal weights you get a moving average; with a kernel like [1, -1] you get differences between neighbours.

np.convolve(a, v, mode) takes three mode options that decide what happens at the edges:

Mode Output length Meaning
"full" (default) len(a) + len(v) - 1 every position where the arrays overlap at all
"same" max(len(a), len(v)) the full result, trimmed to the size of the longer input and centred
"valid" max(len(a), len(v)) - min(len(a), len(v)) + 1 only positions where the kernel fits completely
Python Moving averages and differences with np.convolve

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last lines show the convolution theorem: convolving two signals is the same as multiplying their spectra and transforming back. We padded both transforms to length n (the full output length) so the result is not wrapped around. For small kernels np.convolve is the right tool. For very long signals and long kernels, the FFT route is much faster, and that is what scipy.signal.fftconvolve does for you.

Notice also that np.convolve flips the kernel. For symmetric kernels such as a moving average that makes no difference, but for [1, -1] it is why the result is "this value minus the previous one". If you need correlation (no flip), use np.correlate.

There is a related, deeper treatment of rolling averages in statistics and correlation.

Windowing and spectral leakage

The FFT assumes that the chunk of signal you gave it repeats forever. If the chunk contains a whole number of cycles of a wave, the end of one repeat joins smoothly to the start of the next. If not, there is a jump at the seam, and the energy of that jump smears across many frequency bins. This is spectral leakage.

A window function fades the signal to zero at both ends before the transform, which hides the seam. NumPy provides np.hanning, np.hamming, np.blackman, np.bartlett and np.kaiser. Multiply the signal by the window, and divide by the window's sum when you want to read amplitudes correctly.

Python Leakage with and without a Hann window

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For the wave that fits the window exactly, the plain FFT is perfect: a peak of 1.00 and no leakage. For the 8.5 Hz wave the plain transform reports a peak of only 0.65 and leaks 0.0345 into far-away bins. With the Hann window the far-away leakage falls to 0.0007, at the price of a slightly wider peak. In practice you rarely control how many cycles fit in your recording, so a window is the default choice when you analyse real data.

Sampling rate pitfalls

Two rules decide what an FFT can tell you.

  1. Nyquist limit. With sampling rate fs you can only represent frequencies up to fs / 2. That is exactly the last bin of rfftfreq.
  2. Aliasing. A frequency above the limit does not disappear; it folds back and masquerades as a lower one.
flowchart TD
    A["Real tone at 10 Hz"] --> B["Sample at only 12 Hz"]
    B --> C["Nyquist limit is 6 Hz"]
    C --> D["10 Hz folds back to 12 - 10 = 2 Hz"]
    D --> E["FFT shows a peak at 2 Hz"]

A tone above half the sampling rate is indistinguishable from a slower one after sampling.

Python Aliasing: a 10 Hz tone that looks like 2 Hz

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The 10 Hz samples are identical to those of an inverted 2 Hz wave, so no analysis can tell them apart afterwards. The cure has to happen before sampling: record at more than twice the highest frequency you care about, and filter out higher frequencies first (real audio hardware has an analog anti-aliasing filter for this reason).

Other mistakes to avoid:

  • Forgetting the sampling rate. The FFT bins are in "cycles per recording", not hertz, until you supply d=1 / fs to rfftfreq.
  • Using fft instead of rfft on real data. It works, but returns twice as many numbers, half of them redundant.
  • Reading the raw magnitude as amplitude. Divide by n (and double it for one-sided spectra).
  • Losing the length. Pass n= to irfft for odd lengths.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A signal has 500 samples recorded at 100 samples per second. What is the spacing between FFT bins?

    The resolution is fs / n = 100 / 500 = 0.2 Hz. Longer recordings give finer frequency resolution.
  2. 2. Why call np.fft.rfft instead of np.fft.fft for real-valued data?

    For real input the negative-frequency half of the spectrum is a mirror of the positive half, so rfft returns just the useful part.
  3. 3. What does np.convolve(a, v, mode="valid") return?

    Valid mode drops the edge positions where the kernel would hang over the end, so the output is shorter than the input.
  4. 4. You sample a 70 Hz tone at 100 samples per second. Where will it appear in the spectrum?

    The Nyquist limit is 50 Hz, so 70 Hz folds back to 100 - 70 = 30 Hz.
  5. 5. What is the purpose of multiplying a signal by a Hann window before the FFT?

    Fading the ends hides the jump at the seam where the FFT assumes the signal repeats, which reduces leakage into other bins.

Summary

  • A signal is a 1D array; build one with np.arange(0, T, 1 / fs) and np.sin(2 * np.pi * f * t).
  • np.fft.rfft gives the spectrum of a real signal, np.fft.rfftfreq(n, d=1 / fs) gives the frequency of each bin, and np.fft.irfft(spec, n=n) inverts it.
  • Amplitude is abs(spectrum) * 2 / n; frequency resolution is fs / n.
  • Edit the spectrum (zero bins) to filter; the result is a simple low-pass, high-pass or band-pass filter.
  • np.convolve slides a kernel over a signal: use mode="valid" to avoid edge effects, a normalised kernel for averages, and [1, -1] for differences.
  • Window your data (np.hanning) to reduce leakage, and sample at more than twice your highest frequency to avoid aliasing.

Next: Memory Layout and Strides