Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 38 ยท Projects

Project: Denoise and Analyze a Sensor Signal

Build a signal pipeline with SciPy: simulate a noisy sensor, inspect its FFT spectrum, filter it, detect peaks and report the result with plots.

In this project you play the role of an engineer who has been handed a recording from a pulse sensor. The raw trace is ugly: the baseline drifts, mains hum is mixed in, and random noise hides the shape of each beat. Your job is to clean the signal, find every beat, and report the heart rate with a measure of confidence.

You will combine four tools from the signal-processing chapter: the FFT (The Fast Fourier Transform), filter design (Filter Design and Filtering Signals), peak finding (Spectrograms, Peaks and Resampling) and simple statistics. Because the signal is synthetic, we know the truth, so we can measure how well each step works, something real projects can only dream of.

flowchart LR
    A["Simulate sensor<br/>250 Hz, 12 s"] --> B["Inspect spectrum<br/>rfft"]
    B --> C["Clean up<br/>highpass, notch, lowpass"]
    C --> D["Detect beats<br/>find_peaks"]
    D --> E["Report<br/>heart rate, SNR, plots"]

The pipeline: look first, then choose filters from what the spectrum shows, then measure.

Step 1: Simulate the sensor

The "true" signal is a train of smooth pulses, one every 0.8 seconds (75 beats per minute), each built from a narrow Gaussian bump. On top of it we add three kinds of damage that real sensors suffer from:

  1. Baseline drift: a slow wander, here a 0.15 Hz wave plus a gentle linear trend.
  2. Mains interference: a pure 50 Hz hum from the power line (use 60 Hz if you live in a 60 Hz country).
  3. Broadband noise: random Gaussian noise on every sample.

We use a fixed seed so every run is identical.

Python Step 1: a noisy pulse sensor

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the second panel. The beats are barely visible against the drift and the fuzz. A simple threshold such as "everything above 1.0" would miss beats on the downward slope of the drift and be fooled by the hum. We need to remove each disturbance separately, and the key to choosing how is the spectrum.

Step 2: Look at the spectrum

Every disturbance lives in a different place on the frequency axis. Drift is very slow (below 1 Hz), the hum is one sharp line at 50 Hz, and the noise is spread flat across all frequencies. The pulses themselves have energy at the heart rate (1.25 Hz) and its harmonics, up to roughly 10 Hz because each pulse is only about 100 ms wide.

scipy.fft.rfft returns the spectrum of a real signal, and rfftfreq gives the matching frequency axis. A Hann window (signal.get_window("hann", n)) tapers the ends of the recording to reduce spectral leakage.

Python Step 2: find the disturbances in the spectrum

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The recipe for the filter chain falls straight out of the picture:

  • Everything below roughly 0.5 Hz is drift and must go: a high-pass filter.
  • The single line at 50 Hz is hum: a narrow notch filter.
  • Above about 25 Hz there is nothing but noise (the pulses have almost no energy there): a low-pass filter.

Step 3: Filter the signal

Use zero-phase filtering, sosfiltfilt or filtfilt, which runs the filter forward and backward so that peaks are not shifted in time. A shifted peak would bias any timing measurement. Second-order sections (output="sos") are the numerically safe way to apply higher-order IIR filters.

We chain three filters. The order matters little for linear filters, but applying each one with sosfiltfilt or filtfilt keeps the timing exact. To judge success we compute a signal-to-noise ratio against the known clean signal:

SNR = 10 * log10(power of clean / power of (estimate - clean))

Because the drift is removed by high-pass filtering, the filtered signal sits around zero while the truth sits slightly above, so we compare after subtracting the mean from both.

Python Step 3: high-pass, notch and low-pass

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each stage buys a measurable improvement: the SNR climbs from about -5 dB for the raw trace to about 8 dB at the end, and the Savitzky-Golay alternative lands in the same range, a little higher here. Some residual error is unavoidable: a low-pass filter also blunts the tops of the pulses a little, and the high-pass filter slightly distorts the baseline around each beat. Choosing the cutoffs is always a trade-off between removing noise and preserving the signal.

Because sosfiltfilt filters twice (forward and backward), the effective attenuation is the square of the single-pass response and the effective order doubles. Keep this in mind when you quote a cutoff frequency.

Step 4: Detect the beats

signal.find_peaks locates local maxima and filters them by properties. Two arguments do most of the work:

  • height rejects peaks that are too low to be beats.
  • distance (in samples) enforces a minimum spacing, so one beat cannot be reported twice. With a human heart rate below 200 beats per minute, beats are at least 0.3 s apart, which is 75 samples at 250 Hz.

We can then convert the peak positions to times, compute the intervals between beats, and from them the heart rate and its variability.

Python Step 4: find the beats and the heart rate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The naive search on the raw signal reports more than 300 "peaks" because the noise and the hum create countless small local maxima. After filtering, requiring a prominence (how far a peak stands out from its surroundings) and a minimum distance gives exactly the 15 true beats, with timing errors of at most about 16 ms (4 samples). The heart rate is estimated from the mean interval, which is more robust than counting peaks in a fixed window.

Step 5: Produce the report

A good analysis ends with a short written result and one figure that tells the story. The last step collects the numbers into a report dictionary and draws a summary figure with the raw signal, the filtered signal with detected beats, and the interval series. Keeping the whole pipeline in a single function also makes it easy to rerun on a different recording.

Python Step 5: the whole pipeline in one function

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The pipeline reports about 75 beats per minute and an interval spread of about 10 ms (this comes purely from residual noise, because the synthetic heart is perfectly regular). It also copes with the harder recording, although with more noise you should always check the number of detected beats against what you expect.

Ideas to extend the project

  • Replace the fixed filters by a band-pass filter (0.5 to 25 Hz) and compare the results.
  • Add a few random spikes (artifacts) and remove them with signal.medfilt before filtering.
  • Compute the spectrogram with signal.spectrogram and watch the heart rate drift if you make the beat spacing change over time.
  • Use signal.correlate with a single clean pulse as a template (a matched filter) and detect beats from the correlation peaks.
  • Resample the recording to 125 Hz with signal.resample_poly and see how little the answer changes.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is sosfiltfilt preferred over sosfilt when you want to measure the timing of peaks?

    A single causal filter delays the signal by an amount that depends on frequency. Running the filter forward and backward cancels the delay, so peak times are not biased.
  2. 2. In the spectrum of the raw trace, a sharp line at 50 Hz tells you to use which filter?

    A single-frequency disturbance is removed best with a narrow notch, which leaves neighboring frequencies of the real signal almost untouched.
  3. 3. Which two arguments of find_peaks prevent noise bumps from being reported as beats?

    prominence demands that a peak stand out from its surroundings and distance enforces a minimum spacing in samples, so small noise maxima and double detections are rejected.
  4. 4. Why is the heart rate computed as 60 / mean(intervals) rather than from a single interval?

    Each peak time has a small error. The mean of many intervals averages that error down, giving a steadier estimate.

Summary

  • Always look at the spectrum first: drift is low-frequency, mains hum is a single line, white noise is a flat floor, and the choice of filters follows from that picture.
  • Use a high-pass for drift, a notch for the mains line and a low-pass for broadband noise; apply them with zero-phase sosfiltfilt/filtfilt when timing matters.
  • A synthetic signal with known truth lets you measure improvement with the SNR in decibels.
  • find_peaks with height, prominence and distance finds beats reliably; the mean interval gives the heart rate.
  • Wrap the pipeline in a function so you can reuse it on new recordings, and test it on harder cases.

Next project: model an epidemic with differential equations and fit it to data in Project: Fit an Epidemic Model to Data.