Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 37 ยท Projects and Practice

Project: Fit, Differentiate and Integrate Sensor Data

Build a small numerical pipeline: interpolate missing readings, fit a decay model, estimate rates of change and integrate to total energy, with honest error checks.

Numerical methods rarely arrive one at a time. A real task chains several of them, and each step's error feeds the next. In this project you receive noisy temperature readings from a cooling object (a mug of tea) with a few missing values, and you will answer four questions:

  1. What are the readings at the times that are missing? (interpolation)
  2. What is the model: how fast does it cool, and to what room temperature? (nonlinear least squares)
  3. How fast is it cooling at a given moment? (differentiation)
  4. How much heat has been lost in total? (integration)

The data are simulated here so we know the truth and can check each answer. With real data you would not have this luxury, so the project also practices quantifying uncertainty.

Newton's law of cooling says T(t) = T_room + (T0 - T_room) exp(-k t). We generate data from it with T_room = 21, T0 = 90 and k = 0.045 per minute, add noise of 0.4 degrees, and delete some readings.

Step 1: the data

Python Generate noisy readings with gaps

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Step 2: fill the gaps (interpolation)

Interpolating through noisy points reproduces the noise, but for filling a small gap it is acceptable if the spline is smooth. Compare a cubic spline and linear interpolation against the (hidden) truth at the missing times. See cubic splines.

Python Interpolate the missing readings

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both methods land within about half a degree of the truth, and here the linear values are even a little closer: the noise (a few tenths of a degree) matters more than the curvature between points three minutes apart. Interpolation cannot do better than the noise in its neighbours.

Step 3: fit the physical model (least squares)

Better than interpolating is to fit the model we believe in. We use curve_fit on the kept points and read the uncertainty from the covariance matrix. See least squares curve fitting.

Python Fit Newton's law of cooling

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The residual scatter (about 0.32 degrees, the noise this sample actually contains) is the sign of a model that explains the data. Notice that T_room and k are each a couple of standard errors off the truth: with 18 noisy points these parameters trade off against each other, which the full covariance matrix would show. The fit also gives the missing readings immediately: evaluate model at the lost times.

Step 4: rate of cooling (differentiation)

Differentiating raw noisy data is noisy, as the differentiation lesson warned. The fitted model gives a clean derivative analytically: dT/dt = -k * dT * exp(-k t). Compare with finite differences on the raw data.

Python Derivative from raw data versus from the model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The model-based derivative has a much smaller error. Smoothing before differentiating is not optional with noisy data.

Step 5: total heat lost (integration)

The heat lost by the tea is proportional to the temperature drop. In "degree-minutes", the area between the curve and the room temperature is integral of (T - T_room) dt. We integrate the fit analytically-by-quadrature and the raw data by the trapezoid rule, and compare with the exact value dT (1 - exp(-k t_end)) / k.

Python Integrate the data and the model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All three areas agree to within about 0.6 percent. The trapezoid rule on the raw data happens to be closest here, because noise of both signs cancels in a sum; the fitted-model integral inherits the small parameter errors from the fit. The point is the scale: integration is far more forgiving of noise than differentiation.

Extensions

  • Bootstrap the uncertainty: refit on resampled residuals and report a range for k.
  • Use a stiff solver to simulate Newton's law with a time-varying room temperature and fit that.
  • Weight the fit if the noise grows with temperature (sigma= in curve_fit).
  • Predict when the tea reaches 60 degrees by root finding on the fitted curve.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is a fitted model better than raw finite differences for the cooling rate?

    Differences of noisy neighbours divide the noise by a small spacing; a smooth model removes most of it first.
  2. 2. What does the covariance matrix returned by curve_fit give you?

    The square roots of its diagonal are standard errors of the fitted parameters.
  3. 3. Why does linear interpolation overestimate a convex decaying curve between points?

    For a convex function, the straight line between two points lies above the curve.
  4. 4. What does a residual standard deviation equal to the known noise level suggest?

    When residuals look like pure noise at the expected level, the model captures the systematic behaviour.

Summary

  • A numerical pipeline chains interpolation, fitting, differentiation and integration; know the error behaviour of each step.
  • Fit a model you believe in rather than interpolating noise; derive rates and areas from the fit when data are noisy.
  • Simulated data with known truth let you test every stage before trusting it on real measurements.

Next: Practice problems