Skip to content
Tutorials Center
My learning

Topics

Lesson 7 of 68 · Derivatives

The Derivative as a Rate of Change

Learn the derivative as the limit of a difference quotient, the slope of a tangent line and an instantaneous velocity, with Python finite-difference checks.

A car's speedometer does not tell you how far you have driven. It tells you how fast the distance is changing right now. That number, an instantaneous rate of change, is what a derivative measures. It is the single most useful idea in calculus: velocity from position, current from charge, marginal cost from total cost, and the slope of a loss curve in machine learning are all derivatives.

In this lesson you will build the derivative from scratch. You start with an average rate over an interval, shrink the interval until it becomes a single instant, and see that the result is the slope of a tangent line. Then you will use the same formula on a computer, where it becomes a practical tool called a finite difference, and discover how the step size controls the error. You should be comfortable with limits and with functions and graphs.

Average rate of change

Take a function f(x). Between x = a and x = a + h, its output changes from f(a) to f(a + h). The average rate of change is the change in output divided by the change in input:

TEXT
(f(a + h) - f(a)) / h

This ratio is called the difference quotient. Geometrically it is the slope of the secant line, the straight line through the two points (a, f(a)) and (a + h, f(a + h)).

Suppose a dropped stone has fallen s(t) = 5t² metres after t seconds (a rounded version of free fall). Between t = 1 and t = 3, it falls from 5 m to 45 m, so the average speed is (45 - 5) / 2 = 20 m/s. But the stone speeds up, so 20 m/s is not its speed at any particular moment. To find the speed at exactly t = 1, we shrink the interval.

From secant to tangent

Fix the point a and let h get smaller. The second point slides toward the first, and the secant line rotates until it just touches the curve at a. That limiting line is the tangent line, and its slope is the derivative.

flowchart LR
    A["Secant slope<br/>(f(a+h) - f(a)) / h"] -->|"h gets smaller"| B["Slopes settle<br/>on one number"]
    B -->|"limit as h approaches 0"| C["Derivative f'(a)<br/>= tangent slope"]

Shrinking the interval turns an average rate into an instantaneous rate.

The formal definition uses the limit you met earlier:

TEXT
f'(a) = lim (h -> 0) of (f(a + h) - f(a)) / h

If this limit exists, we say f is differentiable at a, and f'(a) (read "f prime of a") is the derivative there. Other notations mean the same thing: df/dx (Leibniz), dy/dx, or D f. Leibniz's notation reminds you that the derivative is a ratio of a tiny change in y to a tiny change in x, and it is the one engineers use most.

Python Secant slopes approach the tangent slope

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Algebra confirms what the table suggests. For f(x) = x², the difference quotient is ((a + h)² - a²) / h = (2ah + h²) / h = 2a + h. As h goes to 0 this becomes 2a, so f'(a) = 2a. At a = 3 that is 6, exactly the number the secant slopes were approaching. You now know your first derivative formula: the derivative of x² is 2x.

The derivative as a function

We computed f'(a) at one point, but the formula 2a works for every a. So the derivative is itself a function, f'(x) = 2x, which gives the slope of f at each x. Where f' is positive the graph rises, where it is negative the graph falls, and where it is zero the tangent is horizontal.

Velocity and other rates

If s(t) is position, then s'(t) is velocity, and the derivative of velocity is acceleration. For the stone, s(t) = 5t² gives v(t) = s'(t) = 10t, so at t = 1 the speed is 10 m/s and at t = 3 it is 30 m/s. The average 20 m/s we found earlier sits between those two, as it must.

Units carry the story. If s is in metres and t in seconds, then s' is in metres per second: "output units per input unit". The same pattern gives amperes (charge per second), watts (energy per second) or dollars per item.

The derivative is a local property. It depends only on how f behaves very close to a, not on the whole curve.

Derivatives on a computer: finite differences

Computers cannot take a true limit, but they can use a small, fixed h. The forward difference is

TEXT
f'(a) ≈ (f(a + h) - f(a)) / h

A better choice is the central difference, which looks equally far on both sides:

TEXT
f'(a) ≈ (f(a + h) - f(a - h)) / (2h)

Why is it better? The forward difference has an error proportional to h, while the central difference has an error proportional to h². Halving h halves the first error but quarters the second. This is truncation error, the price of stopping before the limit.

There is a catch. If h is too small, f(a + h) and f(a) become nearly equal and the subtraction wipes out most of the significant digits, an effect called round-off error (see Floating-Point Error and Conditioning). The best h balances the two errors. Run this experiment on f(x) = sin(x) at x = 1, where the true derivative is cos(1).

Python Error versus step size

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the table from the top. At first both errors shrink as h shrinks, and the central difference shrinks much faster. Then, around h = 1e-5 to 1e-8, the errors reach a minimum and begin to climb again as round-off takes over. The takeaway: a smaller step is not always better. A common rule of thumb is h near the cube root of machine precision (about 1e-5) for central differences and near its square root (about 1e-8) for forward differences.

NumPy offers np.gradient, which applies these ideas to sampled data. It uses central differences in the interior of the array and one-sided differences at the ends.

Python Velocity from position data

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The interior estimates are exact here because the central difference is exact for quadratics; only the two end points, which use one-sided differences, are off by a little.

When a derivative does not exist

Not every function has a derivative everywhere. Three common failures:

  • A corner, like |x| at x = 0: the slopes from the left (-1) and right (+1) disagree, so the limit does not exist.
  • A jump or other discontinuity: you cannot get a slope at a break. Differentiable implies continuous, but not the other way around.
  • A vertical tangent, like x^(1/3) at 0: the slope grows without bound.

You can see the corner numerically by computing the two one-sided quotients for |x| at 0. They stay at -1 and +1 no matter how small h gets.

Check your understanding

Quick check · 5 questions

of 5 answered You scored / 5 · perfect!

  1. 1. What does the difference quotient (f(a + h) - f(a)) / h represent geometrically?

    It is rise over run between the points (a, f(a)) and (a + h, f(a + h)), which is the slope of the secant line. Letting h go to 0 turns it into the tangent slope.
  2. 2. Using the definition, what is the derivative of f(x) = x^2?

    The difference quotient simplifies to 2x + h, and as h goes to 0 this becomes 2x.
  3. 3. A position s(t) is measured in metres and t in seconds. What are the units of s'(t)?

    A derivative has units of output per input, so s'(t) is in metres per second, a velocity.
  4. 4. Why does making the step h smaller and smaller eventually make a finite-difference estimate worse?

    Truncation error falls as h shrinks, but f(a + h) and f(a) become nearly identical, so their difference loses digits to floating-point round-off. The best h balances both.
  5. 5. Which function is NOT differentiable at x = 0?

    |x| has a corner at 0: the left slope is -1 and the right slope is +1, so no single tangent slope exists.

Summary

  • The derivative f'(a) is the limit of the difference quotient (f(a + h) - f(a)) / h as h approaches 0.
  • Geometrically it is the slope of the tangent line; physically it is an instantaneous rate such as velocity.
  • The derivative of x² is 2x, and f' is itself a function giving the slope at every point.
  • Finite differences approximate derivatives on a computer: the forward error shrinks like h, the central error like h², and round-off limits how small h can usefully be.
  • Corners, jumps and vertical tangents are places where a derivative fails to exist.

Next, learn shortcuts that avoid the limit entirely in Differentiation Rules.