Skip to content
Tutorials Center
My learning

Topics

Lesson 1 of 37 ยท Getting Started

What Are Numerical Methods?

Learn what numerical methods are, why exact formulas often fail, and how computers find approximate answers to equations, integrals and differential equations.

Most of the mathematics you meet in school has tidy, exact answers: solve 2x + 6 = 0 and you get x = -3. Real problems rarely cooperate. There is no formula for the roots of a general fifth-degree polynomial, most integrals have no closed form, and the differential equations that describe weather, bridges and circuits almost never have a neat solution you can write down.

Numerical methods are the procedures that fill that gap. Instead of an exact formula, they build a sequence of simple calculations (additions, multiplications, comparisons) that closes in on the answer, and they tell you how close you are. This course teaches those procedures, shows you how to write each one from scratch in Python, and then compares your version with the professional tools in NumPy and SciPy.

A problem with no formula

Try to solve cos(x) = x. No rearrangement of algebra isolates x. Yet the answer clearly exists: the curve of cosine crosses the straight line y = x once between 0 and 1. A numerical method does not try to isolate x. It guesses, checks, and improves the guess.

Python Solve cos(x) = x by repeated guessing

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each step feeds the previous answer back into cosine. The numbers wobble at first, then slowly settle near 0.739085. You never found a formula, but a loop of three lines gives the answer to many digits. That loop is a tiny numerical method, and you will meet it again as fixed-point iteration in the chapter on root finding.

The big families of problems

Almost everything in this field falls into a handful of families. Each one has its own chapters later in the course.

Four classic jobs

Find roots Solve f(x) = 0
Solve systems Many equations at once
Fit and interpolate Curves through data
Calculus Slopes and areas

Roots, linear systems, curves and calculus come first.

1 / 2

A few more reasons numerical thinking matters:

  • Data is discrete. A sensor gives you numbers at times 0, 0.1, 0.2 seconds, not a formula. To find its speed or total you must differentiate or integrate numbers.
  • Models are too complicated. A bridge model with a million unknowns is solved by software, not by hand.
  • Computers are not exact. Even 0.1 + 0.2 is not exactly 0.3 in a computer. Understanding the size of such errors is half of the subject.

Exact versus numerical, side by side

When an exact answer exists, you can use it to judge a numerical method. That is how you will test every algorithm in this course: apply it to a problem whose true answer you know, and measure the error.

Python Numerical answer versus exact answer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the pattern in the error column: each time n doubles, the error shrinks by about a factor of four. A method whose error follows a predictable pattern like this is one you can trust and control. Spotting these patterns is a skill you will build throughout the course.

Approximation is not guessing

A numerical answer is approximate, but it is never a guess. A good method comes with three guarantees:

  1. It converges. More work (more iterations, a finer grid) brings the answer closer to the truth.
  2. It has a known error behaviour. You can predict how fast the error shrinks, or bound the error.
  3. It is stable. Tiny disturbances in the input, such as rounding, do not blow up into huge changes in the output.
flowchart LR
    A["Real-world problem"] --> B["Mathematical model"]
    B --> C["Numerical method"]
    C --> D["Computer arithmetic"]
    D --> E["Approximate answer<br/>plus error estimate"]
    E -->|"too inaccurate?"| C

From a real problem to a trustworthy approximation. Errors can enter at every arrow.

Where errors come from

Four sources of error appear again and again. Learn to name them now:

Source Example
Modelling error Treating a pendulum as frictionless
Data error A thermometer reads 21.4 instead of 21.43
Truncation error Replacing an infinite series by its first five terms
Rounding error Storing 1/3 with only 16 significant digits

Chapter 2 is entirely about the last two, because they decide whether a method is usable.

How this course works

Every method is introduced the same way: the idea in plain words, the formula, a from-scratch Python implementation, a test on a problem with a known answer, and a comparison with NumPy or SciPy. Writing methods yourself shows you why they work and, more importantly, why they sometimes fail. In real projects you will normally call SciPy; knowing what is inside lets you choose the right routine and read its warnings.

Type the examples instead of only running them. Change the starting values, break a method on purpose, and watch what happens. Numerical analysis is learned by experiment.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why do we need numerical methods?

    Roots of high-degree polynomials, most integrals and most differential equations have no usable formula, so we compute approximations with controlled error.
  2. 2. In the cos(x) = x example, what did the loop do?

    That is fixed-point iteration: x is replaced by cos(x) again and again until the change becomes negligible.
  3. 3. Which of these is a truncation error?

    Truncation error comes from cutting off an infinite or limiting process. Rounding error comes from finite-precision storage.
  4. 4. Why test a new method on a problem with a known exact answer?

    Comparing with a known answer reveals bugs and confirms the error shrinks the way theory predicts.

Summary

  • Numerical methods compute approximate answers when exact formulas are missing or impractical.
  • Main families: root finding, linear systems, interpolation and fitting, differentiation and integration, differential equations, optimization and eigenvalues.
  • A good method converges, has predictable error and is stable.
  • Errors come from the model, the data, truncation and rounding.
  • We test every method against known answers and compare with SciPy.

Next: The Python Toolkit for Numerical Work