Skip to content
Tutorials Center
My learning

Topics

Numerical Methods Tutorial

Learn Numerical Methods step by step: 37 lessons in 9 chapters with examples, quizzes, diagrams and projects.

Start learning →

of lessons completed

Getting Started

  1. 1 What Are Numerical Methods? Learn what numerical methods are, why exact formulas often fail, and how computers find approximate answers to equations, integrals and differential equations.
  2. 2 The Python Toolkit for Numerical Work Set up the Python tools used throughout numerical methods: NumPy arrays, vectorized math, linspace, matplotlib plots and timing, with runnable examples.
  3. 3 Iteration, Convergence and Stopping Rules Understand iterative methods, linear and quadratic convergence, how to measure the order of convergence in code, and how to choose safe stopping criteria.
  4. 4 Your First Method: Square Roots Build your first numerical algorithm: the Babylonian square root method, see why it converges so fast, compare it with bisection, and learn how to test it.

Errors and Floating Point

  1. 5 Absolute and Relative Error Learn how to measure the size of an error with absolute error, relative error, percent error and significant digits, and how to pick the right one for your problem.
  2. 6 Floating-Point Numbers How computers store real numbers: IEEE 754 doubles, sign, exponent and mantissa, machine epsilon, spacing between floats, overflow, underflow, and inexact decimals.
  3. 7 Rounding Error and Cancellation See how small rounding errors grow into big ones: catastrophic cancellation, summation order, Kahan summation, and how to rewrite formulas to avoid losing digits.
  4. 8 Truncation Error and Taylor Series Understand truncation error from cutting off a series, use Taylor's theorem to bound it, measure the order of accuracy O(h^p), and balance it against rounding error.
  5. 9 Conditioning and Stability Learn the difference between ill-conditioned problems and unstable algorithms, compute the condition number of a function, and see Wilkinson's polynomial in action.

Root Finding

  1. 10 The Bisection Method Learn the bisection method for finding roots: the intermediate value theorem, a Python implementation, how many iterations you need, its limits, and a SciPy comparison.
  2. 11 Newton-Raphson Method Derive Newton's method from the tangent line, implement it in Python, see quadratic convergence, learn why it fails and how multiple roots slow it down.
  3. 12 Secant and False-Position Methods Find roots without derivatives: the secant method and the regula falsi (false position) method, with Python code, convergence rates and the Illinois improvement.
  4. 13 Fixed-Point Iteration and Root Finding with SciPy Learn fixed-point iteration and the contraction condition, then use SciPy's brentq, newton, root_scalar and root to solve single equations and nonlinear systems.

Linear Systems

  1. 14 Gaussian Elimination Solve linear systems Ax = b with Gaussian elimination: forward elimination, back substitution, partial pivoting, operation counts and a NumPy comparison, with runnable code.
  2. 15 LU Decomposition and Pivoting Factor a matrix as PA = LU, reuse the factors to solve many right-hand sides cheaply, compute determinants and inverses, and compare with scipy.linalg.lu_factor.
  3. 16 Matrix Norms and Condition Numbers Understand vector and matrix norms, the condition number of a matrix, the Hilbert matrix example, and how the residual differs from the true error when solving Ax = b.
  4. 17 Jacobi and Gauss-Seidel Methods Solve large linear systems iteratively with the Jacobi, Gauss-Seidel and SOR methods: update formulas, diagonal dominance, spectral radius, convergence tests and runnable code.
  5. 18 Special Systems: Cholesky, Tridiagonal and Conjugate Gradient Exploit matrix structure: Cholesky factorisation for symmetric positive definite matrices, the O(n) Thomas algorithm for tridiagonal systems and the conjugate gradient method.

Interpolation and Curve Fitting

  1. 19 Polynomial Interpolation: Lagrange and Newton Build the unique polynomial through your data points with the Lagrange form and Newton divided differences, then compare both with NumPy and SciPy.
  2. 20 The Runge Phenomenon and Chebyshev Nodes See why high-degree interpolation on evenly spaced points oscillates wildly, and how Chebyshev nodes tame the error. Examples in NumPy and SciPy.
  3. 21 Piecewise Interpolation and Cubic Splines Connect data points with linear pieces or smooth cubic splines. Derive the tridiagonal system, solve it from scratch and compare with SciPy's CubicSpline.
  4. 22 Least Squares Curve Fitting Fit lines, polynomials and nonlinear models to noisy data with least squares. Normal equations, QR and lstsq, then SciPy's curve_fit, with working code.

Numerical Differentiation and Integration

  1. 23 Numerical Differentiation: Finite Differences Approximate derivatives with forward, backward and central differences. See truncation versus round-off error, the best step size and complex-step tricks.
  2. 24 Richardson Extrapolation and Romberg Integration Combine two rough estimates into a much better one. Learn Richardson extrapolation for derivatives and the Romberg table for integrals, with code.
  3. 25 Numerical Integration: Trapezoid and Simpson Rules Compute definite integrals with the midpoint, trapezoid and Simpson rules. Learn composite forms, error orders and how to test convergence in Python.
  4. 26 Gaussian and Adaptive Quadrature Get exact answers for polynomials of degree 2n-1 with n points. Learn Gauss-Legendre, adaptive Simpson and SciPy's quad for difficult integrals.

Ordinary Differential Equations

  1. 27 Euler's Method for Ordinary Differential Equations Solve initial value problems step by step with forward and backward Euler. Learn local and global error, order of convergence and stability with Python code.
  2. 28 Runge-Kutta Methods (RK2 and RK4) Reach high accuracy for ODEs without computing derivatives. Implement midpoint, Heun and classic fourth-order Runge-Kutta, and verify their convergence order.
  3. 29 Adaptive Step Size and Stiff ODEs Let the solver choose its own step size with embedded Runge-Kutta pairs, and learn what stiffness is and why implicit methods like BDF and Radau solve it.
  4. 30 Systems of ODEs and Higher-Order Equations Turn second-order equations into first-order systems and solve pendulums, springs and predator-prey models with RK4 and solve_ivp, including boundary value problems.

Optimization and Eigenvalues

  1. 31 One-Dimensional Optimization: Golden Section and Brent Find the minimum of a function of one variable without derivatives. Bracket, golden-section search, successive parabolic interpolation and Brent's method.
  2. 32 Gradient Descent and Newton's Method for Optimization Minimize functions of many variables with gradient descent, line search, momentum and Newton's method. Compare convergence on the Rosenbrock function with SciPy.
  3. 33 Eigenvalues: Power Iteration and Inverse Iteration Compute dominant and smallest eigenvalues of a matrix with the power method, Rayleigh quotient, shifts and inverse iteration. Includes a PageRank-style example.
  4. 34 The QR Algorithm for All Eigenvalues Compute every eigenvalue of a matrix by repeated QR factorization. Learn Gram-Schmidt QR, shifts, Hessenberg reduction, Jacobi's method and the SVD link.

Projects and Practice

  1. 35 Project: A Projectile with Air Drag Combine ODE solvers, events, root finding and optimization to simulate a cannonball with air drag and find the launch angle that gives the longest range.
  2. 36 Project: Fit, Differentiate and Integrate Sensor Data Build a small numerical pipeline: interpolate missing readings, fit a decay model, estimate rates of change and integrate to total energy, with honest error checks.
  3. 37 Numerical Methods Practice Problems with Solutions Eight worked practice problems on interpolation, fitting, differentiation, integration, ODEs, optimization and eigenvalues, each with a checked Python solution.