Skip to content
Tutorials Center
My learning

Topics

Lesson 64 of 72 ยท Projects

Project: Wikipedia and Weather Search App

Build a Flask app that calls other websites' APIs. Use requests with timeouts, handle failures with clear 502 and 504 responses, cache results with a TTL and test it without the network.

Most real applications talk to other services: weather, maps, payments, search, your company's other systems. This project builds a small app with two features, a search box that looks things up in a Wikipedia-style search API and a weather page that asks a forecast API for a city. The feature list is simple. The lesson is in the plumbing: how to call a remote API from Flask safely and politely.

You will build:

  • a small client class around requests with a shared session, a timeout and an identifying User-Agent,
  • error handling that turns every failure into a clear response instead of a crash,
  • an in-memory cache with an expiry time, so repeated searches do not hit the remote API again,
  • tests that never touch the network.

The editors on this site cannot make real network requests from Python. To keep every example runnable, the code uses requests with a fake transport (a mounted adapter that returns prepared responses and can simulate timeouts and errors). Your own code does not change: remove the fake adapter and the same client talks to the real services. The endpoint paths and JSON shapes below mimic the public Wikipedia search API and the Open-Meteo forecast API in simplified form.

The shape of the app

flowchart LR
    B["Browser"] -- "GET /search?q=flask" --> V["Flask view"]
    V --> C{"In cache and fresh?"}
    C -- yes --> R["Render results"]
    C -- no --> K["ApiClient.get_json()<br/>timeout, status check"]
    K -- "HTTP" --> X["Remote API"]
    X --> K
    K --> S["Store in cache"] --> R
    K -- "timeout / error" --> E["502 or 504 page"]

The view never talks to the network directly. All remote calls pass through one client, which makes timeouts, logging and testing simple.

Rules for calling other APIs

Remote calls are the slowest and least reliable thing your app does. A few rules prevent most disasters:

  1. Always set a timeout. requests waits forever by default. A hung remote server then ties up one of your workers for every visitor. Use a (connect, read) tuple such as timeout=(3.05, 5).
  2. Check the status. response.raise_for_status() turns 4xx and 5xx into exceptions.
  3. Expect bad data. The response may not be JSON, or a field may be missing. Validate what you use.
  4. Never trust it in HTML. Text from an API is untrusted input. Jinja autoescape handles that if you do not use |safe.
  5. Cache. The same question asked twice should cost one remote call. It is faster for you and kinder to the provider.
  6. Respect the provider. Read its terms, send a meaningful User-Agent, stay within rate limits, and keep API keys in environment variables (Configuration), never in source code or URLs you log.
  7. Do not call from the request if you can avoid it. Anything slow or bulk belongs in a background job.

Step 1: the client

One class owns the requests.Session (which reuses connections), the base URL, the timeout and the translation of low-level errors into your own exceptions. The rest of the app catches only UpstreamError and UpstreamTimeout.

The fake transport below is the only part that differs from production. It answers the two APIs with prepared JSON and can be told to fail, so we can watch every error path.

Python An API client with timeout, status checks and a fake transport

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that params= builds and percent-encodes the query string for you (never glue user text into URLs by hand), and that raise ... from exc keeps the original error attached for your logs.

For flaky services add automatic retries with a backoff to the session, but only for idempotent requests such as GET (this is read-only code, since it needs a real network):

Python
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

retry = Retry(total=3, backoff_factor=0.5, status_forcelist=[429, 502, 503, 504], allowed_methods=["GET"])
session.mount("https://", HTTPAdapter(max_retries=retry))

Step 2: a cache with an expiry time

A cache stores the answer to a question for a limited time. A plain dictionary with timestamps works for one process and is a good start. In production, share the cache between workers with Redis or Flask-Caching.

The cache below is tiny but has the important parts: a TTL (time to live), a maximum size so memory cannot grow forever, and a clock argument so tests can move time forward without sleeping.

Python A TTL cache with a size limit

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Cache normalized keys: " Flask " and "flask" should be one entry, so lower-case and strip the query. And do not cache errors for long (or at all), otherwise a short outage is remembered for the whole TTL.

Step 3: the Flask app

Now the views. A helper fetch_cached is the only place that combines the cache and the client. Every upstream failure becomes a friendly error: 504 Gateway Timeout when the remote server is too slow, 502 Bad Gateway when it fails or returns nonsense. Those codes tell monitoring systems that your app is fine and a dependency is not.

Python The complete app, with tests against the fake network

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the test output as a checklist of behaviors you now know the app has: the remote snippet's <script> was escaped; three differently written searches made exactly one remote call; a missing parameter is a 400 and an unknown city a 404; a timeout becomes a 504 with a friendly message; and the app recovers when the remote service does, because failures are not cached.

Ideas to extend

  • Replace TTLCache with Flask-Caching on Redis so all workers share it.
  • Add retries with backoff and a circuit breaker that stops calling a failing service for 30 seconds.
  • Move calls to a background job and show "loading" with the streaming and SSE techniques.
  • Show a nicer page: a Jinja template with the forecast, and a search form with <form method="get"> (HTML forms).
  • Log each upstream call with its duration (Logging and Debugging).
  • Add rate limiting on your own endpoints so visitors cannot use your app to hammer the provider.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why must every requests call have a timeout?

    requests has no default timeout. A hung remote service would block workers and eventually your whole app.
  2. 2. Which status code best describes 'a service we depend on answered with an error'?

    502 says your app is fine but the upstream failed; 504 says the upstream was too slow.
  3. 3. Why use params={'srsearch': q} instead of formatting q into the URL string?

    Characters such as & and # have meaning in URLs. params= encodes them safely.
  4. 4. A weather API fails once. Why should you not cache that failure for the full TTL?

    Cache successful results. If you cache an error, use a very short time.

Summary

  • Wrap remote calls in one client class with a session, a timeout, a User-Agent, raise_for_status() and your own exception types.
  • Map failures to clear responses: 504 for timeouts, 502 for bad or failed upstream answers, 400/404 for bad user input.
  • Cache normalized keys with a TTL and a size limit; do not cache errors.
  • Escape remote text (Jinja autoescape), validate shapes, keep API keys in the environment, and respect provider limits.
  • Test with a fake transport so tests are fast, deterministic and offline.

Next: scrape a page when no API exists, politely, in Project: Web Scraping API.