Skip to content
Tutorials Center
My learning

Topics

Lesson 38 of 39 ยท Projects and Practice

Project: Price Tracker

Build a price-tracking scraper: extract prices from product pages, keep a history in SQLite, detect drops, handle failures and send alerts. Runnable offline project.

A price tracker is the classic scraping project, because it combines everything: fetching pages on a schedule, extracting a number from messy HTML, remembering past values, and telling you when something interesting happens. In this project you build the core of one. It watches a few products, records each price in a history table, and raises an alert when a price drops below your target or falls sharply.

As in the previous project, the "website" is a set of HTML strings so everything is runnable and reproducible. The design is the real one: swap in a real fetch for a site you are allowed to scrape and you have a working tool.

Be careful with real shops. Many forbid automated access in their terms, and some have official price feeds or APIs. Check Legal and ethical scraping first, keep the check frequency low (once or twice a day is plenty), and never use a tracker to hammer a site.

Design

flowchart TD
  W["watchlist: product URL + target price"] --> F["fetch page"]
  F --> P["extract price (JSON-LD first, CSS fallback)"]
  P --> H["insert into price history"]
  H --> C{"Below target or big drop?"}
  C -- "yes" --> A["alert"]
  C -- "no" --> N["nothing to do"]
  F -- "error" --> E["log failure, keep going"]

Three decisions make the tracker dependable:

  1. Extract defensively. Try the most stable source first (structured data in JSON-LD), then fall back to a CSS selector.
  2. Record every observation, not only the latest price, so you can see trends and detect drops.
  3. One failure must not stop the run. Each product is processed inside its own try.

Step 1: extract the price

Product pages often include JSON-LD with the price. When it is missing, parse the visible element. This function tries both and returns a number or None:

Python Extract a price with fallbacks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Step 2: history table and alert rules

The tracker's memory is a table with one row per product and day. An alert fires when the new price is below the target, or at least 10% under the previous observation. The function is pure (inputs in, decision out), which makes it easy to test.

Python History and alert rules

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last day triggers two reasons at once: below the target of 80 and down from 91. Reading previous from the database (the observation before today) means re-running the tracker on the same day cannot compare a price with itself.

Step 3: the full run

Now everything together: a fake shop, a watchlist, a run_once(day) function that processes each product in its own try, stores history and collects alerts. Product /p/broken fails on purpose to prove that one failure does not stop the others.

Python The complete tracker, run on three days

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

On the third day the shoe falls below its target (and by more than 10%), the hat falls by more than 10% from the previous day, and the broken product is reported as a failure on every day without affecting the rest.

Taking it live

Moving from this simulation to a tracker that runs by itself:

  • Fetch: a requests.Session with an honest User-Agent, timeouts, and retry with backoff (Rate limiting and retries).
  • Schedule: run once or twice a day with cron (Linux/macOS), Task Scheduler (Windows) or a scheduled CI job.
  • Alerts: print to the log, write an email with smtplib, or post to a chat webhook. Keep credentials in environment variables.
  • Dynamic prices: if the price appears only after JavaScript runs, look for the JSON endpoint first (JSON APIs), and use a browser only if you must (Playwright basics).
  • Currency and regional prices: store the currency with the price, and note which country or store view you scraped.
  • Sanity checks: a price of 0 or one that is 90% lower than yesterday is often a scraping bug, not a bargain. Require a second confirmation before alerting on extreme jumps.
  • Charts: load history into pandas and plot the lines (Saving to CSV, JSON and SQLite).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why try JSON-LD before parsing the visible price element?

    Layout changes often break CSS selectors, while structured data tends to stay the same.
  2. 2. Why store every observation instead of only the latest price?

    History makes drop detection and trends possible.
  3. 3. Why is each product processed inside its own try/except?

    Isolating failures keeps the other products tracked; the failure is still reported.
  4. 4. A price suddenly shows 0.00. What is the best response?

    Extreme values are usually parsing problems; add sanity checks.

Summary

  • A tracker is: watchlist, fetch, extract defensively, record history, decide, alert.
  • Prefer structured data, fall back to selectors, and return None rather than guess.
  • Compare against the previous observation and a target; keep the decision logic pure.
  • Isolate failures per product and sanity-check extreme prices.
  • Run it gently, once or twice a day, and only on sites that permit it.

Next: Practice Exercises