Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 39 ยท Projects and Practice

Project: A Parameterized Sales Report Pipeline

Build an automated monthly sales report: tested functions in a module, a thin parameterized notebook, executed per region, exported to HTML, with checks and a summary file.

This project assembles the Reproducibility and Collaboration chapters into one workflow you could run every month: a thin notebook that tells the story, tested functions that do the work, parameters for region and month, data checks that fail loudly, and a batch run that produces one report per region plus a summary.

The real tools (Papermill, nbconvert, pytest) are not runnable on this site, so each step shows them as read-only commands, and the logic runs here as plain Python on a built-in sample dataset. Following along on your machine, you replace the stand-in functions with the real calls.

The design

flowchart TD
  D["sales.csv"] --> L["load_sales + validate"]
  L --> F["monthly_summary(region, month)"]
  F --> N["report.ipynb (thin)"]
  N --> P["papermill: one run per region"]
  P --> H["nbconvert: HTML report per region"]
  P --> S["summary.json"]

Functions hold the logic; the notebook shows it; the batch runner repeats it.

The project layout:

TEXT
sales-report/
  requirements.txt          # pinned (see the pinning lesson)
  src/salesreport/
    __init__.py
    data.py                 # load_sales, validate
    summary.py              # monthly_summary
  tests/test_summary.py
  notebooks/report.ipynb    # parameters: region, month
  run_all.py                # loops over regions
  out/                      # generated, not in Git

Step 1: sample data

Python Step 1: the dataset

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In the real project load_sales reads a file path (csv.DictReader(open(path)) or pandas.read_csv). One row has a negative amount: a refund that the checks below must handle.

Step 2: validate before analyzing

Checks live in a function so they can be tested and reused. They return problems instead of raising on the first one.

Python Step 2: data validation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Decide what each problem means for your business: a refund may be valid and should be counted separately, whereas an unknown region should stop the run. In the notebook, put assert not blocking_problems, blocking_problems right after loading.

Step 3: the logic as a tested function

Python Step 3: monthly_summary with tests

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All three tests pass and print the March summary for the north region: two orders, revenue 32.5, top product coffee. This function goes into src/salesreport/summary.py and the test class into tests/test_summary.py (see moving code into modules).

Step 4: the thin notebook

notebooks/report.ipynb has these cells:

  1. Markdown: # Sales report plus a sentence of purpose.
  2. Parameters cell (tagged parameters):
Python
region = "north"
month = "2024-03"
data_path = "../data/sales.csv"
  1. Setup and load:
Python
from salesreport.data import load_sales, validate
from salesreport.summary import monthly_summary

sales = load_sales(data_path)
problems = validate(sales)
assert not [p for p in problems if "unknown region" in p], problems
  1. summary = monthly_summary(sales, region, month) and a displayed table.
  2. A chart of revenue by product (a bar chart saved to out/{region}-{month}.png).
  3. Markdown with a short interpretation. Because it is generated for many regions, keep the text generic or write it from the numbers: f"{region} earned {summary['revenue']:.2f} in {month}."

Step 5: run it for every region

The batch runner uses Papermill. On your machine:

Python
import json, papermill as pm

regions = ["north", "south", "east"]
month = "2024-03"
for region in regions:
    pm.execute_notebook(
        "notebooks/report.ipynb",
        f"out/report-{region}-{month}.ipynb",
        parameters={"region": region, "month": month},
    )

and then export each to HTML without code:

Terminal
jupyter nbconvert --to html --no-input out/report-*.ipynb

The runnable stand-in below performs the same loop with monthly_summary and writes the summary as JSON, which is what a downstream dashboard would read.

Python Step 5: batch run and summary file

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Step 6: make it reproducible and automated

  • Pin dependencies in requirements.txt (see pinning dependencies) and install them in a fresh environment for each run.
  • Seed any randomness (not needed here) and record versions in the report footer.
  • Fail on problems: Papermill returns an error and the executed notebook shows the failing cell.
  • Schedule with cron or a workflow file, for example on the first day of each month:
YAML
on:
  schedule:
    - cron: "0 6 1 * *"
jobs:
  report:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install -r requirements.txt && pip install -e .
      - run: pytest -q && python run_all.py
      - uses: actions/upload-artifact@v4
        with: { name: reports, path: out/*.html }
  • Store executed notebooks or HTML with the month in the file name so you have an audit trail.

Extensions

  • Add a comparison against the previous month and a percentage change column.
  • Email the HTML reports (never put credentials in the notebook; see notebook security).
  • Add a Voila front end where a manager picks the region and month (see Voila).
  • Add the notebook linter from the previous project to CI.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why keep the notebook thin and put logic in functions?

    A tested function plus a short narrative notebook is easier to trust and to maintain.
  2. 2. How should refunds (negative amounts) be handled in this design?

    A business rule should be explicit and visible in the summary, not hidden.
  3. 3. What does the batch runner pass to each Papermill run?

    The same notebook runs with different parameter values, producing one executed copy per region.
  4. 4. Which step makes the monthly run fail early when the data is wrong?

    Checks directly after loading stop the run with a clear message before bad numbers spread.

Summary

  • A reliable report pipeline is tested functions, a thin parameterized notebook, validation, a batch loop and automated exports.
  • Papermill runs the notebook per region, nbconvert exports HTML, and a scheduler repeats it.
  • Pinned requirements, fixed inputs and archived outputs make each month's report reproducible.

Next: Project: An Experiment Tracker