Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 38 ยท pyproject and Metadata

src Layout and Package Data

Compare src and flat project layouts, learn how packages are discovered, include data files, and read them reliably with importlib.resources, with live examples.

Where you put your files decides whether your package works after installation, not just on your own machine. Two layouts dominate: the flat layout and the src layout. This lesson explains the difference, how a build backend decides which files go into the wheel, and how to ship and read non-code files such as templates, JSON data or SQL scripts.

Two layouts

TEXT
flat layout                      src layout
-----------                      ----------
weather-cli/                     weather-cli/
  pyproject.toml                   pyproject.toml
  weather_cli/                     src/
    __init__.py                      weather_cli/
    main.py                            __init__.py
  tests/                               main.py
    test_main.py                   tests/
                                     test_main.py

In the flat layout, the package sits next to pyproject.toml. When you run pytest or python from the project root, the current directory is on sys.path, so import weather_cli finds your source folder, not the installed package. Tests may pass even though the built wheel is missing a file, because the test never touched the wheel.

In the src layout the package is one level down, so the project root is not an import location. To import weather_cli you must install it (usually editable, pip install -e .). That forces your tests to exercise what users will get, and catches missing files and wrong package lists early. For this reason most guides and templates recommend the src layout for libraries. Flat layouts remain fine for small applications and scripts.

Python Why the flat layout can hide packaging bugs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

How packages are discovered

A build backend must know which directories are packages to include. There are two strategies.

Explicit (recommended):

TOML
# hatchling
[tool.hatch.build.targets.wheel]
packages = ["src/weather_cli"]

# setuptools
[tool.setuptools.packages.find]
where = ["src"]
include = ["weather_cli*"]

Automatic discovery. setuptools and others detect a single top-level package in src/ or the root, and exclude things like tests, docs and examples. Automatic rules are convenient and change between versions, so large projects often prefer explicit lists.

The following example implements the simplest discovery: walk a tree and collect directories that contain __init__.py. This is essentially find_packages.

Python Discovering packages in a tree

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that weather_cli/data/ has no __init__.py, so it is not a package, only a folder inside one. Its JSON file will be dropped from a wheel unless you say otherwise.

Package data

Package data means any non-Python file that your code reads at runtime: templates, icons, JSON, SQL, certificates, type stubs (py.typed).

  • hatchling includes everything under the package directories that Git tracks (it respects .gitignore), so data files usually "just work". Use [tool.hatch.build.targets.wheel] include/exclude to adjust.
  • setuptools needs a hint: include-package-data = true (the default in pyproject mode) plus a MANIFEST.in or [tool.setuptools.package-data]:
TOML
[tool.setuptools.package-data]
weather_cli = ["data/*.json", "templates/*.html", "py.typed"]
  • flit-core includes all files inside the module directory.

Always verify by listing the built wheel. A wheel is a zip file, so python -m zipfile -l dist/*.whl shows what shipped. In Building an sdist and a Wheel you will create one yourself.

Marking a package as typed

Add an empty file named py.typed to your package and include it, and type checkers will use your inline annotations from the installed package (PEP 561). Forgetting to ship it is one of the most common typing papercuts.

Reading data files correctly

Never open data files relative to the current working directory, and avoid os.path.dirname(__file__) tricks: they break inside zip archives and some installers. Use importlib.resources, which works wherever the package lives.

Python Reading package data with importlib.resources

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

resources.files() returns a Traversable, which behaves like a path (/, joinpath, iterdir, read_text, read_bytes, is_file) but also works for packages inside zip files. as_file gives a temporary real path when an API insists on one.

A minimal src-layout project

TEXT
weather-cli/
  pyproject.toml
  README.md
  LICENSE
  src/
    weather_cli/
      __init__.py          # __version__, public API
      main.py              # def run(): ...
      py.typed
      data/cities.json
  tests/
    test_main.py
TOML
[build-system]
requires = ["hatchling>=1.25"]
build-backend = "hatchling.build"

[project]
name = "weather-cli"
dynamic = ["version"]
requires-python = ">=3.10"
dependencies = ["requests>=2.30"]

[project.scripts]
weather = "weather_cli.main:run"

[tool.hatch.version]
path = "src/weather_cli/__init__.py"

[tool.hatch.build.targets.wheel]
packages = ["src/weather_cli"]
flowchart LR
    A["src/weather_cli/<br/>code + data"] -- "backend collects<br/>listed packages" --> B["wheel<br/>weather_cli/ + .dist-info"]
    B -- "pip install" --> C["site-packages/weather_cli"]
    C -- "importlib.resources" --> D["data read at runtime"]

Only what the backend includes reaches site-packages, so the code must read files through the package, not through your source tree.

After each build, unzip or list the wheel and compare it with your expectations. Missing data files are by far the most common cause of "works on my machine, fails after pip install".

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why do many projects recommend the src layout for libraries?

    With src layout, the code is only importable after installation (usually editable), which mirrors what users run.
  2. 2. What is the recommended way to read a data file shipped inside a package?

    importlib.resources works regardless of where the package is installed, even inside a zip.
  3. 3. What does an empty py.typed file in a package signal?

    PEP 561: py.typed marks a package as typed. It must be included in the wheel.
  4. 4. How can you check which files are in a wheel?

    A wheel is a zip archive, so any zip tool can list its content.

Summary

  • Flat layouts put the package at the project root; src layouts put it under src/ so only the installed package is importable.
  • Backends include packages either by explicit lists or by automatic discovery; prefer explicit for libraries.
  • Package data must be included by backend configuration and verified by listing the wheel.
  • Read data with importlib.resources.files(), never relative to the working directory.
  • Add and ship py.typed for typed libraries.
  • Editable installs (pip install -e .) make src layouts convenient during development.

Next: Building an sdist and a Wheel