Skip to content
Tutorials Center
My learning

Topics

Lesson 98 of 149 ยท Beyond the Basics

Packaging and Publishing a Library

Learn to package a Python library. Use the src layout, write pyproject.toml, build a wheel and sdist with build, test the install, and publish to PyPI with twine.

You wrote a useful module. Now a teammate wants it, or you want to reuse it in three projects, or you want the whole world to be able to type pip install your-library. Copying files around does not scale. Packaging turns your code into a standard bundle that pip understands, with a name, a version, a list of dependencies and optional command-line tools.

In this lesson you will lay out a project, describe it in pyproject.toml, build the two distribution files (a wheel and an sdist), test them in a clean environment, and publish to PyPI. The build and upload commands run on your own computer, so they appear in read-only blocks (every command on this page was run for real with the project shown). If you have not used pip and virtual environments yet, read Packages and pip and Virtual Environments first.

What a package is made of

A module is one .py file; a package is a folder of modules with an __init__.py (see Modules). A distribution is what you upload and install: a file that contains one or more packages plus metadata. The name you pip install (the distribution name) does not have to match the name you import, though it usually should.

We will build a tiny library that turns titles into URL slugs. This is the layout, called the src layout:

TEXT
slugkit/
โ”œโ”€โ”€ pyproject.toml
โ”œโ”€โ”€ README.md
โ”œโ”€โ”€ src/
โ”‚   โ””โ”€โ”€ slugkit/
โ”‚       โ”œโ”€โ”€ __init__.py
โ”‚       โ”œโ”€โ”€ core.py
โ”‚       โ”œโ”€โ”€ cli.py
โ”‚       โ””โ”€โ”€ py.typed
โ””โ”€โ”€ tests/
    โ””โ”€โ”€ test_core.py

Putting the package under src/ has a practical benefit: when you run tests from the project folder, Python cannot accidentally import your half-finished source folder instead of the installed package. You are forced to test what users will really receive. The files:

Python
# src/slugkit/core.py
import re


def slugify(title: str) -> str:
    """Turn a title into a URL-friendly slug."""
    words = re.findall(r"[a-z0-9]+", title.lower())
    return "-".join(words)
Python
# src/slugkit/__init__.py
"""slugkit: tiny slug helper."""

from slugkit.core import slugify

__all__ = ["slugify"]
__version__ = "0.1.0"
Python
# src/slugkit/cli.py
import sys

from slugkit.core import slugify


def main() -> None:
    print(slugify(" ".join(sys.argv[1:])))

__init__.py re-exports the public function, so users write from slugkit import slugify. The empty file py.typed tells type checkers such as mypy that your package ships type hints.

flowchart LR
    A["Source code<br/>+ pyproject.toml"] -->|"python -m build"| B["sdist<br/>.tar.gz"]
    A -->|"python -m build"| C["wheel<br/>.whl"]
    B --> D["PyPI"]
    C --> D
    D -->|"pip install"| E["Users' environments"]

You build two files from the same source, upload them to PyPI, and pip downloads the matching one for each user.

pyproject.toml

Everything about the project lives in one TOML file, pyproject.toml. It has two main parts: [build-system] says how to build the project, and [project] describes what it is.

TOML
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "slugkit-demo"
version = "0.1.0"
description = "Turn titles into URL-friendly slugs."
readme = "README.md"
requires-python = ">=3.10"
license = "MIT"
authors = [{ name = "Ada Lovelace", email = "[email protected]" }]
classifiers = [
    "Programming Language :: Python :: 3",
    "Operating System :: OS Independent",
]
dependencies = []

[project.optional-dependencies]
dev = ["pytest>=8"]

[project.scripts]
slugkit = "slugkit.cli:main"

[project.urls]
Homepage = "https://example.com/slugkit"

[tool.hatch.build.targets.wheel]
packages = ["src/slugkit"]

The pieces:

  • build-system names the build backend, the tool that actually creates the files. hatchling, setuptools, flit-core and poetry-core all work, and all read the same [project] table. Users never need to know which one you chose.
  • name is what people type after pip install. Names are case-insensitive, and runs of -, _ and . count as the same character. Pick a name that is not already taken on PyPI.
  • version should follow semantic versioning: MAJOR.MINOR.PATCH. Bump it for every release; PyPI never lets you upload the same version twice.
  • requires-python stops pip from installing the library on interpreters you do not support.
  • dependencies lists what your code imports, with ranges such as "requests>=2.31,<3". Be as loose as you can, because tight pins make life hard for people who install several libraries together.
  • optional-dependencies defines extras: pip install "slugkit-demo[dev]" also installs pytest.
  • [project.scripts] creates a command. The line slugkit = "slugkit.cli:main" means "install a program called slugkit that imports slugkit.cli and calls main()".
  • [tool.hatch.build.targets.wheel] is backend-specific. Here the project name (slugkit-demo) differs from the package folder (slugkit), so hatchling needs to be told where the code is. Without it the build fails with "Unable to determine which files to ship inside the wheel".

Since Python 3.11 the standard library can read TOML, which makes it easy to see how these tables turn into data:

Python Read pyproject.toml with tomllib

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

tomllib only reads TOML; build tools do the same with your file. The module:function pair is called an entry point. The next example resolves one the way an installed script does:

Python How an entry point becomes a command

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Developing: editable installs

While you work on the library, install it in editable mode inside a virtual environment. Python then imports your source files directly, so edits take effect without reinstalling:

TEXT
python -m venv .venv
.venv\Scripts\activate          # macOS/Linux: source .venv/bin/activate
python -m pip install -e ".[dev]"
python -m pytest
slugkit "Hello, Packaging World!"
TEXT
hello-packaging-world

-e means editable, . means "the project in this folder", and [dev] adds the extra. Because of the src layout, pytest tests the installed package, just as users will use it. See Testing with pytest for the testing side.

Building the distribution files

Install the standard build front end and run it in the project folder:

TEXT
python -m pip install build
python -m build
TEXT
Successfully built slugkit_demo-0.1.0.tar.gz and slugkit_demo-0.1.0-py3-none-any.whl

The dist/ folder now contains two files:

File What it is
slugkit_demo-0.1.0.tar.gz the sdist (source distribution): your source, tests and metadata, so others can rebuild it
slugkit_demo-0.1.0-py3-none-any.whl the wheel: a ready-to-install, pre-built archive

Pip prefers wheels because installing one is just unpacking files; no build step runs on the user's machine. The wheel's file name encodes who can install it: distribution name, version, Python tag (py3), ABI tag (none) and platform tag (any). py3-none-any means pure Python that works anywhere. Libraries with compiled extensions get tags such as cp314-cp314-win_amd64 and need one wheel per platform.

A wheel is nothing magical: it is a ZIP archive. Listing the one we built shows exactly what pip will unpack:

TEXT
slugkit/__init__.py
slugkit/cli.py
slugkit/core.py
slugkit/py.typed
slugkit_demo-0.1.0.dist-info/METADATA
slugkit_demo-0.1.0.dist-info/WHEEL
slugkit_demo-0.1.0.dist-info/entry_points.txt
slugkit_demo-0.1.0.dist-info/RECORD

Your package folder sits at the top, next to a .dist-info folder holding the metadata (including the entry_points.txt that creates the slugkit command). The src/ folder is gone, and tests are not shipped in the wheel. You can look inside one yourself, in memory, with zipfile and the file-name structure:

Python A wheel is a ZIP file

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Testing the build before you publish

Never upload a file you have not installed. Create a brand-new virtual environment, install the wheel from dist/, and try it away from your source folder:

TEXT
python -m venv fresh
fresh\Scripts\pip install dist\slugkit_demo-0.1.0-py3-none-any.whl
fresh\Scripts\slugkit "Hello, Packaging World!"
TEXT
hello-packaging-world

Also run python -m pip install twine followed by python -m twine check dist/*. It validates that the long description will render on PyPI:

TEXT
Checking dist/slugkit_demo-0.1.0-py3-none-any.whl: PASSED
Checking dist/slugkit_demo-0.1.0.tar.gz: PASSED

Common problems at this stage: a missing README.md that readme = ... points to, data files that were not included (non-Python files must be listed in your backend's include settings), and imports that only worked because the source folder was on sys.path.

Publishing to PyPI

PyPI (the Python Package Index, pypi.org) is the public repository pip uses by default. There is also TestPyPI (test.pypi.org), a separate sandbox where you can rehearse an upload without polluting the real index. The steps:

  1. Create an account on TestPyPI and PyPI and enable two-factor authentication.
  2. Create an API token for your account (a token starting with pypi-). Tokens are secrets: keep them out of your repository and shell history (see Secure Python).
  3. Upload to TestPyPI first: python -m twine upload --repository testpypi dist/*. Twine asks for a username (__token__) and the token as the password.
  4. Check the project page, then try installing it from there in a clean environment.
  5. Upload to the real index: python -m twine upload dist/*.

Once a version is published it cannot be replaced, only yanked (hidden from new installs). If you made a mistake, fix it and release the next version, such as 0.1.1.

Many projects skip tokens entirely and use trusted publishing: PyPI trusts a specific GitHub Actions workflow in your repository, so a release is created by pushing a tag, with no long-lived secret anywhere. Setting it up is a form on the project's PyPI page plus a short workflow file, and it is the recommended way once you release regularly.

Package names are public and permanent. Do not upload anything containing credentials, personal data or private code. A published release stays on the index and in mirrors even after you delete the project.

A release checklist

Use this routine for every release:

  1. Update the version (and a changelog entry).
  2. Run the tests and the type checker.
  3. Delete old files in dist/, then run python -m build.
  4. Install the wheel in a clean environment and run a smoke test.
  5. twine check, then upload, then tag the commit in Git (for example v0.1.0).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does the [build-system] table in pyproject.toml do?

    requires and build-backend tell build front ends such as python -m build which backend to install and call. Runtime dependencies belong in [project].
  2. 2. What is the difference between a wheel and an sdist?

    Pip prefers a wheel, which it just unpacks. The sdist is a source archive used when no suitable wheel exists and for archiving.
  3. 3. What does slugkit = "slugkit.cli:main" under [project.scripts] do?

    An entry point has the form module:function. The installer generates a small launcher script that imports the module and calls the function.
  4. 4. In a wheel named slugkit_demo-0.1.0-py3-none-any.whl, what does none-any tell you?

    The ABI tag none and platform tag any mean there is no compiled code, so one wheel serves every platform that has a compatible Python.
  5. 5. You uploaded version 0.1.0 with a bug. What is the right fix?

    PyPI never accepts the same version twice. You can yank the bad release, but the fix has to ship as a new version.

Summary

  • A package is a folder of modules; a distribution is the installable archive with metadata. The src layout makes you test what users will get.
  • pyproject.toml has [build-system] (the backend) and [project] (name, version, requires-python, dependencies, extras, scripts, URLs).
  • Entry points (module:function) turn functions into installed commands.
  • pip install -e ".[dev]" gives an editable development install; python -m build creates an sdist and a wheel in dist/.
  • A wheel is a ZIP file whose name encodes the Python, ABI and platform tags; py3-none-any means pure Python.
  • Test the wheel in a fresh environment, run twine check, rehearse on TestPyPI, then twine upload (or use trusted publishing). Versions are permanent, so ship fixes as new versions.

Next: Secure Python: Secrets, Hashing and Injection.