Skip to content
Tutorials Center
My learning

Topics

Lesson 19 of 40 ยท Modules and Packages

Packages and Environments

Use Julia's built-in package manager: add packages, read Project.toml and Manifest.toml, activate environments, share reproducible setups and create your own package.

Julia ships with a package manager called Pkg, and it is one of the language's best features. Instead of installing everything into one global pile, Julia lets every project have its own environment: a folder with a list of the packages the project needs and the exact versions that were used. Two people who open the same environment get the same code, and two projects on one machine can use different versions of the same package without conflict.

In this lesson you will install and remove packages, learn what Project.toml and Manifest.toml contain, activate and share environments, understand version bounds, and create a small package of your own. The Julia commands are read-only on this site because the page cannot install packages. The Python stand-ins read a real Project.toml with Python's TOML parser and simulate version-compatibility rules, so you can see the same logic running.

The package manager at a glance

You can use Pkg in two ways. In the REPL, press ] at an empty prompt to enter Pkg mode (the prompt changes to something like (@v1.12) pkg>), and press Backspace to leave it. In a script, call the same actions as functions after using Pkg.

JULIA
# Pkg mode                   # Same thing as code
] add JSON3                  # using Pkg; Pkg.add("JSON3")
] rm JSON3                   # Pkg.rm("JSON3")
] status                     # Pkg.status()
] update                     # Pkg.update()
] add [email protected]          # Pkg.add(name="Example", version="0.5.3")

add downloads the package from the General registry, the public list of registered packages, together with everything it depends on. After that you load it with using JSON3 as in Modules and Scope. The first using after an install is slow, because Julia precompiles the package into a cache; later loads are quick.

The everyday Pkg commands

add Install a package and record it.
rm Remove it from the project.
status List what the project uses.
update Move to newer allowed versions.
1 / 2

What is an environment?

An environment is just a folder that contains a Project.toml file, and usually a Manifest.toml next to it. When you start Julia, it is already in a default environment named after the Julia version, such as @v1.12. Installing everything there is fine for quick experiments and tools like Revise, but a real project should have its own.

JULIA
# in a shell
mkdir weather-report
cd weather-report
julia --project=.             # start Julia using this folder as the environment
JULIA
# or, from inside a running session
using Pkg
Pkg.activate(".")             # use ./Project.toml
Pkg.add(["CSV", "DataFrames"])
Pkg.status()
# Status `~/weather-report/Project.toml`
#   [336ed68f] CSV v0.10.15
#   [a93c6f00] DataFrames v1.7.0

The text in square brackets is the first part of the package's UUID, a unique identifier, so two packages with the same name can never be confused. The version numbers above are only an illustration; you will see whatever is current.

Useful variations:

  • Pkg.activate(temp=true) creates a throw-away environment, ideal for trying a package without touching your project.
  • Pkg.activate() with no argument goes back to the default environment.
  • julia --project (without =.) looks for a Project.toml in the current folder or its parents.
  • ] activate --shared myenv creates a named shared environment under ~/.julia/environments.

Julia also stacks environments: packages in your active project are visible, and so are those in the default environment and the standard library. That is why using Test works almost anywhere: standard libraries such as Test, Random, Statistics and LinearAlgebra ship with Julia and need no download, although a project that relies on them should still list them in [deps].

Project.toml: what you asked for

Project.toml is a small, human-readable TOML file. Pkg edits it when you add or rm a package, and you can edit it by hand too.

TOML
name = "WeatherReport"
uuid = "5b3a1c42-7e2d-4a8e-9f0b-2d6c8e1a9f33"
authors = ["Ada Example <[email protected]>"]
version = "0.1.0"

[deps]
CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[compat]
CSV = "0.10"
DataFrames = "1"
julia = "1.10"

The name, uuid and version entries are only needed if the project is itself a package. The [deps] table lists direct dependencies. The [compat] table says which versions you have tested against, and it is worth filling in for any code you share.

Version bounds

A compat entry like DataFrames = "1" is a caret bound: it allows any version from 1.0.0 up to, but not including, 2.0.0. Julia follows semantic versioning, in which a major version bump means breaking changes. The rule for the leftmost non-zero number is what matters:

Entry Allowed range
"1.2.3" 1.2.3 up to but not including 2.0.0
"0.10" 0.10.0 up to but not including 0.11.0
"0.0.4" exactly 0.0.4
"~1.2" 1.2.0 up to but not including 1.3.0
"=1.2.3" exactly 1.2.3
"1, 2" any 1.x or any 2.x

Packages in the 0.x range are treated more strictly, since by convention their minor version is the breaking one. First, a stand-in that reads a real Project.toml string with Python's TOML parser; a later section turns the table into code.

Python Reading a Project.toml

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Manifest.toml: what you got

When Pkg solves your requirements it writes the result, down to every indirect dependency, into Manifest.toml: exact versions, UUIDs and for many packages a tree hash that identifies the exact source. You never edit this file by hand.

The relationship is like a recipe and a receipt. Project.toml says "any CSV 0.10.x will do". Manifest.toml records "we used 0.10.15, which pulled in these 14 other packages at these versions". Anyone with both files can reproduce your setup exactly.

JULIA
# On another machine, after cloning the project
using Pkg
Pkg.activate(".")
Pkg.instantiate()        # downloads exactly what Manifest.toml lists

Which files should you commit to version control?

  • Applications and analyses: commit both files, for reproducibility.
  • Packages meant to be installed by others: commit Project.toml; many authors leave Manifest.toml out (add it to .gitignore), because each user's resolver will pick versions that suit their own environment. Test with the oldest and newest versions your bounds allow.

Related commands: Pkg.update() moves to the newest versions your bounds allow and rewrites the manifest; Pkg.pin("CSV") freezes one package at its current version; Pkg.resolve() recomputes the manifest after you edit Project.toml by hand; Pkg.gc() deletes unused cached versions to free disk space. From Julia 1.11, a manifest may be named like Manifest-v1.11.toml, which lets one project keep a separate manifest for each Julia version.

The package manager solves for versions with the whole dependency graph in mind. If two packages require incompatible versions of a third, add fails with an "Unsatisfiable requirements" message that names the conflict, instead of silently installing a broken mix.

Simulating version bounds

The caret rules are simple enough to write in a dozen lines. This stand-in implements the table above and checks some versions against it, which is a helpful way to see why a package pinned at 0.10 rejects 0.11.0.

Python Which versions satisfy a compat bound?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The real resolver additionally supports ranges, ~ and = forms, and it picks the newest set of versions that satisfies every package's bounds together.

Creating your own package

Once you have reusable code, turn it into a package. The quickest start is Pkg.generate, which creates the minimal skeleton:

JULIA
using Pkg
Pkg.generate("Greeter")
TEXT
Greeter/
โ”œโ”€โ”€ Project.toml
โ””โ”€โ”€ src/
    โ””โ”€โ”€ Greeter.jl

The generated src/Greeter.jl contains a module with a single function, and the file name must match the module name:

JULIA
module Greeter

export greet

greet(name) = "Hello, $name!"

end # module

To use it while you develop, tell Pkg to track the folder directly rather than a released version:

JULIA
using Pkg
Pkg.activate("MyProject")         # some other project
Pkg.develop(path="Greeter")       # record Greeter as a local dependency

using Greeter
greet("Julia")                    # "Hello, Julia!"

With a developed package, edits to the source take effect immediately (together with the Revise package, even without restarting the session). A package normally grows a test/runtests.jl file, documentation, and, for a published package, a git repository whose tags mark releases. Registering it in the General registry makes Pkg.add("Greeter") work for everyone; the community tool Registrator and the [compat] requirement make that smooth. Packages that only your team needs can be added directly from a git URL: Pkg.add(url="https://example.org/team/Greeter.jl").

Two final details. Test-only dependencies belong in test/Project.toml (or the older [extras]/[targets] tables), and Pkg.test() runs the tests in a clean, temporary environment so a package cannot silently rely on something you installed globally. We cover testing in Testing with Test.jl.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which file records the exact versions of every package, including indirect dependencies?

    Manifest.toml is the receipt of a resolved environment. Project.toml holds your direct dependencies and version bounds.
  2. 2. What does Pkg.instantiate() do?

    After cloning a project you run instantiate to install exactly what Manifest.toml lists. update is the command that changes versions.
  3. 3. A project has [compat] Foo = "0.4". Which version is allowed?

    For 0.x versions the minor number is the breaking one, so 0.4 allows 0.4.0 up to but not including 0.5.0.
  4. 4. How do you start working on an unreleased package from a local folder, so that edits are used immediately?

    develop makes the environment track the folder directly instead of a released copy in the depot. pin freezes a version and gc cleans caches.

Summary

  • Pkg mode (]) and the Pkg functions install, remove, update and list packages from the General registry.
  • Each project should have its own environment: a folder with Project.toml, activated with --project or Pkg.activate.
  • Project.toml lists direct dependencies and [compat] bounds; Manifest.toml records the exact resolved versions, and Pkg.instantiate() recreates them.
  • Compat bounds follow semantic versioning: "1.2" allows up to the next major, and "0.10" allows up to the next minor.
  • Pkg.generate creates a package skeleton and Pkg.develop lets you work on it locally.

Next, learn what happens when Julia runs your code in How Julia Compiles Code.