Skip to content
Tutorials Center
My learning

Topics

Lesson 1 of 45 ยท Getting Started

What Is PyTorch?

Meet PyTorch: a Python deep learning library built on tensors and automatic differentiation. Learn eager execution, the ecosystem, install options and how this track runs.

PyTorch is an open-source Python library for building and training neural networks. It gives you two big things: a tensor type (a multi-dimensional array that can live on a GPU) and automatic differentiation (the machinery that computes gradients for you). Everything else, from layers to optimizers to image models, is built on those two ideas.

This lesson explains what makes PyTorch feel the way it does, what else ships around it, how you install it, and how this tutorial lets you learn it even though real PyTorch cannot run inside your browser.

Eager execution: code that runs as you write it

PyTorch runs operations immediately, line by line, like ordinary Python. When you multiply two tensors, the multiplication happens right then and you can print the answer. This style is called eager execution or "define by run".

Older frameworks used static graphs: you first described the whole computation as a graph object, compiled it, and only then pushed data through. That is fast to optimize but awkward to debug, because your Python line numbers no longer match what is executing.

Python
import torch

x = torch.tensor([1.0, 2.0, 3.0])
y = x * 2 + 1          # runs right now
print(y)                # tensor([3., 5., 7.])

if y.sum() > 10:        # plain Python control flow works
    print("big")

Because the graph is built while your code runs, you can use if, for, print, a debugger or a breakpoint anywhere. The graph can even change shape from one run to the next. You will see how PyTorch records it in Computational Graphs and the Chain Rule.

Two ideas carry PyTorch

Tensors N-dimensional arrays, with GPU support.
Autograd Gradients computed automatically.

Layers, losses and optimizers are built from these two.

1 / 2

The ecosystem around the core

The torch package is the core, but you will meet its neighbours quickly:

  • torch.nn: layers, losses and the Module base class.
  • torch.optim: optimizers such as SGD and Adam.
  • torch.utils.data: Dataset and DataLoader for feeding batches.
  • torchvision: image datasets, transforms and pretrained vision models.
  • torchaudio: audio loading, transforms and models.
  • torchtext-style tools and third-party libraries: Hugging Face Transformers, PyTorch Lightning and many others build on PyTorch.
  • torch.compile: an opt-in feature that captures your model and generates faster code while you keep writing eager-style Python. See Making Training Faster.

None of these change how a tensor behaves. They are all conveniences built on the core you learn first.

How PyTorch compares with other frameworks

PyTorch competes with TensorFlow, Keras, JAX and older frameworks such as Caffe and Apache MXNet. The main differences are that PyTorch runs eagerly (every line executes as you write it, so ordinary Python debugging works), that you usually write the training loop yourself, and that most new research code and pretrained models appear in PyTorch first. The Deep Learning Landscape compares them in detail and also maps the applications and challenges of deep learning.

Installing PyTorch

Install the build that matches your hardware. The official site has a selector that prints the exact command; the usual shapes look like this:

Terminal
# CPU only (works everywhere, fine for learning)
pip install torch torchvision torchaudio

# A GPU build uses an extra index URL chosen on the PyTorch install page
# (for example a CUDA build for NVIDIA GPUs)

Check that it works:

Python
import torch

print(torch.__version__)
print(torch.cuda.is_available())             # True on a working NVIDIA GPU setup
print(torch.backends.mps.is_available())     # True on Apple silicon builds

Use a virtual environment so PyTorch's large dependencies stay away from other projects. If you do not have a GPU, do not worry: every idea in this track works on a CPU, only slower for big models. Free hosted notebooks are another way to get a GPU for experiments.

Version note: PyTorch ships new minor versions several times a year. The core ideas in this track are stable across them. When a function was renamed or a default changed, the lesson says so.

How this track works

Real PyTorch is a large native library with compiled GPU kernels. It cannot run in the browser Python used on this site. So each lesson uses two kinds of code:

  1. Read-only PyTorch blocks (plain code blocks like the ones above) show the real API. Copy them into your own environment to run them.
  2. Live stand-ins (the Try it editors) run real Python in your browser using NumPy. They rebuild the idea from scratch: a tiny autograd class, a layer stack that behaves like nn.Module, an SGD loop. Each lesson says clearly which part is real PyTorch and which is the stand-in.

This is not a compromise; it is a good way to learn. If you have built a gradient by hand, loss.backward() stops being magic. If you have not used NumPy arrays yet, start with Introduction to NumPy, since a PyTorch tensor is very close to a NumPy array. New to Python itself? Begin at Python Introduction.

Here is the first stand-in. It runs eagerly, just like PyTorch: each line computes at once, and a tiny class remembers how a result was made.

Python Eager execution in plain Python

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The history tuple is a very small cousin of PyTorch's graph: it is built as the code executes. Real PyTorch records far more (which tensors fed each operation), which is what lets it run the chain rule backwards.

Eager versus compiled, side by side

Eager code pays a small overhead per operation: Python decides, then a kernel runs. For tiny operations that overhead dominates. Compiling a whole function lets the library fuse many small steps into one. You can see the principle with NumPy: one big vectorized call beats a Python loop doing the same arithmetic.

Python Why batching operations matters

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

PyTorch tensors follow the same rule: you express work on whole tensors, not on single numbers.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does eager execution mean in PyTorch?

    In eager mode each operation executes as soon as Python reaches it, so you can print, branch and debug normally.
  2. 2. Which two ideas form the core of PyTorch?

    Tensors store the data and autograd computes gradients. Layers and optimizers are built on top.
  3. 3. Which package provides image datasets, transforms and pretrained vision models?

    torchvision is the vision companion library. torchaudio handles audio and torch.optim holds optimizers.
  4. 4. Why does this track use NumPy for its live examples?

    Real PyTorch code is shown read-only; the live editors use NumPy stand-ins that show how the ideas work inside.

Summary

  • PyTorch is a Python library built on GPU-capable tensors and automatic differentiation.
  • It uses eager execution: operations run immediately, so normal Python tools and control flow work.
  • The ecosystem adds torch.nn, torch.optim, data utilities, torchvision, torchaudio and the opt-in torch.compile.
  • Install the build that matches your hardware; CPU is enough for learning.
  • On this site PyTorch code is read-only and the live editors use NumPy stand-ins that say what is real and what is not.

Next: create and inspect tensors in Your First Tensors.