Skip to content
Tutorials Center
My learning

Topics

PyTorch Tutorial

Learn PyTorch step by step: 45 lessons in 12 chapters with examples, quizzes, diagrams and projects.

Start learning →

of lessons completed

Getting Started

  1. 1 What Is PyTorch? Meet PyTorch: a Python deep learning library built on tensors and automatic differentiation. Learn eager execution, the ecosystem, install options and how this track runs.
  2. 2 The Deep Learning Landscape A map of deep learning for PyTorch learners: how it evolved, where it is used, why it is hard, how to cope, and how PyTorch compares with TensorFlow, Keras, Caffe and MXNet.
  3. 3 Your First Tensors Create PyTorch tensors with tensor, zeros, ones, arange, rand and randn, read shape, dtype and device, set seeds, and compare tensors with NumPy arrays.

Tensors in Depth

  1. 4 Indexing, Slicing and Reshaping Index and slice PyTorch tensors, use boolean masks, and reshape with view, reshape, permute, squeeze, unsqueeze, cat and stack while understanding shared storage.
  2. 5 Tensor Math and Broadcasting Do tensor math in PyTorch: elementwise ops, matmul, reductions with dim and keepdim, broadcasting rules, in-place operations and einsum, with live NumPy demos.
  3. 6 Dtypes, Precision and NumPy Interop Learn PyTorch dtypes (float32, float64, float16, bfloat16, ints), type promotion, to(), and how from_numpy and numpy() share memory, with a live precision demo.

Autograd

  1. 7 Computational Graphs and the Chain Rule Understand how autograd works: forward and backward passes, computational graphs and the chain rule, walked through by hand and rebuilt in a tiny Python Value class.
  2. 8 Autograd in Practice Use PyTorch autograd with confidence: requires_grad, backward(), gradient accumulation, zero_grad, no_grad, detach and retain_graph, with live stand-ins.
  3. 9 Build a Tiny Autograd Tensor Write a NumPy-backed Tensor with add, mul, matmul, relu and sum, a topological backward pass and broadcasting-aware gradients, then train a small network.
  4. 10 Custom Autograd Functions and Hooks Write a torch.autograd.Function with its own backward, verify it with gradcheck, use tensor hooks, higher-order gradients and torch.func, with live checks.

Building Models

  1. 11 Neurons, Perceptrons and Feedforward Networks Learn the ideas behind every PyTorch model: the artificial neuron, the perceptron and its limits, multi-layer perceptrons, forward propagation and the main network types.
  2. 12 Linear Regression the PyTorch Way Fit a line the PyTorch way with data, model, loss, optimizer and the five-line training loop, then run the same loop live in NumPy gradient descent.
  3. 13 Modules, Parameters and Layers Understand nn.Module, forward, nn.Linear, parameters, named_parameters, state_dict, Sequential, ModuleList and train/eval mode, with a live NumPy layer stack.
  4. 14 Activation Functions and Loss Functions Learn ReLU, GELU, sigmoid, tanh and softmax, plus MSELoss, CrossEntropyLoss and BCEWithLogitsLoss, logits vs probabilities and numerical stability, live.
  5. 15 Initialization, Normalization and Dropout Learn how Xavier and Kaiming initialization, BatchNorm, LayerNorm and Dropout keep deep networks trainable, and how train and eval modes change them.

Training

  1. 16 Optimizers: SGD, Momentum, Adam Understand torch.optim: how SGD, momentum, RMSprop, Adam and AdamW update weights, what weight decay does, and why zero_grad and step order matter.
  2. 17 Learning Rate Schedules and Gradient Clipping Learn StepLR, cosine, warmup, OneCycle and ReduceLROnPlateau in PyTorch, how clip_grad_norm_ stops exploding gradients, and how to choose a learning rate.
  3. 18 Datasets and DataLoaders Learn how PyTorch's Dataset and DataLoader feed data to a model: batch_size, shuffle, num_workers, collate_fn, samplers, random_split and IterableDataset.
  4. 19 The Complete Training Loop Write a complete PyTorch training loop: epochs and batches, train/validation split, correct metric averaging, eval under no_grad, early stopping and logging.
  5. 20 Overfitting and Regularization Understand bias, variance and overfitting with live examples, then fix them with weight decay, dropout, data augmentation and early stopping in PyTorch.

Computer Vision

  1. 21 Convolutions and CNNs Learn how convolutions work and how to build a small CNN in PyTorch: Conv2d, kernels, stride, padding, pooling, output shapes and receptive fields.
  2. 22 CNN Architectures, Pooling and ResNet Tour the landmark CNN designs from LeNet to ResNet, understand pooling types and residual connections, and see why skip connections fix the degradation problem.
  3. 23 Image Classification Pipeline Build an image classification pipeline: torchvision datasets, transforms, normalization, a small CNN, evaluation and a confusion matrix, with live stand-ins.
  4. 24 Transfer Learning and Fine-Tuning Reuse pretrained networks in PyTorch: load weights, freeze layers, replace the classifier head, set discriminative learning rates and decide when to fine-tune.

Sequences and Transformers

  1. 25 RNNs, LSTMs and GRUs Understand recurrent networks in PyTorch: hidden state, nn.RNN, nn.LSTM, nn.GRU, batch_first, padding and packing, and why gradients vanish, with live NumPy stand-ins.
  2. 26 Embeddings and Text Data Prepare text for PyTorch: tokenization, vocabularies, padding and masks, nn.Embedding lookups, mean pooling and a text classifier skeleton with live NumPy stand-ins.
  3. 27 Attention from Scratch Understand scaled dot-product attention, queries, keys and values, masks, multi-head attention and positional encodings, with PyTorch code and live NumPy stand-ins.
  4. 28 Building a Transformer Block Assemble a transformer block in PyTorch: attention, residual connections, LayerNorm, feed-forward layers, a causal LM head and parameter counting.
  5. 29 Fine-Tuning, PEFT and LoRA Learn how to adapt large pretrained models cheaply: full fine-tuning, parameter-efficient methods, a LoRA layer written in PyTorch, adapters and prompt tuning.

Generative Models

  1. 30 Autoencoders and Variational Autoencoders Build autoencoders in PyTorch: encoder, bottleneck and decoder, reconstruction loss, denoising, sparse and convolutional variants, and the variational autoencoder.
  2. 31 Generative Adversarial Networks (GANs) Understand GANs and build one in PyTorch: generator and discriminator, the minimax loss, a training loop, common failures, and variants such as cGAN, WGAN, CycleGAN and StyleGAN.

Saving, Speed and Hardware

  1. 32 Saving and Loading Models Save and load PyTorch models safely: state_dict, torch.save and torch.load, weights_only, resumable checkpoints with optimizer state, ONNX and TorchScript.
  2. 33 GPUs, Devices and Mixed Precision Run PyTorch on CPU, CUDA or Apple GPUs: .to(device), pinned memory, autocast, GradScaler, bfloat16 and memory use, with live float16 precision demos.
  3. 34 Making Training Faster Find and fix PyTorch bottlenecks: profiling, DataLoader workers, torch.compile, channels_last, gradient accumulation and benchmarking, with live NumPy demos.
  4. 35 Distributed Training Overview Understand PyTorch multi-GPU training: DataParallel versus DistributedDataParallel, process groups, gradient synchronization, FSDP and torchrun, with a NumPy simulation.

Debugging and Practice

  1. 36 Debugging PyTorch Code Find PyTorch bugs fast: shape and dtype errors, device mismatches, NaN losses, vanishing gradients, anomaly detection, overfitting one batch and seeds.
  2. 37 Testing and Deploying Models Write unit tests for models and losses, check determinism and eval mode, then learn the options for serving, quantizing and exporting a PyTorch model.

Projects

  1. 38 Project: A Neural Network Without PyTorch Build a classifier from scratch: a NumPy autograd Tensor, a Module and Linear layers, softmax cross-entropy and Adam, then compare with the PyTorch code.
  2. 39 Project: Handwritten Digit Classifier Build a handwritten digit classifier end to end: the PyTorch CNN pipeline, then a live scikit-learn version with a confusion matrix and an error gallery.
  3. 40 Project: Sentiment Classifier Build a sentiment classifier: tokenize, build a vocabulary, pad, embed, pool and train. PyTorch code is read-only, with live NumPy and scikit-learn versions.
  4. 41 Project: A Character-Level Language Model Build a character-level language model: count a bigram table, train the same model with gradients, add context, and sample text with temperature.
  5. 42 Project: Forecasting a Time Series Forecast a time series: build sliding windows, set honest baselines, split by time, and compare a PyTorch LSTM (read-only) with a live autoregressive model.

Practice and Review

  1. 43 Practice Exercises with Solutions Eight PyTorch exercises with worked solutions: tensor shapes, broadcasting, gradients by hand, module design, loop bugs and conv shapes, with live NumPy checks.
  2. 44 PyTorch Interview Questions Prepare for PyTorch interviews: tensors vs arrays, autograd, zero_grad, eval vs no_grad, DataLoader, BatchNorm, mixed precision and common pitfalls, with demos.
  3. 45 PyTorch Cheat Sheet A one-page PyTorch cheat sheet: tensor creation, shapes, autograd, nn layers, losses, optimizers, a training loop template, device handling and saving models.