Skip to content
Tutorials Center
My learning

Topics

Deep Learning Tutorial

Learn Deep Learning step by step: 73 lessons in 9 chapters with examples, quizzes, diagrams and projects.

Start learning →

of lessons completed

First Steps

  1. 1 What Is Deep Learning? Learn what deep learning is, how it differs from machine learning, what a neural network learns, and when it wins, with runnable NumPy examples.
  2. 2 The Math You Need A practical refresher of the math behind neural networks: vectors, matrices, dot products, derivatives, gradients and the chain rule, checked with NumPy.
  3. 3 The Artificial Neuron Build an artificial neuron in NumPy: inputs, weights, bias, weighted sum and activation. Plot its decision boundary and run it on a whole batch at once.
  4. 4 The Perceptron Learning Rule Implement the perceptron algorithm in NumPy, teach it AND and OR, watch its decision boundary move, and see why XOR is impossible for a single neuron.

Neural Networks From Scratch

  1. 5 Layers and the Forward Pass Stack dense layers in NumPy: weight matrix shapes, batches as rows, a two-layer forward pass, parameter counts, and why depth needs a non-linearity.
  2. 6 Activation Functions Compare sigmoid, tanh, ReLU, leaky ReLU, GELU and softmax in NumPy. Plot each with its derivative, see saturation and dying ReLU, and learn how to choose.
  3. 7 Loss Functions Learn how loss functions score a network's mistakes: mean squared error, binary and categorical cross-entropy, and stable log-softmax, with NumPy code.
  4. 8 Gradient Descent Learn gradient descent from scratch: the update rule, learning rate effects, loss curves, 2D paths, and batch versus stochastic versus mini-batch, in NumPy.
  5. 9 Backpropagation by Hand Work through backpropagation on a tiny network with real numbers: computational graph, chain rule, forward and backward values, checked with code.
  6. 10 Backpropagation in NumPy Turn backpropagation into matrix code: gradients for dense layers, ReLU and softmax cross-entropy, a reusable Layer class and the shape rules that catch bugs.
  7. 11 Gradient Checking Verify a hand-written backward pass with finite differences: centered differences, relative error, per-parameter checks, tolerances, and finding a bug.
  8. 12 Your First Trained Network Train your first two-layer neural network in NumPy on XOR and the two-moons dataset: full training loop, loss and accuracy curves, and decision boundaries.
  9. 13 Weight Initialization Learn why zero weights fail, how signals vanish or explode in deep networks, and how Xavier and He initialization keep training stable, with NumPy code.

Training in Practice

  1. 14 Momentum, RMSProp and Adam Build SGD with momentum, Nesterov, RMSProp, Adam and AdamW from scratch in NumPy, compare them on a ravine surface, and learn sensible default settings.
  2. 15 Learning Rate Schedules Plot and compare step decay, cosine, warmup and one-cycle learning rate schedules in NumPy, run a learning rate range test, and see the effect on loss curves.
  3. 16 Mini-Batches, Epochs and Data Loading Learn how shuffling, batch size, epochs and iterations fit together, write a reusable batch generator, and see why mini-batch gradients are noisy.
  4. 17 Overfitting, Underfitting and Regularization Understand train/validation/test splits, bias and variance, learning curves, L1 and L2 weight decay and early stopping, with NumPy examples on noisy data.
  5. 18 Dropout and Data Augmentation Implement inverted dropout in NumPy, understand train versus eval mode and the ensemble view, then augment arrays with flips, shifts and noise and see when each helps.
  6. 19 Batch Norm and Layer Norm Learn how batch norm and layer norm work: forward and backward passes in NumPy, running statistics, where to place them, and why they speed training.
  7. 20 Evaluating Models Measure a model honestly: confusion matrix, precision, recall, F1, ROC and AUC, calibration, imbalanced data, regression metrics and validation strategy.
  8. 21 Debugging Training Runs Debug a network that will not learn: overfit one batch, fix NaNs, tame exploding gradients with clipping, find dead units, and read loss curves.
  9. 22 Building a Mini Deep-Learning Framework Turn loose training code into a small framework: Module, Linear, activations, loss, SGD and Adam classes plus a fit loop, mapped to PyTorch's design.
  10. 23 Automatic Differentiation From Scratch Build reverse-mode automatic differentiation: a scalar Value class with a computation graph, topological order, backward(), and a tensor version.

Convolutional Networks

  1. 24 Images as Tensors See how images become arrays: pixels, channels, the (N, C, H, W) layout, normalization, the scikit-learn digits dataset, and flattening versus structure.
  2. 25 The Convolution Operation Learn how convolution works: sliding a kernel over an image, padding, stride, the output size formula and edge-detecting filters, all in NumPy.
  3. 26 Convolution, Pooling and Feature Maps Build a convolutional layer in NumPy with many filters and channels, add max and average pooling, count parameters and visualize feature maps.
  4. 27 Training a CNN From Scratch Derive the backward pass of convolution and max pooling, implement it with im2col in NumPy, and train a small CNN on 8x8 handwritten digits.
  5. 28 Classic CNN Architectures A tour of LeNet, AlexNet, VGG, Inception, ResNet and EfficientNet: the ideas behind each design, parameter counts and a residual block in NumPy.
  6. 29 Transfer Learning and Fine-Tuning Learn transfer learning: reuse pretrained features, freeze layers, fine-tune with small learning rates, and see when it helps, with a NumPy experiment.
  7. 30 Object Detection From R-CNN to YOLO: learn bounding boxes, IoU, non-maximum suppression and anchor matching, and build each one yourself in NumPy with runnable code.
  8. 31 Image Segmentation Learn semantic and instance segmentation, build a U-Net style encoder-decoder with skip connections in NumPy, and see why Dice loss beats accuracy.

Sequences, Attention and Transformers

  1. 32 Working With Sequences Prepare sequence data for neural networks: windowing time series, tokenizing text, one-hot and index encoding, padding and masking, and why order matters.
  2. 33 Recurrent Neural Networks Learn how recurrent neural networks keep a hidden state, share weights across time, and predict the next character. Build an RNN cell in NumPy.
  3. 34 Backpropagation Through Time Train an RNN with backpropagation through time in NumPy, see vanishing and exploding gradients with real numbers, and fix them with clipping and truncation.
  4. 35 LSTM and GRU Understand LSTM and GRU gates and the cell state, build an LSTM forward and backward pass in NumPy, and forecast a noisy sine wave with it.
  5. 36 Embeddings Learn embeddings as trainable lookup tables: train word vectors on a toy corpus in NumPy, compare them with cosine similarity, and embed categories.
  6. 37 Seq2Seq, Teacher Forcing and Beam Search Build an encoder-decoder RNN in NumPy, see why teacher forcing speeds up training but causes exposure bias, and compare greedy decoding with beam search.
  7. 38 The Attention Mechanism Understand attention as a soft dictionary lookup. Implement scaled dot-product attention in NumPy, plot weights, apply masks and see why we divide by sqrt(d).
  8. 39 Self-Attention and Multi-Head Attention Build self-attention and multi-head attention in NumPy, learn why order needs positional encodings (sinusoidal, learned, rotary), and visualize heads.
  9. 40 The Transformer Block Build a transformer block in NumPy: attention, MLP, residual connections and layer norm, pre-norm versus post-norm, causal masking and parameter counts.
  10. 41 Training a Tiny Language Model Build and train a character-level, decoder-only transformer in NumPy with a full backward pass, then sample text with temperature and top-k.
  11. 42 How Large Language Models Work Understand large language models: pretraining, BPE tokenizers, scaling laws, fine-tuning, RLHF, the KV cache, context windows and limits, with runnable code.

Generative and Other Models

  1. 43 Autoencoders Build an autoencoder in NumPy on handwritten digits: the bottleneck, reconstruction loss, a 2-D latent space plot, denoising and anomaly detection.
  2. 44 VAEs, GANs and Diffusion Models Learn how generative models create new data: the VAE reparameterization trick and KL term, the GAN game, and the diffusion forward and reverse process.
  3. 45 Neural Networks in Reinforcement Learning Learn reinforcement learning basics: the agent loop, Q-learning tables, a Q-network, a policy-gradient method in NumPy, exploration and instability.
  4. 46 Classic Unsupervised Networks Meet three classic networks that learn without labels: restricted Boltzmann machines, self-organizing maps and Hopfield memories, each built in NumPy.
  5. 47 RBF, Siamese, Capsule and Spiking Networks Meet four specialist network types in NumPy: RBF networks, Siamese networks with contrastive and triplet loss, capsule routing, and spiking neurons.
  6. 48 Graph Neural Networks Learn how graph neural networks work: message passing, the GCN layer, node classification, over-smoothing and readouts, all built from scratch in NumPy.

Frameworks and Production

  1. 49 Deep Learning Frameworks Compared Compare deep learning frameworks and their history: Theano, Caffe, TensorFlow, Keras, PyTorch, MXNet and JAX, with eager vs graph execution demonstrated in code.
  2. 50 PyTorch Essentials Learn PyTorch by mapping it onto the NumPy network you already built: tensors, autograd, nn.Module, DataLoader, optimizers, the training loop and GPUs.
  3. 51 TensorFlow and Keras Essentials Learn the Keras workflow (Sequential, functional API, compile, fit, callbacks, saving) by mapping it onto a mini framework you can run in the browser.
  4. 52 Hugging Face Transformers Pipelines Learn the Hugging Face transformers workflow: pipelines, tokenizers, AutoModel classes and a fine-tuning loop, with runnable NumPy stand-ins for each step.
  5. 53 Experiments and Reproducibility Run deep-learning experiments you can trust: fix seeds, store configs, log metrics, compare random and grid search, and version your data and models.
  6. 54 Deploying and Serving Models Take a trained network to production: save weights, export with ONNX, shrink it with int8 quantization and pruning, serve it, and watch for data drift.
  7. 55 Model Compression Shrink neural networks with structured pruning, per-channel int8 quantization and knowledge distillation, each demonstrated in runnable NumPy code.
  8. 56 Responsible Deep Learning Build models that are fair, private and robust: measure bias with fairness metrics, try a gradient attack, read saliency maps and write a model card.

Projects

  1. 57 Project: Handwritten Digit Classifier Build a complete digit classifier: split and scale the data, train an MLP with Adam in NumPy, read the confusion matrix, and study the errors it makes.
  2. 58 Project: Predicting House Prices With a Network Build a regression neural network in NumPy to predict house prices from tabular data, scale the features, stop early, and compare it with linear and boosted trees.
  3. 59 Project: Shape Recognition With a CNN Generate images of circles, squares and triangles, train a small convolutional network from scratch, use augmentation to generalize, and inspect learned filters.
  4. 60 Project: Sentiment Classifier Build a sentiment classifier in NumPy: bag-of-words and embedding networks versus logistic regression, then use error analysis to find what they miss.
  5. 61 Project: Time-Series Forecaster Forecast a noisy seasonal series with an MLP, an RNN and an LSTM written in NumPy, and measure every model against naive baselines and a growing horizon.
  6. 62 Project: Cat vs Dog and CIFAR-10 Classifier Build an image classifier project: data augmentation and transfer learning with a NumPy CNN on toy images, plus the PyTorch recipe for CIFAR-10 and cats vs dogs.
  7. 63 Project: Chest X-ray Pneumonia and Lung Cancer Detection Build a medical-imaging classifier project: class imbalance, recall versus precision, threshold choice and occlusion maps, plus the transfer-learning recipe.
  8. 64 Project: Text Generation with an LSTM Train a character-level LSTM in NumPy to write text, track loss and perplexity, and compare greedy, temperature, top-k and top-p sampling.
  9. 65 Project: Face and Emotion Detection Build a two-stage vision pipeline in NumPy: a sliding-window face detector and a CNN emotion classifier, and learn why the second stage must train on the first stage's crops.
  10. 66 Project: Audio Classification Turn sound into spectrograms with NumPy, classify five kinds of audio with a CNN, and fix noise problems with augmentation and SpecAugment-style masking.
  11. 67 Project: Human Activity and Sign Language Recognition Recognize hand signs and body activities from pose keypoints in NumPy: landmark normalization for static signs, and an LSTM against MLPs for motion sequences.
  12. 68 Project: Anomaly Detection with Autoencoders Detect anomalies with an autoencoder in NumPy: train on normal data only, score by reconstruction error, choose a threshold and compare with PCA and z-scores.
  13. 69 Project: Image Captioning Build an image captioning model in NumPy: a CNN encoder feeds an LSTM decoder, trained with teacher forcing, then evaluate captions with exact match and BLEU.
  14. 70 Project: Machine Translation with a Transformer Build a tiny encoder-decoder transformer with cross-attention and a full backward pass in NumPy, train it to translate a toy language pair, and check it with gradient checking.

Practice and Review

  1. 71 Practice Exercises Ten graded deep learning exercises with runnable solutions: neurons, softmax, backpropagation, optimizers, dropout, convolutions, attention, layer norm and parity.
  2. 72 Deep Learning Interview Questions Fifteen common deep learning interview questions with short answers and runnable NumPy demos: vanishing gradients, batch norm, overfitting, attention cost and more.
  3. 73 Deep Learning Cheat Sheet A one-page deep learning reference: core formulas, tensor shapes, activation and loss pairings, optimizer defaults, a debugging checklist and an architecture chooser.