Skip to content
Tutorials Center
My learning

Topics

Lesson 2 of 45 ยท Getting Started

The Deep Learning Landscape

A map of deep learning for PyTorch learners: how it evolved, where it is used, why it is hard, how to cope, and how PyTorch compares with TensorFlow, Keras, Caffe and MXNet.

PyTorch is a tool; deep learning is the craft it serves. This lesson steps back to give you the map: what deep learning is, how it got here, where it is used, what makes it difficult in practice, and where PyTorch sits among the other frameworks. Everything here is conceptual, with two small NumPy experiments (PyTorch cannot run in the browser) that make the challenges tangible.

What deep learning is

Deep learning is machine learning with neural networks that have many layers. Each layer transforms its input into a slightly more abstract representation, and the whole stack is trained end to end by gradient descent. Classical machine learning usually needs a human to design features (edge detectors, word counts); deep learning learns the features from raw data.

It became practical in the 2010s because three things arrived together: large labeled datasets, GPUs that multiply matrices in parallel, and better training tricks (ReLU, good initialization, normalization, dropout, better optimizers) covered later in this track.

A short history

  1. 1943 to 1958: the McCulloch-Pitts neuron and Rosenblatt's perceptron.
  2. 1969 to 1980s: the XOR limitation stalls single-layer models; backpropagation (popularized in 1986) makes multi-layer training work.
  3. 1990s: LeNet reads handwritten digits with convolutions; LSTMs fix vanishing gradients in sequences.
  4. 2012: AlexNet wins ImageNet by a wide margin using GPUs and ReLU; the deep learning boom begins.
  5. 2014 to 2015: GANs, ResNets, and attention for translation.
  6. 2017 onward: the transformer takes over language, then vision, audio and multimodal models.

The architectures from this history each have a lesson: Neurons, Perceptrons and Feedforward Networks, CNN Architectures and ResNet, Recurrent Networks, Autoencoders, GANs and Transformers.

Where deep learning is used

Area Examples
Vision Image classification, object detection, segmentation, video analysis, medical imaging
Language Translation, chatbots, text generation, summarization, search
Speech and audio Speech recognition, voice assistants, music generation
Recommendation Feeds, shopping and video suggestions
Healthcare Tumor detection, protein structure and drug discovery, personalized treatment
Finance Fraud detection, risk scoring, algorithmic trading signals
Security and science Intrusion detection, weather and materials research
Robotics and games Control from camera input, game-playing agents (with reinforcement learning)
Generative AI Image, audio, code and text generation

Deep learning matters because it is general (one approach covers all of these), scalable (more data and compute keep improving results) and transferable (a pretrained model can be adapted, see Transfer Learning).

Why deep learning is hard

Powerful does not mean easy. These are the recurring challenges.

  • Data quality and quantity. Models need lots of representative, correctly labeled data. Noisy labels and missing groups quietly cap performance.
  • Computational cost. Training large models needs GPUs, days of time and money. Memory is often the limit before speed is.
  • Overfitting and generalization. A model can memorize its training set and fail on new data (Overfitting and Regularization).
  • Interpretability. A network with millions of weights is a black box; explaining one decision is hard (tools such as LIME and SHAP approximate local explanations).
  • Bias and ethics. Models inherit the biases in their data, and can reinforce them at scale.
  • Adversarial attacks. Tiny, deliberate changes to an input can flip a prediction.
  • Hyperparameter sensitivity. Learning rate, batch size and architecture choices change results dramatically.
  • Hardware limits and deployment. A model that works in a notebook may be too slow or large for a phone.

Strategies that help

Challenge Typical response
Little data Augmentation, transfer learning, synthetic data
Compute cost Cloud GPUs, mixed precision, smaller or distilled models (Performance)
Overfitting Dropout, weight decay, early stopping, more data
Opaque decisions Explanation tools, simpler baselines, careful evaluation
Bias Audit data and metrics per group, balance data, regular reviews
Tuning Systematic search, learning rate schedules (Learning Rate Schedules)
Deployment Quantization, pruning, export formats (Testing and Deployment)

Experiment: how big is your model?

Compute and memory challenges start with parameter counts. A float32 parameter takes 4 bytes, and training with Adam keeps roughly three more copies (gradient plus two moment estimates), so training needs about four times the weights' size before counting activations.

Python Parameter count and memory of dense networks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Experiment: an adversarial attack

An adversarial example nudges every input feature a tiny amount in exactly the direction that hurts most. For a linear classifier the gradient of the score with respect to the input is just the weight vector, so the fast gradient sign method (FGSM) subtracts eps * sign(w) from a correctly classified positive example. In PyTorch you would get that gradient from x.grad after loss.backward(); here we can write it directly.

Python FGSM on a linear classifier

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each feature moved by at most eps, yet the score drops by eps * sum(|w|), which grows with the number of features. High-dimensional inputs such as images are especially fragile, which is why robustness is a research field of its own.

PyTorch and the other frameworks

A deep learning framework gives you tensors, automatic differentiation, layers, optimizers and GPU support so you do not write backpropagation yourself.

Framework Notes
PyTorch Define-by-run (eager) execution, Pythonic, dominant in research and increasingly in production. Developed at Meta, now governed by the PyTorch Foundation.
TensorFlow Google's framework. Built around computation graphs (tf.function), strong deployment tools (TensorFlow Serving, TFLite).
Keras A high-level API for building models quickly. Keras 3 runs on top of TensorFlow, PyTorch or JAX.
JAX Functional, NumPy-like with jit and grad; popular for research at scale.
Apache MXNet Scalable, multi-language, with the Gluon API; used heavily by AWS but now retired from active development.
Caffe An early C++ framework for convolutional networks configured with text files; fast for inference, rigid for research. Caffe2 was merged into PyTorch.

PyTorch versus TensorFlow

  • Execution. PyTorch runs operations immediately, so you can print, debug with pdb and use ordinary if and for. TensorFlow 2 is also eager by default but compiles with tf.function for speed. PyTorch offers torch.compile for the same (Performance).
  • API style. PyTorch has you write the training loop; Keras hides it behind model.fit. PyTorch's explicit loop is more flexible, and the track covers it in The Training Loop.
  • Deployment. TensorFlow's mobile and serving tooling is mature; PyTorch has TorchScript, ONNX export and ExecuTorch.
  • Ecosystem. Most new research code and pretrained models (the Hugging Face hub in particular) are PyTorch first.

For a newcomer, the best framework is the one you can debug. That is why this track uses PyTorch.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the 'deep' in deep learning refer to?

    Depth is the number of layers; each layer learns a more abstract representation.
  2. 2. Which pair of developments helped trigger the 2012 deep learning boom?

    ImageNet-scale data and GPU training made deep convolutional networks practical.
  3. 3. An adversarial example is...

    Small, targeted perturbations can change a prediction while looking unchanged to a human.
  4. 4. What is a key difference between PyTorch and Keras model.fit?

    PyTorch gives you full control of the loop; Keras offers a one-call high-level API.

Summary

  • Deep learning is representation learning with many-layered networks, powered by data, GPUs and good training tricks.
  • It is used across vision, language, speech, recommendation, healthcare and generative AI.
  • The recurring challenges are data, compute, overfitting, interpretability, bias, adversarial fragility and deployment.
  • PyTorch's eager execution and explicit loops make it easy to debug and the common choice for research; TensorFlow, Keras, JAX, MXNet and Caffe fill other niches.

Next: install PyTorch and write your first tensors in Your First Tensors, or revisit What Is PyTorch?.