Skip to content
Tutorials Center
My learning

Topics

Lesson 3 of 45 ยท Getting Started

Your First Tensors

Create PyTorch tensors with tensor, zeros, ones, arange, rand and randn, read shape, dtype and device, set seeds, and compare tensors with NumPy arrays.

A tensor is PyTorch's name for a multi-dimensional array of numbers. A single number is a 0-D tensor, a list of numbers is 1-D, a table is 2-D, a batch of color images is 4-D. Every input, weight, gradient and output in a neural network is a tensor.

If you know NumPy arrays, you already know most of this lesson. The differences are what make tensors useful for deep learning: they can live on a GPU, and they can track gradients. This lesson covers creating tensors, the three attributes you will check constantly (shape, dtype, device), and reproducible random numbers.

Creating tensors

All of the following is real PyTorch, shown read-only:

Python
import torch

a = torch.tensor([1, 2, 3])               # from a Python list
b = torch.tensor([[1.0, 2.0], [3.0, 4.0]])

z = torch.zeros(2, 3)                      # 2 rows, 3 columns of 0.
o = torch.ones(3)
r = torch.arange(0, 10, 2)                 # tensor([0, 2, 4, 6, 8])
l = torch.linspace(0, 1, 5)                # 5 evenly spaced numbers
e = torch.eye(3)                           # identity matrix
f = torch.full((2, 2), 7.0)

u = torch.rand(2, 3)                       # uniform in [0, 1)
n = torch.randn(2, 3)                      # standard normal (mean 0, std 1)

torch.tensor(data) always copies the data. The shape-based factories (zeros, ones, rand, randn) take the sizes either as separate arguments or as a tuple. There are *_like versions that copy another tensor's shape and dtype:

Python
x = torch.randn(4, 5)
y = torch.zeros_like(x)      # shape (4, 5), same dtype and device as x

Every tensor has three facts

shape The size along each dimension.
dtype The kind of number stored.
device CPU or GPU memory.

Most bugs are a mismatch in one of these three.

1 / 2

Shape, dtype and device

Python
x = torch.randn(2, 3)

print(x.shape)      # torch.Size([2, 3])
print(x.dtype)      # torch.float32
print(x.device)     # cpu
print(x.ndim)       # 2
print(x.numel())    # 6 elements in total
print(x.size(0))    # 2, the length of dimension 0

The default float type is float32, not float64 like NumPy. Integer lists become int64. That is a frequent source of surprises: neural network layers expect float tensors, so feeding them an int tensor fails with a dtype error. Dtypes get their own lesson: Dtypes, Precision and NumPy Interop.

The device says where the numbers live. A new tensor starts on the CPU. Moving it to a GPU is one call, and operations require all their inputs on the same device:

Python
if torch.cuda.is_available():
    x = x.to("cuda")
    print(x.device)   # cuda:0

GPUs get a full lesson later in GPUs, Devices and Mixed Precision.

Reproducible randomness

Random tensors differ on every run unless you seed the generator:

Python
torch.manual_seed(42)
print(torch.rand(3))   # the same three numbers every time on the same setup

Seeding makes a notebook or a bug report reproducible. The exact numbers depend on the PyTorch version and device, so do not hard-code them in tests; check shapes and statistics instead.

Moving between NumPy and PyTorch

Python
import numpy as np

arr = np.array([1.0, 2.0, 3.0])
t = torch.from_numpy(arr)     # shares memory with arr, no copy
back = t.numpy()              # shares memory again (CPU tensors only)

Because memory is shared, changing one changes the other. That is covered carefully in the dtype lesson.

A NumPy stand-in

You cannot run PyTorch here, but NumPy arrays have almost the same creation functions, so you can practice the ideas. The table maps one to the other.

PyTorch NumPy
torch.zeros(2, 3) np.zeros((2, 3))
torch.arange(0, 10, 2) np.arange(0, 10, 2)
torch.randn(2, 3) rng.standard_normal((2, 3))
x.shape, x.ndim a.shape, a.ndim
x.numel() a.size
x.dtype (default float32) a.dtype (default float64)
Python Array facts: shape, dtype, size

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first line of output is a (2, 3) int64 ndim 2 numel 6: Python integers become 64-bit integers in both libraries, while floats differ (NumPy float64, PyTorch float32) unless you ask otherwise.

What a tensor adds on top of an array

A tensor is an array plus two abilities that arrays lack. The first is device placement (CPU or GPU). The second is gradient tracking: you can mark a tensor with requires_grad=True and PyTorch records every operation on it so it can compute derivatives later.

Python
w = torch.tensor([2.0, 3.0], requires_grad=True)
loss = (w * w).sum()
loss.backward()
print(w.grad)       # tensor([4., 6.])

We will build that machinery by hand in Computational Graphs and the Chain Rule. For now, the stand-in below checks the same numbers with NumPy: the derivative of w**2 is 2*w.

Python The gradient PyTorch would compute

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

This stand-in is plain NumPy math. The requires_grad and backward() calls above are the real PyTorch way to get [4., 6.] without writing the derivative yourself.

Tip: When something breaks, print x.shape, x.dtype and x.device first. Most tensor errors are one of those three.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the default dtype of torch.randn(3)?

    PyTorch's default floating point type is float32, unlike NumPy's float64.
  2. 2. What does torch.tensor([1, 2, 3]) produce in terms of dtype?

    A list of Python integers becomes an int64 tensor. Write 1.0 or pass dtype=torch.float32 to get floats.
  3. 3. Why call torch.manual_seed(42) before generating random tensors?

    Seeding the generator makes runs reproducible, which helps debugging and comparing experiments.
  4. 4. Which method returns the total number of elements in a tensor x?

    numel() multiplies all dimension sizes. ndim is the number of dimensions and shape lists their sizes.

Summary

  • A tensor is an N-dimensional array that can live on a GPU and track gradients.
  • Create tensors with torch.tensor, zeros, ones, arange, linspace, eye, rand and randn, plus *_like helpers.
  • Check shape, dtype and device whenever something breaks.
  • Floats default to float32, Python integers to int64.
  • torch.manual_seed makes random tensors reproducible on the same setup.
  • from_numpy and .numpy() share memory with the array.

Next: slice and reshape tensors in Indexing, Slicing and Reshaping.