Skip to content
Tutorials Center
My learning

Topics

Lesson 10 of 55 ยท Data Representation

Images, Sound and Video as Data

See how pixels, RGB colors, sampling rate, bit depth and frames turn pictures, sound and video into numbers, and why raw media files are so large.

Text is easy to turn into numbers because it is already made of separate symbols. Pictures, sounds and movies are different: in the real world they are smooth and continuous. A photo has endless gradations of color, and a sound wave rises and falls without any steps. To store them, a computer measures the real thing at many points and writes each measurement down as a number. This process is called digitization.

Once you see that every image is a grid of numbers, and every recording is a long list of numbers, many everyday questions get simple answers: why a photo from your phone takes several megabytes, why a song is smaller than a film, why "higher quality" costs more storage. This lesson builds on Bits and Bytes and the idea from Text and Character Encoding that bytes have no meaning until a program knows how to read them.

Images: a grid of pixels

A digital image is a rectangular grid of tiny squares called pixels (picture elements). Each pixel stores one value, or a few values, that describe its color. The resolution is the size of the grid, written width by height: 1920 x 1080 means 1,920 columns and 1,080 rows, or about 2.07 million pixels.

The simplest image is black and white: one bit per pixel, 0 for off and 1 for on. A row of eight pixels fits exactly in one byte. The first TryIt draws a tiny picture from text, then packs each row into a single byte, exactly as a 1-bit image file would store it.

JS An 8 x 8 bitmap

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The eight hexadecimal values at the bottom are the entire picture. Anyone who knows that the data is an 8 x 8, 1-bit image can redraw it.

Color with RGB

To show color, screens mix three lights: red, green and blue. A color pixel stores three numbers, one per channel, usually from 0 (none of that light) to 255 (full). That is one byte per channel, so a pixel takes 3 bytes (24 bits), the common "true color" format. Combinations give 256 x 256 x 256 = 16,777,216 colors.

Color Red Green Blue Hex code
Red 255 0 0 #ff0000
Yellow 255 255 0 #ffff00
Teal 0 128 128 #008080
White 255 255 255 #ffffff
Black 0 0 0 #000000

The hexadecimal color codes you see in web design are simply the three bytes written in hex: two digits for red, two for green, two for blue (see Binary, Octal and Hexadecimal). Equal values in all three channels give a shade of gray.

Because colors are numbers, editing becomes arithmetic. A fade is a weighted average of two colors, and converting to gray is a weighted sum that gives green the largest weight, because our eyes are most sensitive to it.

JS Colors are just numbers

Press Run (or Ctrl + Enter) to see the console output here.

No output.

How big is an image?

Without compression, the size is the number of pixels times the bytes per pixel. A 1920 x 1080 photo at 3 bytes per pixel needs 1920 x 1080 x 3 = 6,220,800 bytes, about 5.9 MiB. A 4K picture (3840 x 2160) is four times that. Real files such as JPEG are far smaller because they are compressed, which is the subject of Compression and File Formats.

Two kinds of image exist. Raster images (photos, PNG, JPEG) store pixels and get blurry or blocky when enlarged. Vector images (SVG) store shapes and instructions, such as "draw a circle here with this radius", so they stay sharp at any size but suit logos and diagrams better than photographs.

Sound: measuring a wave

Sound is a pressure wave in the air. A microphone turns it into a changing voltage. To store it, the computer samples: it measures the voltage many times per second and writes each measurement as a number. Two settings decide the quality:

  • Sampling rate: how many measurements per second, in hertz (Hz). CDs use 44,100 Hz; phone calls use about 8,000 Hz.
  • Bit depth: how many bits describe each measurement, which sets how many distinct levels exist. 8 bits give 256 levels, 16 bits give 65,536.
flowchart LR
  A["Sound wave<br/>in the air"] --> M["Microphone<br/>voltage"]
  M --> S["Sampler<br/>measure N times per second"]
  S --> Q["Quantizer<br/>round to one of 2^bits levels"]
  Q --> F["List of numbers<br/>stored in a file"]

Digitizing sound: measure at regular moments, round each measurement to a level, store the numbers.

Rounding to a level introduces a small error (quantization noise), and more bits mean smaller steps. The sampling rate decides which pitches can be captured: a rate can represent frequencies only up to half of itself. That limit is the Nyquist limit. Human hearing reaches roughly 20,000 Hz, so 44,100 Hz is enough to cover it.

The next TryIt samples a 2 Hz wave 16 times in one second with only 3 bits (8 levels) and plots the stored numbers as stars. Even with so few levels you can see the wave shape.

JS Sampling a wave

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Stereo sound stores two channels, one for each speaker, so it doubles the data. A minute of uncompressed CD audio takes about 10 MiB, which is why formats such as MP3 and AAC exist.

When the sampling rate is too low

If a wave is sampled too slowly, the numbers can describe a completely different, slower wave. This effect is called aliasing. The same effect makes wheels seem to spin backward in films, because the camera samples motion at a fixed rate.

Video: pictures in a hurry

A video is a sequence of still images called frames, shown quickly enough that the eye sees motion. Common frame rates are 24 frames per second (film), 30 or 60 (video, games). Add a sound track and you have the two kinds of data you just met, played in sync.

The size of raw video explains why compression is not optional. One second of 1080p at 30 frames per second, uncompressed, is 1920 x 1080 x 3 bytes x 30, about 187 MB. An hour would be hundreds of gigabytes. Video codecs (H.264, H.265, AV1) save space mostly by noticing that neighboring frames are almost the same and storing only what changed.

JS Aliasing and the size of raw video

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The two waves give identical samples, so after recording nobody can tell which one it was. That is why audio recorders filter out frequencies above the Nyquist limit before sampling.

What is real and what is simplified

The stand-ins above work with tiny, uncompressed data so that you can see every number. Real files add a header (width, height, rate, format) before the data, and almost always compress it. The principle is the same everywhere: pick a grid or a clock, measure, round, and write numbers.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. How many bytes does one uncompressed 24-bit RGB pixel need?

    24 bits is 3 bytes: one byte each for red, green and blue.
  2. 2. What does a higher sampling rate allow a recording to capture?

    A sampling rate can represent frequencies up to half its value, so a higher rate captures higher pitches (at the cost of more data).
  3. 3. What is the main reason raw video files are so large?

    Each frame is a full grid of pixels, and 24 to 60 frames are needed every second, so the data adds up quickly.
  4. 4. Which color is #ffff00 in RGB?

    ff ff 00 means red 255, green 255, blue 0. Full red plus full green light looks yellow.

Summary

  • Digitization measures something continuous at many points and stores each measurement as a number.
  • An image is a grid of pixels; resolution is the grid size and each pixel usually stores red, green and blue bytes.
  • Sound is stored as samples; sampling rate sets the highest frequency, bit depth sets the number of levels.
  • Video is a series of frames plus sound; raw video is huge, which is why compression is essential.
  • Uncompressed size is simple multiplication: items per second (or pixels) times bits per item, divided by eight for bytes.

Next, learn how files are shrunk and recognized in Compression and File Formats.