Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 55 ยท Networks and the Internet

How the Web Works

Follow a web page from your browser to a server and back: URL anatomy, HTTP requests and responses, status codes, HTML, the HTTPS handshake idea and cookies.

When you type an address and press Enter, a page appears within a second. In that moment your browser works out where the server is, opens a connection, sends a request, receives a reply, and turns the reply into pixels. Nearly every step uses something from the previous lessons: DNS finds the address, packets carry the bytes, and layered protocols keep it organized.

This lesson walks through that trip: the parts of a URL, the HTTP request and response, status codes, how HTML leads to more requests, what HTTPS adds, and how cookies let a site remember you. You will also run a real web server and client in the same program. It builds on The Internet, IP Addresses and DNS.

The web and the Internet are different things

The Internet is the network. The World Wide Web is one service that runs over it: a collection of documents and applications that link to one another, delivered by the HTTP protocol. Email, online games and video calls use the Internet without being "the web". Three technologies define the web: URLs to name things, HTTP to fetch them and HTML (with CSS and JavaScript) to describe what the browser shows.

Anatomy of a URL

A URL (uniform resource locator) is the address of a resource. Take this one:

https://www.example.com:8443/guides/intro.html?lang=en&page=2#setup

Part Value Meaning
Scheme https Which protocol to use
Host www.example.com Which server (looked up through DNS)
Port 8443 Which program on the server; omitted when it is the default (443 for HTTPS, 80 for HTTP)
Path /guides/intro.html Which resource on that server
Query lang=en&page=2 Extra parameters, as name=value pairs joined with &
Fragment setup A place inside the page; it is handled by the browser and never sent to the server

Characters that have a special meaning in a URL, or that are not allowed, are written as a percent sign plus two hexadecimal digits, so a space becomes %20. Programming languages have a built-in URL type that takes URLs apart for you, as the next stand-in shows.

JS Taking a URL apart

Press Run (or Ctrl + Enter) to see the console output here.

No output.

HTTP: request and response

HTTP is a simple text-based conversation. The browser (the client) sends a request; the server sends back a response. A request has a first line with the method, the path and the version, then headers (extra information as Name: value lines), a blank line, and an optional body.

TEXT
GET /guides/intro.html HTTP/1.1
Host: www.example.com
Accept: text/html

The common methods are GET (fetch something, with no side effects), POST (send data, such as a submitted form), PUT and PATCH (replace or change a resource) and DELETE. The response starts with a status line, followed by headers and a body:

TEXT
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 1256

<!doctype html>
<html>...

HTTP is stateless: each request stands alone, and the server does not automatically remember the previous one. Everything that feels continuous, like staying logged in, is built on top, as you will see with cookies.

sequenceDiagram
  participant B as Browser
  participant S as Web server
  B->>B: Look up the host with DNS
  B->>S: Open a connection (TCP, then TLS for HTTPS)
  B->>S: GET /index.html
  S-->>B: 200 OK + HTML
  B->>S: GET /style.css
  S-->>B: 200 OK + CSS
  B->>S: GET /logo.svg
  S-->>B: 200 OK + image
  B->>B: Build the page and draw it

One page view usually means many requests: the HTML first, then everything it refers to.

Status codes

The three-digit status code tells the client what happened. The first digit gives the family:

  • 1xx: informational (rarely seen by users).
  • 2xx: success. 200 OK is the normal answer; 201 Created follows a successful creation; 204 No Content means success with no body.
  • 3xx: redirection. 301 Moved Permanently and 302 Found send the client to the address in the Location header; 304 Not Modified says "your cached copy is still good".
  • 4xx: the client made a mistake. 400 Bad Request, 401 Unauthorized (you need to sign in), 403 Forbidden (you may not), 404 Not Found, 429 Too Many Requests.
  • 5xx: the server failed. 500 Internal Server Error, 502 Bad Gateway, 503 Service Unavailable.

The famous "404" is just one code out of many; a well-built site uses the right code so that browsers, search engines and programs know how to react.

From HTML to a page

The body of the first response is usually HTML, which describes the structure of the page. The browser reads it, builds an internal tree of elements, and notices references to other resources: stylesheets, scripts, images, fonts. It requests each of them (often in parallel, over connections that are reused), applies the CSS and runs the JavaScript. JavaScript can change the page, or request more data from servers in the background, which is how a web app updates without reloading. If you want to learn to write pages yourself, the HTML tutorial is the place to start.

Running a server and a client

The real web needs a server on a remote machine, but the protocol is the same on your own computer: a program can listen for HTTP requests on the loopback address 127.0.0.1, and another program, or the same one, can send them. The next stand-in uses Node.js on the site's code runner. It starts a tiny server and then makes real requests to it with fetch, showing the methods, status codes, a redirect and a cookie.

Node.js A real HTTP server and client on loopback

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Read the output from top to bottom. The first visit has no cookie, so the server sets visits=1; the client's jar sends it back on the second request, and the count becomes 2. The server used nothing but the cookie to know it was the same client. The /old path answered 301 with a Location header (the client here asked not to follow it). The POST returned 201 with the echoed text, and an unknown path gave 404.

Cookies

A cookie is a small piece of text that a server asks the browser to store, using a Set-Cookie header. The browser then adds it to later requests to the same site in a Cookie header. That is the mechanism behind logins (a random session ID identifies you, while the account data stays on the server), shopping carts, language choices and, unfortunately, tracking by advertisers. Useful flags include HttpOnly (page scripts cannot read it), Secure (only sent over HTTPS) and SameSite (limits sending it from other sites). Treat a session cookie like a password: anyone who steals it can pretend to be you.

HTTPS: the idea

Plain HTTP travels as readable text, so anyone along the path, such as a shared Wi-Fi hotspot, could read or alter it. HTTPS is HTTP inside an encrypted channel created by TLS. The rough idea of the connection set-up, the handshake:

  1. The browser says which TLS versions and ciphers it supports.
  2. The server answers with its choice and a certificate: a document that ties the site's name to a public key, signed by a certificate authority that the browser already trusts.
  3. The browser checks that the certificate is valid, unexpired and matches the host name.
  4. Both sides use public-key cryptography to agree on a fresh secret key that only they know.
  5. From then on, everything is encrypted with that key.

This gives you three things: privacy (nobody can read the traffic), integrity (nobody can change it unnoticed) and authentication (you are talking to the real site, not an impostor). The padlock in the address bar means the certificate checked out; it does not mean the site itself is trustworthy, as scam sites can have certificates too. The cryptography is explained further in Passwords, Hashing and Encryption.

What is real and what is a stand-in? URLs, HTTP messages, status codes and cookies are all real. The server and client in the example are real Node.js code, but both run inside the site's sandbox on one machine, with no DNS and no TLS involved, and the cookie jar is hand-written because fetch keeps no cookies. Browsers add caching, connection reuse (HTTP/2 and HTTP/3) and many security rules that the stand-in skips.

Inside a browser, and the many kinds of server

What a web browser does

A web browser (Chrome, Firefox, Safari, Edge) is a program that requests web pages and turns them into something you can see and use. Under the surface it is several cooperating parts:

flowchart TD
  UI["User interface<br/>address bar, tabs, buttons"] --> NET["Networking<br/>HTTP, DNS, TLS"]
  NET --> PARSE["Parsers<br/>HTML to DOM, CSS to styles"]
  PARSE --> RENDER["Rendering engine<br/>layout and painting"]
  PARSE --> JS["JavaScript engine"]
  JS --> RENDER
  RENDER --> SCREEN["Screen"]
  NET --> CACHE[("Cache, cookies,<br/>local storage")]

The browser fetches resources, parses them, runs scripts and paints the result.

  1. The user interface takes your URL or search.
  2. The networking part looks up the address (DNS), opens a connection (secured with TLS for HTTPS) and sends an HTTP request.
  3. Parsers read the HTML into a tree of elements, the DOM, and the CSS into style rules. Images, scripts and style sheets referenced by the page are fetched too.
  4. The JavaScript engine runs the page's scripts, which can change the DOM.
  5. The rendering engine works out the size and position of every element (layout) and draws it (painting).
  6. A cache stores files so that the next visit is faster, and the browser stores cookies and other site data. Each tab and site runs in its own protected process (a sandbox), so a bad page cannot easily harm the rest of the browser.

Different browsers use different engines (Blink in Chrome and Edge, WebKit in Safari, Gecko in Firefox), which is why a page can look slightly different, and why web developers test in several.

Cookies in more detail

A cookie is a small text value that a server asks the browser to keep and send back with later requests to the same site. Since HTTP itself has no memory, cookies are how a site recognizes you as the same visitor between page loads: a session cookie holds a random session ID that points to your logged-in state on the server and disappears when you close the browser, while a persistent cookie has an expiry date (for "remember me" or language choice). Cookies set by the site you visit are first-party; those from other domains embedded in the page are third-party and are used for tracking (see Online Privacy and Tracking). Never share a session cookie: whoever has it can act as you.

Kinds of web servers and servers in general

A web server is a program (and by extension the machine running it) that listens for HTTP requests and sends back responses. The common ones are Apache, Nginx and Microsoft IIS. A static server just sends files from disk; a dynamic site passes the request to an application server that runs code (PHP, Python, Java, Node.js), usually talks to a database, and builds the response. Larger sites add more types of server behind the scenes:

Server Job
Web server Answers HTTP requests, serves files, hands dynamic requests to application code
Application server Runs the program logic of a site
Database server Stores and queries data (see What Is a Database?)
File server Shares files on a network
Mail server Sends and delivers email
DNS server Translates names to IP addresses
Proxy / reverse proxy Passes requests on behalf of others; a reverse proxy sits in front of web servers and can cache, filter and balance load
CDN edge server Keeps copies of content near visitors for speed (see Email, Cloud and Everyday Services)

One more idea: a "server" is a role, not a special kind of computer. Any computer running a program that waits for requests is a server, and the same computer can be a server for one thing and a client for another.

A tour of web protocols

A protocol is an agreed set of rules for communicating. A single page load uses several of them at different layers. Here is who does what:

Protocol Layer Job Default port
IP (Internet Protocol) Internet Addresses and routes packets between networks none (below ports)
TCP Transport Reliable, ordered streams: splits data into segments, acknowledges them, resends lost ones used by HTTP, email, FTP
UDP Transport Fast, no guarantees; good for calls, games and DNS used by DNS, video calls
DNS Application Translates names like example.com to IP addresses 53
HTTP Application Requests and responses between browser and server 80
HTTPS Application HTTP inside an encrypted TLS connection 443
FTP / SFTP Application Transferring files (SFTP is the secure version) 21 / 22
SMTP Application Sending email between servers 25, 587
IMAP / POP3 Application Reading email from a mailbox 143, 110
SSH Application Secure remote command line 22

When you open https://example.com/, DNS finds the IP address, TCP opens a reliable connection to that address on port 443, TLS encrypts it, and HTTP carries the request and the page. IP packets carry every piece across the networks in between. This is the layering idea from Networking Basics made concrete.

Python Which protocol and port does a URL use?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which part of a URL is never sent to the server?

    The fragment is used by the browser to jump to a place in the page, so it is not part of the request.
  2. 2. A browser receives status 404. What does it mean?

    4xx codes are client-side errors; 404 means Not Found. A server crash would be a 5xx code.
  3. 3. HTTP is described as stateless. What does that mean?

    Features such as logins are built by sending an identifier (often a cookie) with each request.
  4. 4. What does the TLS certificate prove during the HTTPS handshake?

    The certificate links the host name to a public key. It authenticates the server's identity, not the quality or honesty of the site.
  5. 5. Which status code family means the server failed to handle a valid request?

    5xx codes such as 500 and 503 signal server-side problems.

Summary

  • The web is a service on the Internet that uses URLs, HTTP and HTML.
  • A URL has a scheme, host, optional port, path, query and fragment; special characters are percent-encoded.
  • HTTP is a request-response protocol with methods, headers, status codes and an optional body, and it is stateless.
  • Status codes group into 2xx success, 3xx redirects, 4xx client errors and 5xx server errors.
  • One page view triggers many requests as the browser fetches the resources the HTML refers to.
  • Cookies let servers recognise a returning browser; HTTPS uses TLS and certificates for privacy, integrity and authentication.

Next: Email, the Cloud and Everyday Online Services.