Skip to content
Tutorials Center
My learning

Topics

Lesson 2 of 36 Β· Getting Started

The Fallacies of Distributed Computing

Meet the eight classic false assumptions about networks, from reliability to zero latency, and simulate a flaky link with a seeded drop model in Python.

In the 1990s engineers at Sun Microsystems collected a list of assumptions that newcomers to networked programming keep making, and that keep causing outages. They became known as the eight fallacies of distributed computing. A fallacy is a belief that feels true because it is true on your laptop, and false in production.

Every technique in this tutorial (timeouts, retries, replication, consensus) exists because one of these beliefs is wrong. Knowing the list by heart lets you spot design mistakes early, in your own code and in code reviews.

Fallacies 1 to 4

Network is reliable Packets get lost, duplicated and reordered.
Latency is zero Every call across a network costs real time.
Bandwidth is infinite Pipes have a size, and bytes queue up.
Network is secure Anyone on the path may read or alter traffic.
1 / 2

1. The network is reliable

A local function call cannot lose its arguments. A network request can. Cables are cut, switches reboot, buffers overflow, firewalls drop packets, and a server may crash between receiving your request and answering it.

Protocols such as TCP hide some of this by retransmitting lost packets. They cannot hide a connection that breaks for good, or a peer that is down. Your program still sees timeouts and resets, and must be written for them. Because you cannot always tell whether a failed request was processed, safe retries need care (see idempotency and retries).

Let us measure the effect. We model a link that independently drops each message with some probability, using a seeded random generator so the result is the same every time you run it. A client sends 1,000 requests, and each needs both the request and the reply to arrive.

Python A flaky link with a seeded drop model

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the 5% row. Only 5% of messages are lost, but a request needs two messages, so about 12% of calls fail without retries (the simulation shows 88% success). Losses compound along every hop. Retries fix most of it, but each retry adds delay and load, a trade-off the messaging lesson explores.

2. Latency is zero

A call inside one process takes nanoseconds. A call to another machine in the same data center takes about half a millisecond, and across an ocean about a hundred. You cannot buy your way out of the speed of light.

The practical consequence: count round trips. A page that makes 50 sequential service calls at 2 ms each spends 100 ms just waiting, before any real work. Chatty designs that look fine in a test with everything on localhost become slow in production.

3. Bandwidth is infinite

Bandwidth is how many bytes per second a link can carry. Latency is how long the first byte takes. Both matter, and total time is roughly:

time = latency + size / bandwidth

Small messages are dominated by latency; large ones by bandwidth. Sending a 50 MB response to many clients can saturate a link and slow everyone else, including traffic that is tiny.

Python Latency versus bandwidth

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The same megabyte takes about 50 seconds when sent as a thousand tiny calls, and 0.15 seconds in one batch. Batching, pipelining and caching all exist to reduce round trips.

4. The network is secure

Traffic may cross networks you do not control. Anyone on the path can sometimes read it, change it or pretend to be a server. "It is on our internal network" is not a defense: attackers who reach one machine move sideways.

Treat every connection as hostile: encrypt in transit (TLS), authenticate both sides, authorize each request, and never trust data just because it came from "inside". Note that most of this tutorial assumes honest-but-faulty nodes, and treats malicious behavior separately in failure models.

5. Topology does not change

Servers are added, removed, restarted and moved. Cloud instances get new IP addresses. Routes change, and a network that was a single hop yesterday may be three hops today.

Hard-coded addresses break. Instead use service discovery, DNS with sensible time-to-live values, and clients that re-resolve and reconnect. Algorithms for membership and failure detection (covered in failure detectors) exist because the set of nodes is never fixed.

6. There is one administrator

Real systems cross organizational borders: your cloud provider, a payments company, a CDN, another team's service. Each runs on its own schedule, upgrades at its own time and has its own rules. Nobody can say "restart everything".

So protocols must tolerate mixed versions (old and new nodes talking at once) and you need contracts that evolve safely: add fields, do not repurpose them, and keep old behaviors until all callers moved.

7. Transport cost is zero

Sending data is not free. Turning objects into bytes (serialization) uses CPU. Moving them uses bandwidth, which cloud providers bill, especially across regions. Building and maintaining the network is expensive too.

Look for chatty protocols, oversized payloads and repeated transfers of data nobody reads. A compact binary format, compression and caching can all pay for themselves.

8. The network is homogeneous

Your system will include different operating systems, languages, hardware, network speeds and protocol versions. A phone on weak cellular service is a very different peer from a server in a rack.

Rely on standard, well-specified formats rather than assumptions about the other side, and test with slow and broken peers rather than only fast, healthy ones.

The ninth fallacy: clocks agree

A common addition to the list is the belief that all machines agree on the time. They do not: each has its own oscillator which drifts, and synchronization only narrows the gap. If you order events by timestamps from different machines you can get the wrong order. The next section of the tutorial, starting with clocks and time, is about this.

None of these fallacies says the network is bad. Networks are astonishingly good. They say the network is not perfect, and that your design must keep working in the imperfect cases, because at scale the rare case happens every minute.

Check your understanding

Quick check Β· 4 questions

of 4 answered You scored / 4 Β· perfect!

  1. 1. A link drops 5% of messages and a call needs a request plus a reply. About what share of calls fail with no retries?

    A call succeeds only if both messages arrive, which has probability 0.95 times 0.95, about 90%. So roughly 10% fail.
  2. 2. Which formula best approximates the time to transfer a message?

    There is a fixed cost to start (latency) plus the time to push the bytes through (size over bandwidth). Small messages are dominated by the first term, large ones by the second.
  3. 3. Why is "we are on an internal network, so no encryption needed" a fallacy?

    The network is not secure. A single compromised host can observe or tamper with internal traffic, so each connection should be authenticated and encrypted.
  4. 4. What is the most direct design response to "latency is not zero"?

    Because each call across the network has a real cost, reducing the number of sequential round trips is the most effective fix.

Summary

  • The eight fallacies are: the network is reliable, latency is zero, bandwidth is infinite, the network is secure, topology does not change, there is one administrator, transport is free, and the network is homogeneous.
  • Message loss compounds: a call needing two messages fails about twice as often as one message.
  • Total transfer time is latency plus size over bandwidth, so count round trips and batch.
  • Add the ninth: clocks do not agree.
  • Design for the imperfect case, since at scale rare events are routine.

Next: Failure and System Models, where we state exactly which failures we assume.