Skip to content
Tutorials Center
My learning

Topics

Distributed Systems Tutorial

Learn Distributed Systems step by step: 36 lessons in 10 chapters with examples, quizzes, diagrams and projects.

Start learning →

of lessons completed

Getting Started

  1. 1 What Is a Distributed System? Learn what a distributed system is, why we spread work over many machines, what it costs, and build a first toy cluster of nodes in Python.
  2. 2 The Fallacies of Distributed Computing Meet the eight classic false assumptions about networks, from reliability to zero latency, and simulate a flaky link with a seeded drop model in Python.
  3. 3 Failure and System Models Learn crash-stop, crash-recovery, omission and Byzantine failures, plus synchronous and asynchronous timing models, with a failure-injection simulator.

Time and Ordering

  1. 4 Physical Clocks, Drift and Skew Understand wall clocks versus monotonic clocks, clock drift, NTP and last-write-wins hazards, with Python simulations of drifting node clocks.
  2. 5 Lamport Clocks and Happens-Before Learn the happens-before relation and implement Lamport logical clocks with message passing to get a consistent total order of events in Python.
  3. 6 Vector Clocks and Causality Build vector clocks in Python to tell causal order from concurrency, then see causal delivery and version vectors that detect conflicting writes.

Communication

  1. 7 Messages, RPC and Delivery Guarantees Learn how nodes talk: request/response vs messages, RPC, timeouts, at-most-once, at-least-once and the exactly-once myth, with a lossy-network simulation.
  2. 8 Idempotency, Retries and Backoff Make retries safe with idempotency keys and a dedup table, then compare fixed, exponential and jittered backoff in a retry-storm simulation.
  3. 9 Queues, Pub/Sub and Event Logs Compare point-to-point queues, publish/subscribe and replayable logs. Learn ordering, offsets and consumer groups by building a mini log broker.

Replication

  1. 10 Replication: Leaders and Followers Learn single-leader replication: why we copy data, synchronous vs asynchronous replication, replication lag and failover pitfalls with a toy store.
  2. 11 Consistency Models Understand consistency models: linearizability, sequential, causal and eventual consistency, read-your-writes and monotonic reads, with stale-read demos.
  3. 12 Quorums and Tunable Consistency Learn quorum replication: N, W and R, why R + W > N guarantees overlap, sloppy quorums, hinted handoff and read repair, with a runnable quorum store.
  4. 13 Multi-Leader and Leaderless Replication Learn how multi-leader and leaderless (Dynamo-style) replication handle concurrent writes: conflicts, last-write-wins, merging and anti-entropy, with simulations.
  5. 14 CRDTs: Conflict-Free Replicated Data Types Learn CRDTs: how G-Counters, PN-Counters, G-Sets, OR-Sets and LWW-registers merge without conflicts, with the three merge laws and convergence tests.

Partitioning and Storage

  1. 15 Partitioning and Sharding Learn how databases split data across nodes: range vs hash partitioning, hot spots, rebalancing, secondary indexes and consistent hashing with virtual nodes.
  2. 16 Distributed Transactions: 2PC and Sagas Learn atomic commit across services: how two-phase commit works and why it blocks, how sagas use compensations, and the outbox pattern, with simulators.
  3. 17 Distributed File Systems How distributed file systems work: file access models, client caching and validation, sharing semantics, and chunked, replicated storage in the style of GFS and HDFS.

Consensus and Coordination

  1. 18 CAP, PACELC and Real Trade-offs What the CAP theorem really says, why partitions are not optional, how PACELC adds the latency trade-off, and a CP vs AP partition simulation in Python.
  2. 19 Failure Detectors and Heartbeats Learn how distributed systems notice dead nodes: heartbeats, timeouts, false positives, the phi-accrual idea and SWIM-style gossip membership, with simulations.
  3. 20 Leader Election Learn how a cluster picks one leader: the bully and ring algorithms, terms and epochs, split brain, and fencing tokens. Includes a runnable simulation.
  4. 21 Consensus and Paxos Understand the consensus problem, why FLP says it cannot be fully solved, and how single-decree Paxos stays safe. Run a Paxos simulation in your browser.
  5. 22 Raft Step by Step Learn Raft: terms, leader election, log replication, the commitment rule and safety. Run a small Raft cluster in your browser, including a leader crash.

Building Blocks

  1. 23 Distributed Locks, Leases and Fencing Why distributed locks are hard: lease expiry, GC pauses and stale clients. Learn fencing tokens and run a safe lock service simulation in your browser.
  2. 24 Distributed Caching Learn cache-aside and write-through, invalidation, cache stampedes, request coalescing and TTL jitter, with runnable Python simulations.
  3. 25 Load Balancing and Distributed Rate Limiting Compare round robin, least connections and power of two choices, then learn token bucket and sliding window rate limits across nodes, with simulations.
  4. 26 Gossip Protocols and Probabilistic Structures See how gossip spreads data through a cluster in about log n rounds, and how Bloom filters and HyperLogLog sketches keep that traffic small.

Reliability and Operations

  1. 27 Resilience Patterns Learn timeouts, circuit breakers, bulkheads, load shedding, backpressure and hedged requests, and run simulations that show why each one helps.
  2. 28 Observability and Tail Latency Understand logs, metrics and traces, why averages hide pain, and how fan-out multiplies tail latency, with NumPy experiments you can run.
  3. 29 Testing Distributed Systems Learn deterministic simulation, fault injection, property-based testing and linearizability checking, with a runnable seeded chaos tester.

Projects

  1. 30 Project: Replicated Key-Value Store Build a quorum-replicated key-value store with versions, read repair and node failures, then test it with seeded random fault injection.
  2. 31 Project: A Mini Raft Cluster Build a five-node Raft simulation with elections, log replication, crashes and recovery, driven by a deterministic scheduler and invariant checks.
  3. 32 Project: Reliable Job Queue Build a job queue with visibility timeouts, retries, a dead-letter list and idempotent workers, then prove with a simulation that at-least-once processing is safe.
  4. 33 Project: Causally Ordered Chat Build a chat where replies never appear before the message they answer: vector clocks, a causal delivery buffer, an out-of-order network and convergence tests.

Practice and Review

  1. 34 Practice Exercises with Solutions Twelve hands-on distributed systems exercises with runnable Python solutions: Lamport and vector clocks, quorums, retries, CRDTs, sharding and Raft commits.
  2. 35 Interview Questions Common distributed systems interview questions with concise answers: CAP, consensus, replication, idempotency, sharding, plus how to walk through a design prompt.
  3. 36 Distributed Systems Cheat Sheet A one-page distributed systems cheat sheet: delivery guarantees, consistency levels, quorum formulas, algorithm summaries, decision tables and a failure checklist.