Skip to content
Tutorials Center
My learning

Topics

Lesson 92 of 119 ยท Architecture and Deployment

Microservices

Learn when microservices help, how to draw service boundaries, and how gateways, discovery, timeouts, retries and circuit breakers work, with runnable Java demos.

A microservice architecture splits one application into several small services. Each service owns one business capability (catalog, members, loans, payments), has its own database, and is deployed on its own schedule. The services talk over the network, usually with HTTP/JSON or messages.

Microservices became famous because companies like Netflix and Amazon used them to let hundreds of teams ship independently. They also became infamous, because many small teams adopted them and got a distributed monolith: all the complexity of a network, none of the independence. In this lesson you will learn when the trade is worth it, how to draw boundaries, and the infrastructure pieces you meet in every microservice system: API gateways, service discovery and centralized configuration. Most of the lesson is about resilience, because once calls cross a network, failure is normal. You will build timeouts, retries with backoff, a circuit breaker and a tiny gateway in plain Java, and then see their Spring Cloud and Resilience4j equivalents.

This lesson builds on Layered Architecture and Calling APIs with HttpClient.

Monolith, modular monolith, microservices

Monolith Modular monolith Microservices
Deployment One unit One unit Many independent units
Module boundaries Often blurry Enforced in code (packages, modules) Enforced by the network
Calls between parts Method calls (fast, reliable) Method calls Network calls (slow, can fail)
Data One database One database, schema per module Database per service
Transactions Local ACID Local ACID Sagas, eventual consistency
Scaling Whole app Whole app Per service
Team independence Low Medium High
Operational cost Low Low High (CI/CD, monitoring, tracing per service)

Microservices solve organizational problems more than technical ones: many teams that must release independently, parts with very different scaling or availability needs, or parts written in different stacks. If you have one team, or don't yet know where the boundaries are, start with a modular monolith: package by feature (as shown in the previous lesson), keep modules from touching each other's tables, and split later when a real need appears. Splitting a well-modularized monolith is straightforward; merging badly cut services is painful.

Drawing service boundaries

Good boundaries follow business capabilities, not technical layers. "Catalog", "Membership" and "Lending" are services; "database service" and "validation service" are not. Domain-Driven Design calls such a boundary a bounded context: inside it, words like "book" have one precise meaning. The catalog's Book has an ISBN and a description; the lending service's Book only needs an id and whether it's on loan.

Rules that keep services independent:

  • Each service owns its data. No other service reads or writes its tables. If lending needs a member's name, it asks the membership API or keeps its own copy updated by events. A shared database couples services more tightly than any API.
  • High cohesion, low coupling. Things that change together belong together. If every feature needs coordinated changes in three services, the boundary is wrong.
  • Avoid chatty synchronous chains. If request A calls B, which calls C, which calls D, the availability of A is roughly the product of all four. Four services at 99.9% each give about 99.6%.
  • Prefer asynchronous events for "tell others what happened": LoanCreated, MemberBlocked. Messaging with Kafka and RabbitMQ covers this.

Data across services: sagas and the outbox

Without a shared database, "borrow a book and charge a fee" can't be one ACID transaction. A saga splits it into local transactions, each publishing an event that triggers the next step, with compensating actions for failures ("fee payment failed, so cancel the loan"). Sagas can be choreographed (services react to each other's events) or orchestrated (one coordinator tells each service what to do).

A related problem: a service must save to its database and publish an event. If it does both separately, a crash in between loses the event. The transactional outbox pattern writes the event into an outbox table in the same local transaction; a separate process reads the table and publishes the events.

The infrastructure around services

API gateway

Clients shouldn't need to know that /api/books is served by one service and /api/members by another. An API gateway is the single entry point. It routes requests to services and handles cross-cutting concerns once: authentication (validating the JWT, see JWT and OAuth 2), rate limiting, CORS, request ids, TLS termination. Popular choices are Spring Cloud Gateway, Kong, NGINX, Envoy, and cloud gateways.

Here is the core of a gateway in plain Java: two "services" on their own ports, a registry that maps service names to addresses, and a gateway that routes by path prefix, adds a request id header and returns 502 Bad Gateway when a service is unreachable.

Java A tiny API gateway with a service registry

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

The client only knows the gateway's address. The services could move to other ports or machines and only the registry would change. The X-Request-Id added by the gateway shows up in each service's response, which is how logs from different services are tied together (real systems use the W3C traceparent header, as in Actuator, Metrics and Observability).

With Spring Cloud Gateway, routes are configuration. lb:// means "look the service up in discovery and load-balance across its instances":

YAML
spring:
  cloud:
    gateway:
      server:
        webflux:                     # older versions: spring.cloud.gateway.routes
          routes:
            - id: catalog
              uri: lb://catalog-service
              predicates:
                - Path=/api/books/**
              filters:
                - StripPrefix=1
            - id: members
              uri: lb://member-service
              predicates:
                - Path=/api/members/**
              filters:
                - StripPrefix=1

Service discovery and configuration

Instances come and go as they scale and restart, so hard-coded addresses don't work. Service discovery keeps a live list of instances: with Eureka or Consul, each instance registers itself and sends heartbeats; on Kubernetes, a Service gives a stable DNS name (http://catalog-service) and the platform load-balances, which is why many teams on Kubernetes no longer need a separate registry.

Configuration also needs a home. Spring Cloud Config serves properties from a Git repository; on Kubernetes, ConfigMaps and Secrets become environment variables or files that Spring Boot reads like any other property source (Configuration, Properties and Profiles).

Resilience: designing for failure

On a network, calls can be slow, fail halfway, or succeed while the response is lost. Microservice code must assume all of this happens daily. The basic toolkit, in the order you should apply it:

  1. Timeouts on every remote call.
  2. Retries with backoff, only for safe operations.
  3. Circuit breakers to stop calling a service that is down.
  4. Bulkheads to limit how much of your capacity one dependency can consume.
  5. Fallbacks to degrade gracefully.

Timeouts

A call without a timeout can wait forever. While it waits, it holds a thread, a connection and memory; enough of them and your service stops responding too. This is how one slow service takes down everything that calls it, a cascading failure.

HttpClient has two timeouts: connectTimeout (on the builder) for establishing the connection, and timeout (on each request) for the whole response. The example calls a server that takes two seconds to answer:

Java A request timeout against a slow service

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Choose timeouts from real latency data: a little above the dependency's p99, and always shorter than your own caller's timeout. If the gateway gives up after 5 seconds, a service that waits 10 seconds on its dependency is doing work nobody will receive.

Retries with exponential backoff and jitter

Many failures are transient: a pod restarting, a dropped connection, a brief overload answered with 503. Retrying fixes those. Retrying badly makes things worse: if a struggling service gets every request three times immediately, its load triples exactly when it can least handle it.

Good retries:

  • Only retry what is safe. GET, PUT and DELETE are idempotent by HTTP's definition. A POST that creates an order is not, unless the API supports an idempotency key header so duplicates are detected (REST API Design).
  • Only retry transient errors: connection failures, timeouts, 502, 503, 504, and 429 (respecting Retry-After). Never retry 400 or 404; the answer won't change.
  • Back off exponentially (100 ms, 200 ms, 400 ms, ...) and add jitter (randomness), so thousands of clients don't retry in synchronized waves.
  • Cap the attempts, typically 2 to 4 in total.
Java Retry with exponential backoff and jitter

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

The 503s were retried with growing waits; the 404 returned immediately. Remember that retries multiply across layers: if the gateway, service A and service B each retry three times, one user click can turn into 27 calls to B. Retry at one layer, usually the one closest to the failing dependency.

Circuit breakers

If a dependency is down, even well-behaved retries waste time: every request waits for a timeout before failing. A circuit breaker tracks recent failures and, once they cross a threshold, opens: calls fail immediately (or use a fallback) without touching the dependency. After a cool-down it goes half-open and lets a few trial calls through. If they succeed, it closes again; if they fail, it opens for another cool-down.

TEXT
           failures >= threshold                 cool-down elapsed
  CLOSED --------------------------> OPEN -----------------------> HALF_OPEN
    ^                                  ^                               |
    |        trial call succeeds       |      trial call fails         |
    +----------------------------------+-------------------------------+

The example implements this state machine with a controllable clock, so you can see each transition:

Java A circuit breaker state machine

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

After three failures the breaker opened, and the next two requests got the fallback without calling the catalog at all. After the cool-down, one trial call failed, so it opened again. The next trial succeeded and closed it. That fail-fast behavior protects both sides: your threads aren't stuck in timeouts, and the struggling service gets breathing room to recover.

Production breakers such as Resilience4j's use a sliding window (for example "50% of the last 20 calls failed") instead of consecutive failures, can count slow calls as failures, and allow several trial calls in the half-open state.

Bulkheads and fallbacks

A bulkhead (named after the walls that keep a ship from sinking when one compartment floods) limits concurrent calls to each dependency, for example with a Semaphore or a dedicated thread pool. If the recommendation service hangs, at most 10 requests wait for it, and the rest of your service keeps working. A rate limiter caps calls per time window, protecting a dependency or enforcing a quota.

A fallback decides what to return when a call fails: a cached value, a default ("recommendations unavailable"), a reduced response, or a clear error. Good fallbacks are designed with the product team; returning stale but labeled data is often far better than a 500.

Resilience in Spring

Resilience4j is the standard library for these patterns in Java. With its Spring Boot integration, annotations wrap your methods and configuration lives in YAML:

Java
@Service
class CatalogClient {

    private final RestClient rest;

    CatalogClient(RestClient.Builder builder) {
        this.rest = builder.baseUrl("http://catalog-service").build();
    }

    @CircuitBreaker(name = "catalog", fallbackMethod = "titleFallback")
    @Retry(name = "catalog")
    public String title(long bookId) {
        return rest.get().uri("/books/{id}", bookId).retrieve().body(BookDto.class).title();
    }

    private String titleFallback(long bookId, Throwable cause) {
        return "(title unavailable)";
    }
}
YAML
resilience4j:
  circuitbreaker:
    instances:
      catalog:
        sliding-window-size: 20
        failure-rate-threshold: 50
        wait-duration-in-open-state: 30s
        permitted-number-of-calls-in-half-open-state: 3
  retry:
    instances:
      catalog:
        max-attempts: 3
        wait-duration: 100ms
        enable-exponential-backoff: true
        retry-exceptions:
          - java.io.IOException

Set the RestClient timeouts too (through its request factory), because neither annotation adds one. Spring Framework 7 also ships basic resilience annotations in the core: @Retryable and @ConcurrencyLimit (a bulkhead), enabled with @EnableResilientMethods. They cover simple cases without an extra library; for circuit breakers you still need Resilience4j.

Spring Cloud at a glance

Spring Cloud is a family of projects for distributed systems, released together as a "release train" that matches Spring Boot versions:

Project Purpose
Spring Cloud Gateway API gateway: routing, filters, rate limiting
Spring Cloud Netflix Eureka Service registry and discovery client
Spring Cloud Config Centralized configuration from Git or Vault
Spring Cloud LoadBalancer Client-side load balancing for lb:// names
Spring Cloud OpenFeign Declarative HTTP clients from interfaces (Spring's own @HttpExchange interfaces are a modern alternative)
Spring Cloud Circuit Breaker An abstraction over Resilience4j and others
Spring Cloud Stream Message-driven services on Kafka or RabbitMQ
Spring Cloud Kubernetes Discovery and config from Kubernetes APIs

On Kubernetes, the platform already handles discovery, load balancing and configuration, and a service mesh (Istio, Linkerd) can add retries, timeouts and mutual TLS outside the application. Pick one place for each concern so they don't fight each other.

The costs to remember

Before splitting, be honest about the price:

  • Operational load: every service needs a pipeline, monitoring, alerting, logs, tracing, security updates and on-call ownership.
  • Testing is harder: end-to-end tests across services are slow and flaky; contract tests (Spring Cloud Contract, Pact) check that a provider still matches what consumers expect.
  • Consistency is eventual: users may briefly see data that other services haven't caught up with.
  • Latency adds up with each network hop, plus serialization on both sides.
  • Versioning: services deploy independently, so APIs and event formats must stay backward compatible.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Two services, loans and members, both read and write the same members table directly. What is the main problem?

    Each service should own its data. A shared table is a hidden contract that blocks independent changes and deployments.
  2. 2. A POST /orders call times out. Should a client retry it automatically?

    A timeout doesn't tell you whether the server processed the request. POST isn't idempotent unless the API makes it so with a key.
  3. 3. In the circuit breaker demo, what happens to a request while the breaker is OPEN and the cool-down hasn't elapsed?

    Failing fast is the purpose of the open state. It frees your threads and gives the failing service time to recover.
  4. 4. Why add random jitter to exponential backoff?

    Without jitter, clients retry in synchronized waves that can knock the recovering service over again.
  5. 5. The gateway, service A and service B each retry failed calls up to 3 times (3 attempts in total). Service C, called by B, is down. Roughly how many calls can one client request cause to C?

    Retries multiply across layers, 3 x 3 x 3 = 27. Retry at one layer only, usually closest to the failing dependency.

Summary

  • Microservices split an application by business capability into independently deployed services that own their data. They trade simple method calls and ACID transactions for team independence and per-service scaling.
  • Start with a modular monolith when the team is small or boundaries are unclear; split when there's a concrete need.
  • Use bounded contexts for boundaries, avoid shared databases and long synchronous call chains, and coordinate data with events, sagas and the transactional outbox.
  • An API gateway is the single entry point (routing, auth, rate limits); service discovery maps names to live instances; config comes from Config Server or Kubernetes.
  • Resilience toolkit: timeouts on every call, retries with exponential backoff and jitter only for safe and transient failures, circuit breakers to fail fast, bulkheads to isolate dependencies, and designed fallbacks.
  • Resilience4j and Spring Cloud provide production versions; Spring Framework 7 adds @Retryable and @ConcurrencyLimit in the core.

Next: Messaging with Kafka and RabbitMQ shows how services communicate asynchronously with queues and topics.