Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 39 ยท Projects and Practice

Project: A Chat Service with Bidirectional Streaming

Build a gRPC chat service with bidirectional streams: protocol design, room broker, history replay, backpressure, rate limiting and graceful shutdown, with a runnable Go broker.

Chat is the classic showcase for bidirectional streaming: the client sends messages whenever the user types, and the server pushes messages from other users whenever they arrive. Neither side waits for the other. In this project you design the streaming protocol, build the room broker that does the real work, and learn the production concerns that toy examples skip: slow consumers, history on join, rate limiting and shutdown.

The gRPC server and generated stream types cannot run on this site, so protocol and handler code are read-only. The broker (rooms, subscribers, history, backpressure, limits) is plain Go with channels and runs here. A gRPC stream's Send and Recv map directly onto the channels used in the demo.

The protocol

Use one long-lived stream per user session. The first message must be a Join, after which the client sends Say messages; the server sends Events.

PROTO
service ChatService {
  rpc Chat(stream ClientMessage) returns (stream ServerEvent);
}

message ClientMessage {
  oneof kind {
    Join join = 1;
    Say say = 2;
    Leave leave = 3;
  }
}
message Join  { string room = 1; string user = 2; int64 resume_after_seq = 3; }
message Say   { string text = 1; }
message Leave {}

message ServerEvent {
  int64 seq = 1;                       // per-room, gapless, increasing
  google.protobuf.Timestamp at = 2;
  oneof kind {
    Message message = 3;
    Presence presence = 4;             // someone joined or left
    Notice notice = 5;                 // server notices such as "slow down"
  }
}
message Message  { string user = 1; string text = 2; }
message Presence { string user = 1; bool online = 2; }
message Notice   { string text = 1; }

Design choices: a oneof makes each message exactly one kind and evolves well (add a new member later); a per-room sequence number seq gives every event a stable order and lets a reconnecting client say "resume after 42"; presence and notices are events, not RPCs, so a client has one place to process everything.

sequenceDiagram
    participant A as Alice
    participant S as Chat server
    participant B as Bob
    A->>S: Join(room=go, user=alice)
    S-->>A: history (last N events)
    B->>S: Join(room=go, user=bob)
    S-->>A: Presence(bob online)
    A->>S: Say("hi")
    S-->>A: Message(alice: hi) seq 3
    S-->>B: Message(alice: hi) seq 3
    B->>S: Leave
    S-->>A: Presence(bob offline)

One stream per user; the server fans each event out to every member of the room.

The stream handler (read-only)

A bidirectional handler has two jobs that must run at the same time: reading what the client sends and writing what the server pushes. Use one goroutine per direction and finish when either ends:

Go
func (s *server) Chat(stream chatv1.ChatService_ChatServer) error {
    first, err := stream.Recv()
    if err != nil {
        return err
    }
    join := first.GetJoin()
    if join == nil || join.GetUser() == "" {
        return status.Error(codes.InvalidArgument, "first message must be Join with a user")
    }

    sub, history := s.broker.Join(join.GetRoom(), join.GetUser(), join.GetResumeAfterSeq())
    defer s.broker.Leave(sub)

    ctx := stream.Context()
    g, ctx := errgroup.WithContext(ctx)

    g.Go(func() error { // server -> client
        for _, ev := range history {
            if err := stream.Send(toProto(ev)); err != nil {
                return err
            }
        }
        for {
            select {
            case <-ctx.Done():
                return ctx.Err()
            case ev, ok := <-sub.Events:
                if !ok {
                    return status.Error(codes.ResourceExhausted, "too slow, disconnected")
                }
                if err := stream.Send(toProto(ev)); err != nil {
                    return err
                }
            }
        }
    })

    g.Go(func() error { // client -> server
        for {
            msg, err := stream.Recv()
            if err == io.EOF {
                return nil
            }
            if err != nil {
                return err
            }
            if say := msg.GetSay(); say != nil {
                if err := s.broker.Say(sub, say.GetText()); err != nil {
                    return err
                }
            }
        }
    })
    return g.Wait()
}

Two rules apply. Only one goroutine may call Send on a stream and only one may call Recv; the two directions may run in parallel. And the handler returning is what ends the stream, so every path must return.

The broker (runnable)

The broker owns rooms. Each subscriber has a bounded channel; when it is full the subscriber is too slow, and the broker disconnects it instead of letting memory grow or blocking everyone else. Each room keeps a ring of recent events for history and resume.

Go Chat room broker with history and backpressure

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Walk through what happens: the history ring gives Bob the recent messages on join; every broadcast gets a sequence number; Bob's bounded queue overflows and he is disconnected with a clear error instead of stalling Alice; Carol resumes by sequence number and receives only newer events.

Production concerns

  • Backpressure: always bound the per-client queue, as above. Unbounded buffers turn one slow phone into a server out-of-memory event.
  • Rate limiting: limit messages per user (a token bucket in an interceptor or in Say); answer abuse with RESOURCE_EXHAUSTED plus a RetryInfo detail (rich errors).
  • Authentication: authenticate the stream once, when it opens (metadata is sent with the first headers), and take the user name from the token, not from the Join message, or users could impersonate each other. See token authentication.
  • Liveness: configure keepalive so dead mobile connections are noticed (see tuning). Stream deadlines are rarely appropriate; use MaxConnectionAge with graceful reconnect and the resume_after_seq field instead.
  • Scaling out: with several server instances a room's members may be on different machines. Fan out through a shared pub/sub (Redis, NATS, Kafka), or route all users of a room to one instance with consistent hashing (see load balancing).
  • Graceful shutdown: send a Notice ("server restarting, reconnect"), then call GracefulStop with a timeout and Stop afterwards; streams are long-lived, so GracefulStop alone can wait forever.
  • Metrics: active streams, messages per second, kicked consumers, per-room size.

Stretch goals

  1. Add typing indicators as a new oneof member and observe that old clients ignore it.
  2. Persist history in a database and serve resume_after_seq from it.
  3. Add direct messages with Say { string to = 2; }.
  4. Write a bufconn test with three clients: message ordering, kick on overflow, resume.
  5. Compare this design with Server-Sent Events or WebSockets: what do you gain from protobuf and flow control, and what do browsers need (gRPC-Web does not support bidirectional streaming)?

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is the per-subscriber queue bounded?

    Unbounded buffering lets one slow consumer consume all memory; a bounded queue allows detecting and dropping it.
  2. 2. How many goroutines may call Send on one gRPC stream at the same time?

    Send is not safe for concurrent use on the same stream. One goroutine may Send while another Recvs.
  3. 3. What does resume_after_seq let a reconnecting client do?

    Per-room sequence numbers give events a stable order, so a client can ask for what it missed.
  4. 4. Where should the server get the user's identity from?

    Identity must come from verified credentials. A user field inside the message can be forged.

Summary

  • Bidirectional streaming fits chat: one long-lived stream per user, oneof messages in both directions.
  • A handler runs a sender and a receiver goroutine and ends when either does; only one goroutine may Send, only one may Recv.
  • The broker fans out events with per-room sequence numbers, keeps a history ring and uses bounded queues for backpressure.
  • Add rate limits, authenticate on stream open, plan resume, scaling out and graceful shutdown.

Next: A gRPC Production Checklist