Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 36 ยท Servers: STUN, TURN and SFU

SFU in Practice: Simulcast, SVC and Layer Selection

See how simulcast and scalable video coding let an SFU send each viewer the right quality. Configure sendEncodings with rid, read parameters and simulate layer selection.

An SFU forwards video without decoding it, which is what makes it cheap. But a pure forwarder has a problem: if Ana sends one 720p stream at 1.5 Mbit/s, the SFU can only pass that same stream to everyone. A viewer on a weak mobile connection cannot receive it, and everyone else is stuck with the quality of the worst link, or the weak viewer just freezes.

The fix is to have the sender produce several qualities at once, so the SFU can pick the right one per viewer without touching the pixels. Two techniques do this: simulcast and SVC. This lesson shows how a browser is asked to produce them and how an SFU chooses between them.

Honest note: the browser side is real code you can run here. The SFU itself is a server product (mediasoup, LiveKit, Janus, Jitsi Videobridge and others) and cannot run in a page, so the selection logic is shown as a labelled stand-in.

Simulcast: several independent encodings

With simulcast, the sender encodes the same camera into, for example, three separate RTP streams of different resolution and bitrate, all in one RTCRtpSender. Each stream gets an identifier called a rid (restriction identifier).

rid Resolution Typical bitrate
f (full) 1280x720 1.5 Mbit/s
h (half) 640x360 500 kbit/s
q (quarter) 320x180 150 kbit/s

The sender uploads about 2.2 Mbit/s in total, and the SFU forwards exactly one of the three to each viewer.

flowchart LR
  S["Sender camera"] --> E["Encoder makes q, h, f"]
  E --> SFU(("SFU"))
  SFU -- "f" --> V1["Viewer on fast Wi-Fi"]
  SFU -- "h" --> V2["Viewer on 4G"]
  SFU -- "q" --> V3["Viewer on weak link or small thumbnail"]

One upload, three outputs, no transcoding on the server.

You request simulcast with sendEncodings when you add a transceiver:

JS Ask the browser for three simulcast layers

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Notice scaleResolutionDownBy: the browser makes the q layer four times smaller in each dimension (so 640x360 becomes 160x90). The rid values are labels chosen by you; the SDP gets a=rid and a=simulcast lines that the SFU reads. Order the layers from lowest to highest quality, which most browsers expect.

You can switch layers on and off while sending, which is how senders save CPU when the SFU says nobody needs the top layer:

JS Turn a layer off with setParameters

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Always call getParameters() first, change the object it returns, then pass it to setParameters(). Browsers reject parameters that do not come from a recent getParameters().

SVC: one stream with layers inside

Scalable Video Coding puts the layers inside one stream. A single encoding carries a base layer plus enhancement layers; the SFU drops the higher ones to reduce quality, by looking only at packet headers. You ask for it with scalabilityMode:

JavaScript
pc.addTransceiver(track, {
  direction: 'sendonly',
  sendEncodings: [{ scalabilityMode: 'L3T3' }], // 3 spatial and 3 temporal layers
});

Common modes: L1T3 (one resolution, three frame-rate layers), L3T3 (three resolutions, three frame rates), and L3T3_KEY / S3T3 variants. SVC works with VP9 and AV1 (and H.264 in some browsers) and uses less upload than simulcast, because the layers share data. Support varies, so check RTCRtpSender.getCapabilities('video') and have a simulcast fallback.

Simulcast SVC
Streams uploaded Several independent One layered
Upload cost Higher Lower
Codec support Any (VP8, H.264...) VP9, AV1, some H.264
SFU switching Switch streams (needs keyframe) Drop layers (smooth)

How an SFU chooses a layer

For each viewer, the SFU estimates the available download bandwidth and checks how large the video is shown on screen. Then it forwards the best layer that fits both. When a switch is needed it asks the sender for a keyframe so the new layer can start cleanly.

JS Stand-in: pick a layer per viewer

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Two inputs drive the decision: the link capacity and the tile size. A thumbnail never needs the 720p layer, even on gigabit fibre, so smart SFUs save bandwidth on both ends.

Dominant speaker and last-N

Large rooms add a second trick: forward video only for the last N active speakers (say 5) and let everyone else be audio-only or a still image. With an SFU, cost is roughly participants * N instead of participants squared. Combined with simulcast, this is how rooms with hundreds of participants stay affordable.

Common mistakes

  • Wrong layer order. List from low to high quality.
  • Forgetting that simulcast costs CPU. Three encodes per camera; limit it on weak devices.
  • Calling setParameters without getParameters first.
  • Assuming SVC everywhere. Check codec capabilities and keep a fallback.
  • Not planning keyframes. Layer switches wait for one; frequent switching causes visible freezes.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the rid in sendEncodings identify?

    Each simulcast stream gets a restriction identifier, which appears in the SDP so the SFU can tell the layers apart.
  2. 2. How does SVC differ from simulcast?

    Simulcast uploads independent streams; SVC uploads one stream whose layers can be dropped.
  3. 3. Why does an SFU request a keyframe when it switches a viewer to another simulcast layer?

    Delta frames reference earlier frames of their own stream; a switch needs a self-contained frame to start decoding.
  4. 4. What must you do before calling sender.setParameters()?

    Browsers require the parameters object to originate from a recent getParameters() call.

Summary

  • Simulcast makes several independent encodings (q, h, f) in one sender; an SFU forwards one per viewer.
  • SVC embeds layers in one stream and is cheaper to upload but depends on codec support.
  • Set layers with sendEncodings; adjust live with getParameters() then setParameters().
  • SFUs choose layers by bandwidth and tile size, and use last-N speakers in big rooms.

Next: Reading getStats for Call Quality