Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 37 ยท Sharing and Deployment

Capacity Planning, Share Links and Going Public

Estimate how many visitors a Gradio app can serve, simulate waiting lines and bounded queues, and learn what share=True, server_name and launch options expose before you go public.

You have a demo that works on your laptop. Before you send the link around, ask two questions: how many people can use it at once, and who exactly can reach it? The queue lesson explains the settings (default_concurrency_limit, concurrency_limit, max_size). This lesson is about using them: estimating capacity, seeing the effect of slots in a simulation, and understanding the launch options that decide who can reach your machine.

The real Gradio is not available on this site, so its code is read-only. The runnable blocks are plain-Python simulations, labelled stand-ins, so you can try the numbers.

A picture of the line

flowchart LR
    U1["Visitor 1"] --> Q["Waiting line<br/>(max_size)"]
    U2["Visitor 2"] --> Q
    U3["Visitor 3"] --> Q
    Q --> W1["Slot 1"]
    Q --> W2["Slot 2"]
    W1 --> R["Results back to each browser"]
    W2 --> R

Requests wait in one line; a limited number of slots run your function at once.

Capacity comes from two numbers: how long one call takes, and how many slots you have. If a call takes 4 seconds and you have 2 slots, you finish about 30 calls a minute. If more than 30 visitors a minute press Submit, the line grows without limit.

Simulating waiting times

This simulation gives each request a fixed length and lets you vary the number of slots, the same trade-off as default_concurrency_limit.

Python How many slots do you need?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The numbers fall quickly as slots are added, because requests start earlier for everyone behind the first ones. The catch in real life: more slots need more memory and compute, so a GPU model usually stays at one or two slots, while a function that mostly waits on a hosted API can use many.

Bounding the line with max_size

An unbounded line is a trap: if visitors arrive faster than you serve them, the wait grows forever. A bounded queue turns that into a clear "busy" message.

Python A bounded queue

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first request starts, the next three wait and the last two are rejected. In Gradio a rejected visitor sees an error that the queue is full, which is better than an hour-long wait.

A quick capacity formula

Throughput is slots divided by seconds per call. This block turns that into a planning table:

Python Throughput table

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A 20-second model call with one slot serves only 3 visitors a minute, which is why heavy models get batching, smaller models or paid GPUs, not just more slots.

Launch options: who can reach the app

launch() decides the network exposure:

Argument Meaning
server_name="127.0.0.1" (default) Only your own machine can connect
server_name="0.0.0.0" Every machine on the network can connect; needed inside Docker
server_port=7860 The port; Gradio tries following ports if it is busy
share=True Creates a temporary public *.gradio.live URL that tunnels to your machine
max_threads=40 Worker threads for plain functions, above the concurrency limits
show_error=True Shows Python exceptions in the browser (development only)
prevent_thread_lock=True Returns from launch() instead of blocking, for tests and notebooks
Python
demo.launch(share=True)
# Running on local URL:  http://127.0.0.1:7860
# Running on public URL: https://<random>.gradio.live

A share link works only while your script runs and expires after about a week. Anyone who has the URL can run your function on your machine, so treat it as public. It is handy for showing a colleague a demo in five minutes, but not a deployment: your laptop must stay on, and your queue limits are your only protection. For anything lasting, use a server or a Space, covered in deploying to Hugging Face Spaces, and add authentication if the function is private or expensive.

Before you share: a checklist

  1. Set default_concurrency_limit to what your hardware can handle, and max_size so waits stay reasonable.
  2. Put limits on input size (max_file_size, text length).
  3. Remove secrets from the code and from error messages.
  4. Test with several browser tabs at once and watch memory.
  5. Decide what happens when the app is full: a polite message beats a hang.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A call takes 5 seconds and you have 2 slots. About how many calls per minute can you serve?

    60 * 2 / 5 = 24 calls per minute.
  2. 2. What does max_size protect against?

    When the line is full, new requests are rejected instead of waiting forever.
  3. 3. What is true of share=True?

    The link works while your script runs and exposes your function to anyone with the URL.
  4. 4. Why use server_name="0.0.0.0" in a container?

    The default 127.0.0.1 is reachable only from inside the container.

Summary

  • Capacity is slots divided by seconds per call; a bounded queue turns overload into a clear message.
  • Network-bound functions can use many slots; GPU-bound ones rarely more than one or two.
  • share=True is a temporary public tunnel, not a deployment.
  • Check limits, secrets and error messages before sending the link to anyone.

Next: Deploying to Hugging Face Spaces