Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 37 ยท State and Events

Generators and Streaming Output

Stream results from a Gradio function with yield, show progress with gr.Progress, stream audio and webcam input, and cancel long jobs cleanly. Includes runnable generator stand-ins.

A normal function returns once, so the user stares at a spinner until the whole result is ready. A generator function uses yield instead of return and sends results in pieces, so the page updates while the function is still working. This is how chatbots type word by word, how image models show intermediate steps, and how a long job reports progress. The Gradio code is read-only; stand-ins use real Python generators so you can see the sequence.

yield instead of return

Python
import time
import gradio as gr

def count_down(start):
    for n in range(int(start), 0, -1):
        yield f"T-minus {n}"
        time.sleep(1)
    yield "Liftoff!"

demo = gr.Interface(count_down, gr.Number(value=3), gr.Textbox())

Pressing Submit with 3 shows T-minus 3, then one second later T-minus 2, then T-minus 1, and finally Liftoff!. Each yield replaces the output value. Gradio detects generator functions automatically; no flag is needed. The Submit button turns into a Stop button (the stop_btn option) while the generator runs, and pressing it cancels the generator at its next yield.

Yielding several outputs

When the event has several outputs, yield a tuple with one value per output. To leave an output alone, yield gr.skip() (or gr.update() in older versions) for it:

Python
def work(n):
    for i in range(int(n)):
        yield f"step {i + 1}/{int(n)}", gr.skip()
    yield "done", "All finished"

Use skip to avoid re-sending an unchanged large output, such as an image, on every tick.

Streaming text, the chatbot way

Python
def stream_reply(prompt):
    reply = f"You said: {prompt}. Streaming is nice."
    shown = ""
    for word in reply.split():
        shown += word + " "
        yield shown
        time.sleep(0.05)

The text box grows word by word. Note the pattern: the function yields the whole text so far, not just the new word, because each yield replaces the component's value. (Chatbots in later chapters follow the same rule for the latest message.)

Progress bars

For work without natural stream pieces (a loop over files), use gr.Progress:

Python
def process(files, progress=gr.Progress()):
    results = []
    for file in progress.tqdm(files, desc="Processing"):
        results.append(len(file))
        time.sleep(0.2)
    return results

# or manually
def process2(n, progress=gr.Progress()):
    for i in range(int(n)):
        progress((i + 1, n), desc="Working")
        time.sleep(0.1)
    return "done"

Add a parameter whose default is gr.Progress() and Gradio injects the tracker. progress.tqdm(iterable) works like tqdm, drawing a bar and counts over the output component; progress(0.5) or progress((done, total)) sets it by hand. Progress is shown on the output components that the event writes to, for as long as the function runs. Event option show_progress="hidden" hides it.

Streaming input

Some components send data continuously: gr.Audio(streaming=True) delivers microphone chunks and gr.Image(streaming=True, sources=["webcam"]) delivers frames. These use the stream event:

Python
gr.Image(sources=["webcam"], streaming=True).stream(
    fn=detect, inputs=cam, outputs=annotated, time_limit=30, stream_every=0.5
)

stream_every sets how often to send a frame to your function and time_limit caps each user's streaming time. The function should be fast: frames queue up behind a slow model, so keep resolution small and drop frames with concurrency_limit settings (see queue and concurrency).

Cancelling and cleaning up

Pressing Stop (or closing the tab) stops the generator at the next yield by raising GeneratorExit inside it. Use try/finally to release resources:

Python
def long_job(n):
    handle = open_resource()
    try:
        for i in range(int(n)):
            yield f"{i}"
    finally:
        handle.close()          # runs on completion and on cancel

For work that is not a loop (one long model call), you cannot interrupt in the middle; the Stop button only takes effect when control returns to a yield. Design long jobs as many short steps.

Stand-in 1: a generator and its updates

This stand-in consumes a generator the way Gradio does, taking each yielded value as the new output.

Python Stand-in: consuming a generator

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first list is every page update, in order. The close() call raises GeneratorExit inside the generator, which is how a Stop click reaches your code.

Stand-in 2: a cleanup-safe generator and a progress bar

Python Stand-in: finally block and a text progress bar

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In both the finished and the cancelled case the resource is closed, which is why try/finally belongs in every generator that opens something.

Practical advice

  • Yield the complete current value, not a delta.
  • Keep yields reasonably spaced (hundreds of yields per second overwhelm the browser): yield every few tokens, not every character.
  • Use gr.skip() for outputs that did not change.
  • Generators and queue: generator events hold a concurrency slot until they finish, so very long streams reduce how many other users can be served.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What makes a Gradio function stream results?

    Gradio detects generator functions and updates the output on each yield.
  2. 2. In a text-streaming generator, what should each yield contain?

    Each yield replaces the component's value, so yield the whole current text.
  3. 3. How do you add a progress bar to a loop?

    Gradio injects the tracker when the parameter default is gr.Progress().
  4. 4. Why use try/finally in a generator that opens a file?

    Cancelling raises GeneratorExit at the yield, so cleanup code in finally runs.

Summary

  • yield turns a function into a streaming function; each yield replaces the output.
  • Yield whole values, use gr.skip() for unchanged outputs, and tune the yield rate.
  • gr.Progress and progress.tqdm show progress for loops.
  • Streaming components (stream event) send audio or frames continuously.
  • Use try/finally in generators so Stop and errors clean up.

Next: Queue, Concurrency and Rate Limits