Skip to content
Tutorials Center
My learning

Topics

Lesson 21 of 37 ยท Models and Demos

Image and Audio Demos

Build Gradio demos for pictures and sound: gr.Image types, gr.Audio formats, numpy arrays, file paths, sample rates, plus runnable pixel and waveform stand-ins.

Images and sound are where Gradio shines, because uploading a photo or recording your voice in the browser would otherwise take a lot of front-end work. The catch is knowing what your function receives. An image can arrive as a NumPy array, a PIL object or a file path; audio can arrive as a sample rate plus an array, or as a path. This lesson explains each option and why you would choose it.

Real Gradio cannot run on this site, so its code is read-only. The runnable blocks use plain Python lists to play the role of pixels and audio samples, which is exactly the data a model gets after Gradio's conversion. They are labelled stand-ins.

The Image component

Python
import gradio as gr
import numpy as np

def invert(img):
    return 255 - img          # img is an H x W x 3 uint8 array

demo = gr.Interface(
    fn=invert,
    inputs=gr.Image(type="numpy", label="Upload"),
    outputs=gr.Image(label="Inverted"),
)
demo.launch()

After launch, the page shows an upload box (with a webcam and clipboard option), and after you drop a photo in, the output panel shows the photo with its colors inverted. The type argument decides what your function receives:

type= Your function gets Use it when
"numpy" (default) numpy.ndarray, shape (height, width, 3), uint8 NumPy, OpenCV, scikit-image
"pil" PIL.Image.Image Pillow, torchvision transforms, Hugging Face processors
"filepath" str path to a temp file Libraries that read from disk, large images

The returned value can be any of the three as well. Other useful arguments: sources=["upload", "webcam"], image_mode="RGB" or "L" (grayscale), height/width for display and format="png" for the output file type.

Related components: gr.ImageEditor (draw, crop, mask), gr.AnnotatedImage (boxes and masks drawn over a picture, perfect for object detection), gr.Gallery (many images) and gr.Sketchpad for hand-drawn input such as digit recognition.

Pixels are just numbers

A color image is a grid of pixels; each pixel is three numbers from 0 to 255 (red, green, blue). Everything a "filter" does is arithmetic on those numbers. The block below builds a 4-by-4 image as nested lists and applies the two classic operations.

Python Pixels as numbers (stand-in for a NumPy array)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Red becomes (0, 255, 255) when inverted (cyan), and the gray version turns each pixel into three equal values. With NumPy the same inversion is just 255 - img, because arrays apply arithmetic to every element at once.

Resizing and normalizing for a model

Most vision models want a fixed size and floats between 0 and 1 (or standardized with a mean and deviation). That is your preprocessing step, not Gradio's:

Python
def preprocess(img):
    pil = Image.fromarray(img).resize((224, 224))
    x = np.asarray(pil, dtype="float32") / 255.0
    x = (x - mean) / std
    return x[None, ...]          # add a batch dimension

Two details matter in practice. Uploaded photos can be huge; resizing before the model keeps memory in check. And uploaded PNGs can have an alpha channel, so set image_mode="RGB" on the component so the model always sees three channels.

The Audio component

Python
import gradio as gr

def loudness(audio):
    sample_rate, samples = audio       # type="numpy" returns a tuple
    peak = abs(samples).max()
    return f"{len(samples) / sample_rate:.1f} seconds, peak {peak}"

demo = gr.Interface(
    fn=loudness,
    inputs=gr.Audio(sources=["microphone", "upload"], type="numpy"),
    outputs="text",
)

The input shows a record button and an upload area; the output text reads something like 3.2 seconds, peak 18344. With type="numpy" the function receives (sample_rate, array); with type="filepath" it receives a path to a WAV file, which suits speech models and libraries like librosa or soundfile that read files. The sample rate (samples per second, commonly 16,000, 22,050 or 44,100) must match what your model was trained on, so resample when needed.

Audio output works in reverse: return (sample_rate, array) or a file path and the page shows a player. streaming=True on a microphone input sends chunks as the user talks, for live transcription.

A runnable stand-in: synthesizing and measuring a waveform

An audio signal is a list of numbers sampled at a fixed rate. This block generates one second of a 440 Hz tone at a low sample rate, then computes duration, peak and RMS loudness, the same kind of work as the loudness function above.

Python Audio as samples (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A pure sine wave of amplitude 0.5 has an RMS of about 0.354 (amplitude divided by the square root of 2). After normalizing, the peak is 0.9. Gradio's Audio returns integer sample arrays for some formats, so divide by 32768 if your model expects floats between -1 and 1.

Practical tips

  • Keep file sizes reasonable: use max_file_size="10mb" in launch() and show a clear message when a clip is too long.
  • Provide examples with sample files so visitors can try the demo without uploading anything.
  • Return the processed image plus a text or label output for a richer demo: outputs=[gr.Image(), gr.Label()].
  • For webcams, set streaming=True and a small stream_every to build live demos, but make sure predict is fast enough.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which gr.Image type gives the function a NumPy array?

    type="numpy" is the default and returns an (H, W, 3) uint8 array.
  2. 2. What does gr.Audio(type="numpy") give the function?

    numpy audio arrives as the sample rate plus the sample array.
  3. 3. Why set image_mode="RGB" on an image input?

    RGBA images would have four channels and break models expecting three.
  4. 4. Which component draws boxes and masks over a picture?

    gr.AnnotatedImage overlays bounding boxes or segmentation masks with labels.

Summary

  • gr.Image(type=...) chooses between NumPy arrays, PIL images and file paths.
  • Pixels are numbers from 0 to 255; filters are arithmetic on them.
  • gr.Audio gives (sample_rate, array) or a file path; match the sample rate to the model.
  • Resize and normalize inside your own preprocessing step.
  • Provide examples and file-size limits so demos stay fast and friendly.

Next: Text and NLP Demos