Skip to content
Tutorials Center
My learning

Topics

Lesson 7 of 37 ยท Interfaces and Components

Images, Audio, Video and Files

Work with Gradio media components: Image type options, Audio as (sample rate, array), Video, File, upload limits and the temporary-file rules behind them.

Media components are where Gradio really saves time: an image upload box with a webcam, a microphone recorder, a video player and a file drop zone are each one line of Python. But the Python values you receive depend on options such as type, and getting them wrong gives confusing errors. This lesson covers the value types, the options and the safety rules. Gradio code is read-only; stand-ins with NumPy show the data shapes.

Image

Python
import gradio as gr

def to_gray(img):
    # img is a NumPy array of shape (height, width, 3) with uint8 values
    return img.mean(axis=2).astype("uint8")

demo = gr.Interface(
    fn=to_gray,
    inputs=gr.Image(label="Photo"),
    outputs=gr.Image(label="Grayscale"),
)

The page shows a drop zone ("Drop Image Here - or - Click to Upload") with webcam and clipboard buttons, plus an output image. After an upload, the function receives a NumPy array. The relevant options:

Option Values Effect
type "numpy" (default), "pil", "filepath" Which Python type your function receives
image_mode "RGB", "RGBA", "L" Color mode on load
sources ["upload", "webcam", "clipboard"] Which input buttons appear
height, width pixels Display size
interactive bool False makes it display-only

Pick the type to match your library:

  • numpy for OpenCV-style and NumPy code (array of uint8, shape (H, W, 3); beware that OpenCV expects BGR order, Gradio gives RGB).
  • pil for Pillow code and many Hugging Face pipelines.
  • filepath when the library wants a path (the file is a temporary copy).

For output, you may return a NumPy array, a PIL image or a file path, and the component converts.

Audio

Python
audio = gr.Audio(sources=["microphone", "upload"], type="numpy")

With type="numpy" the function receives a tuple (sample_rate, data), where data is a NumPy array of integers (often 16-bit) with shape (samples,) for mono or (samples, channels) for stereo. With type="filepath" you receive a path to a temporary file. To return audio, produce the same tuple: return sample_rate, array. Streaming microphone input needs streaming=True and a generator or a stream event (see generators and streaming).

A frequent bug: integer audio must be converted before math, because int16 arithmetic overflows. Convert to float32 and divide by 32768 to work in the range -1 to 1, then convert back before returning.

Video and File

Python
gr.Video(label="Clip")        # file path in, file path out
gr.File(label="Upload", file_count="multiple", file_types=[".csv", ".txt"])
gr.UploadButton("Add files", file_types=["image"])
gr.Gallery(label="Results")   # list of images

Video gives your function a file path, because videos are too large to hold as arrays. File gives a path (or a list of paths when file_count="multiple"). Use file_types to filter in the picker and validate again on the server: the extension filter is a convenience, not security.

How Gradio stores files

Uploads are saved to a temporary directory managed by Gradio (inside the system temp folder), and your function receives the path of that copy. Rules that follow:

  • Treat the path as read-only input; copy it if you must keep it, because the cache is cleaned up eventually (delete_cache can be configured).
  • Outputs written as files must be inside an allowed location. Gradio only serves files from its temp cache, the working directory's allowed paths and paths you declare with allowed_paths=[...] in launch(). This protects your disk from path tricks.
  • Never pass an arbitrary user-provided string as a path to your own open() calls.

Do not add allowed_paths=["/"] to make an error go away. That makes any file readable by anyone who can reach the app. Allow the narrowest folder that holds your assets.

Stand-in 1: an image is just an array

Python Stand-in: process an image as a NumPy array

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last line shows the bug that the conversion avoids: 255 + 100 wraps to 99 in uint8.

Stand-in 2: audio as (sample_rate, array)

Python Stand-in: build and halve a tone

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

This is the same tuple a real gr.Audio(type="numpy") hands your function and expects back.

Practical advice

  • Resize big images before running a model. Phone photos are millions of pixels and slow down everything.
  • Cap upload size and duration: check os.path.getsize(path) or the array length at the top of the function and raise gr.Error when too large.
  • Handle None: when the user presses Submit with no upload, media inputs give None.
  • Clean up temp files you create (use tempfile.NamedTemporaryFile(delete=False) and remove later, or let Gradio's cache handle returned files).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does gr.Image() give your function by default?

    type="numpy" is the default. Use type="pil" or "filepath" for the other forms.
  2. 2. What is the Python value for gr.Audio(type="numpy")?

    The tuple holds the sampling rate and the samples. You return the same shape for output audio.
  3. 3. Why convert int16 audio to float32 before multiplying?

    Scaling in float and converting back avoids overflow.
  4. 4. Is file_types a security check?

    A client can send any file through the API, so validate on the server.

Summary

  • Image gives NumPy by default; choose pil or filepath to match your library.
  • Audio gives (sample_rate, array); convert integers to floats for math.
  • Video and File pass temporary file paths.
  • Uploads live in a managed temp cache; keep allowed_paths narrow.
  • Handle None, large files and wrong types in the function.

Next: Choices, Sliders and Dates