Skip to content
Tutorials Center
My learning

Topics

Lesson 103 of 156 · Browser APIs

Media and Speech APIs

Capture the camera and microphone with getUserMedia, record with MediaRecorder, and talk with speech synthesis - permissions, constraints, errors and cleanup.

Browsers can do more than show pages. With a few lines of JavaScript, a page can ask for the camera or microphone, show the live picture in a <video> element, record a clip, or read a paragraph aloud. Video calls, voice notes, screen recorders, language-learning apps and accessibility tools are all built from the same handful of APIs.

This lesson covers the three you meet most often: getUserMedia() to capture a camera or microphone as a MediaStream, MediaRecorder to turn a stream into a file, and the Web Speech API to speak text (and, in some browsers, to listen). All of them involve permissions and hardware that can fail in many ways, so a large part of the lesson is about handling errors and cleaning up properly.

About the live editors. The editors on this site run in a sandbox with no camera, no microphone and no permission prompts, so real capture would only fail. The runnable demos here use small mock objects with the same methods and events, so you can see the logic that surrounds the real calls. The code that touches real hardware is shown read-only.

The big picture

Everything starts with a MediaStream: a bundle of live tracks (MediaStreamTrack), where each track is one audio or video source. You can send a stream to a <video> element, to a MediaRecorder, to a canvas (see Canvas Drawing from JavaScript) or to a WebRTC connection.

flowchart LR
  A["Camera and microphone"] -->|"getUserMedia()"| B["MediaStream<br/>(audio and video tracks)"]
  B --> C["video element<br/>(live preview)"]
  B --> D["MediaRecorder"]
  D -->|"dataavailable"| E["Blob chunks"]
  E --> F["One Blob: a playable or<br/>downloadable file"]

A stream flows from the hardware to a preview and, through MediaRecorder, into a file.

Capturing with getUserMedia

navigator.mediaDevices.getUserMedia(constraints) asks the browser to open devices. It returns a promise for a MediaStream. The browser shows a permission prompt the first time, and the user can refuse.

JavaScript
// Read-only: needs a real camera, a user gesture and an https:// page
const video = document.querySelector("#preview");

async function startCamera() {
  const stream = await navigator.mediaDevices.getUserMedia({
    audio: true,
    video: { width: { ideal: 1280 }, height: { ideal: 720 }, facingMode: "user" },
  });
  video.srcObject = stream; // note: srcObject, not src
  await video.play();
  return stream;
}

A few facts that surprise beginners:

  • HTTPS only. getUserMedia is available only in a secure context: https:// pages and http://localhost. On plain HTTP, navigator.mediaDevices is undefined.
  • Permissions can be blocked by policy. A page inside an <iframe> needs allow="camera; microphone" on the frame. This is the reason it fails inside sandboxed editors.
  • srcObject, not src. A MediaStream is not a URL. Set video.srcObject, and add the muted and playsinline attributes if you want autoplay to work, especially on phones.
  • Request only what you need. Asking for both audio and video when you need just a microphone is rude and increases the chance of a refusal.

Constraints

The argument is a set of constraints. Each property can be a plain value, { ideal: ... } (try to match, but accept something else), { min, max } or { exact: ... } (fail if impossible).

Constraint Example Meaning
width, height { ideal: 1280 } Video resolution
frameRate { max: 30 } Frames per second
facingMode "user" or "environment" Front or back camera on phones
deviceId { exact: id } One specific device
echoCancellation, noiseSuppression true Audio processing

Prefer ideal over exact. With exact, a laptop with a 720p camera simply fails when you ask for 4K.

Choosing a device

navigator.mediaDevices.enumerateDevices() lists cameras, microphones and speakers. Before the user grants permission, the label of each device is an empty string (to prevent fingerprinting), so build a device picker after the first successful capture.

JavaScript
// Read-only
const devices = await navigator.mediaDevices.enumerateDevices();
const cameras = devices.filter((d) => d.kind === "videoinput");
for (const camera of cameras) {
  console.log(camera.label, camera.deviceId);
}

Handling errors

getUserMedia() rejects with a DOMException, and the name tells you what happened. Each name deserves its own friendly message:

JS Turning capture errors into messages

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Show the message in the page, not only in the console, and always offer a way forward, such as a button that tries again or an upload field as a fallback.

Stopping the stream

A camera keeps running (and its indicator light stays on) until you stop every track. Removing the <video> element is not enough.

JavaScript
// Read-only
function stopStream(stream) {
  stream.getTracks().forEach((track) => track.stop());
}

Stop streams when the user leaves the screen, when you switch devices, and when the tab is hidden if your app does not need it. Screen sharing works the same way with navigator.mediaDevices.getDisplayMedia(), which always shows the browser's own picker.

Recording with MediaRecorder

new MediaRecorder(stream, options) records a stream. It does not give you one file at the end; it delivers chunks through the dataavailable event, and you assemble them into a Blob.

JavaScript
// Read-only
const recorder = new MediaRecorder(stream, { mimeType: "audio/webm" });
const chunks = [];

recorder.addEventListener("dataavailable", (event) => {
  if (event.data.size > 0) chunks.push(event.data);
});

recorder.addEventListener("stop", () => {
  const blob = new Blob(chunks, { type: recorder.mimeType });
  audio.src = URL.createObjectURL(blob); // play it back
});

recorder.start(1000); // a chunk every second; omit for one chunk at stop()
// ... later
recorder.stop();

Notes you will need in practice:

  • Supported formats differ. Chromium and Firefox record audio/webm or video/webm; Safari records audio/mp4 or video/mp4. Check with MediaRecorder.isTypeSupported("audio/webm") and fall back, or leave mimeType out and read recorder.mimeType afterward.
  • stop() flushes the last chunk. The final dataavailable fires before the stop event, so build the Blob in stop.
  • States. recorder.state is "inactive", "recording" or "paused"; pause() and resume() move between the last two.
  • Memory. Chunks live in memory. For long recordings, use the timeslice argument and upload chunks as they arrive.

The mock below follows exactly this contract, so you can watch the order of events and see how chunks add up to one Blob:

JS Collecting chunks into a Blob

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Once you have a Blob, you can play it with URL.createObjectURL(blob), upload it with fetch() and a FormData, or offer it for download. See Files and Blobs for those steps, and remember to call URL.revokeObjectURL() when you no longer need the URL.

Speech synthesis: making the page talk

The Web Speech API has two halves. Speech synthesis (text to speech) is available in all modern browsers and needs no permission. You create a SpeechSynthesisUtterance, choose a voice and call speechSynthesis.speak().

JavaScript
// Read-only: it would make sound, and it needs a user gesture in most browsers
const utterance = new SpeechSynthesisUtterance("Your order has shipped.");
utterance.lang = "en-US";
utterance.rate = 1;   // 0.1 to 10, 1 is normal
utterance.pitch = 1;  // 0 to 2
utterance.volume = 1; // 0 to 1

utterance.addEventListener("end", () => console.log("finished speaking"));
speechSynthesis.speak(utterance);

Voices come from the operating system, so every user has a different list. speechSynthesis.getVoices() may return an empty array on the first call, because voices load asynchronously. Wait for the voiceschanged event:

JavaScript
// Read-only
function loadVoices() {
  return new Promise((resolve) => {
    const voices = speechSynthesis.getVoices();
    if (voices.length) return resolve(voices);
    speechSynthesis.addEventListener("voiceschanged", () => {
      resolve(speechSynthesis.getVoices());
    }, { once: true });
  });
}

const voices = await loadVoices();
const english = voices.filter((v) => v.lang.startsWith("en"));
utterance.voice = english[0] ?? null; // null keeps the default voice

The synthesis object keeps a queue: calling speak() repeatedly queues utterances in order. cancel() empties the queue, pause() and resume() control playback. Some engines stop reading very long texts after roughly 15 seconds, so split long text into sentence-sized utterances. This mock shows the splitting and the queue:

JS Splitting text into a speech queue

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Speech recognition

The other half, SpeechRecognition (spelled webkitSpeechRecognition in Chromium), turns speech into text. It is not available everywhere, it needs microphone permission, and in Chromium the audio is sent to a remote service. Treat it as an optional enhancement and always keep a text input as the main path.

JavaScript
// Read-only
const Recognition = window.SpeechRecognition ?? window.webkitSpeechRecognition;

if (Recognition) {
  const recognition = new Recognition();
  recognition.lang = "en-US";
  recognition.interimResults = true;
  recognition.addEventListener("result", (event) => {
    const last = event.results[event.results.length - 1];
    console.log(last[0].transcript, last.isFinal ? "(final)" : "(interim)");
  });
  recognition.start();
} else {
  console.log("Speech recognition is not supported here.");
}

Project: a text-to-speech reader and a translator widget

Two small widgets put the pieces above to work, and both teach the same design lesson: depend on a small interface, not on one engine. Speech and translation features are patchy across browsers and devices, so each widget talks to an adapter with the same shape. The real engine is one implementation; a fallback is another. If the first one is missing, the widget still works.

Widget 1: A reader that highlights the sentence being spoken

The reader splits the text into sentences (long utterances get cut off by some engines, as noted above), speaks them one at a time and highlights the current one. It offers Play, Pause, Resume and Stop, plus voice, rate and pitch controls.

Four details keep it robust:

  • Feature detection. If speechSynthesis is missing, or the device has no voices, the widget switches to a silent practice engine that paces itself with timers. The highlighting still works, and a note explains what happened.
  • Voices load late. The select is filled again on voiceschanged.
  • Always cancel() before a new run. Otherwise old utterances queue up behind the new ones.
  • One sentence at a time. Speaking sentence by sentence (and chaining on end) makes the highlight exact and lets Stop work instantly. Utterance error events with the reason canceled or interrupted are normal after cancel() and are not real errors.
Text-to-speech reader with highlighting

Press Play. On a device with voices you will hear the text; otherwise you will see the highlight move at reading speed, which is enough to test the logic. The runId counter is a handy trick: each Play increments it, and the old loop notices that it is no longer the current run and quietly stops.

Note the last line: pagehide cancels speech so the browser does not keep talking after the visitor navigates away (some engines do).

Widget 2: A translator with a fallback chain

Translation has the same "which engine is available?" problem, plus a few of its own: it needs a network or a downloaded model, it can be slow, and answers can arrive out of order when someone types quickly. A resilient widget is built from three ideas:

  1. A chain of providers, tried in order: the built-in Translator API (an on-device model in recent Chromium browsers, which needs no server and keeps text private), then your own server endpoint (which would call a translation service with a secret key; never put that key in browser code), and finally a small offline phrasebook so the widget never shows an empty result for common phrases.
  2. Debouncing (wait until typing pauses) so you do not call a provider on every keystroke.
  3. Cancellation: each request carries a number, and an answer that is not the newest one is ignored, so a slow old answer can never overwrite a fresh one.

The built-in API looks like this. It is read-only here because most browsers (and this sandbox) do not have it:

JavaScript
// Read-only: the on-device Translator API (Chromium) with feature detection
if ("Translator" in self) {
  const pair = { sourceLanguage: "en", targetLanguage: "es" };
  const availability = await Translator.availability(pair);   // "unavailable" | "downloadable" | "downloading" | "available"
  if (availability !== "unavailable") {
    // Creating a translator may download a model, so it needs a user gesture (like a click)
    const translator = await Translator.create({
      ...pair,
      monitor(m) { m.addEventListener("downloadprogress", (e) => console.log(`model ${Math.round(e.loaded * 100)}%`)); },
    });
    console.log(await translator.translate("Good morning"));
  }
}

The demo implements the whole chain. Provider 1 uses the real API when it exists. Provider 2 is a stand-in for your server: tick Pretend the server is down to see the chain fall through. Provider 3 is the phrasebook.

Translator widget with fallback providers

Try "Good morning" (the server does not know it, so the phrasebook answers and the line under the result says which providers were skipped). Type "How are you?" and the server stand-in answers. Tick Pretend the server is down and the chain falls through to the phrasebook, or to the polite "can't translate" message if it does not know the phrase.

What to carry over to your own widgets:

  • Return where the answer came from (via). It makes debugging easy and lets you show honest messages such as "translated on your device".
  • Treat AbortError differently from real failures. A cancelled request is not a reason to try the next provider.
  • Keep secrets on the server. A browser can never hide an API key; your endpoint adds it.
  • Tell people when quality is limited. A phrasebook or machine translation is a convenience, not a guarantee, so link to the original text and avoid using it for anything legal, medical or safety-critical.

Good habits

  • Ask in context. Request the camera when the user presses "Start video call", not on page load. Pages that prompt immediately are often blocked by users.
  • Feature-detect. Check navigator.mediaDevices?.getUserMedia, typeof MediaRecorder !== "undefined" and "speechSynthesis" in window before you rely on them.
  • Respect privacy. Show clearly when you are recording, never upload without consent, and stop tracks as soon as you are done.
  • Offer alternatives. Captions for spoken output, file upload instead of live capture, typing instead of dictating. Some users have no hardware, and some cannot hear or speak.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. Why does navigator.mediaDevices.getUserMedia not exist on a plain http:// page?

    Capturing the camera or microphone is a powerful feature, so browsers expose it only in secure contexts.
  2. 2. What do you have to do to turn the camera light off after you stop showing the video?

    The device stays open until every track is stopped. Removing or clearing the element does not release the hardware.
  3. 3. Where should you build the final Blob from a MediaRecorder?

    stop() flushes the last chunk through dataavailable and only then fires stop, so all the data is in your array by that point.
  4. 4. speechSynthesis.getVoices() returns an empty array on the first call. What is the usual fix?

    Voices load asynchronously in many browsers. The voiceschanged event signals that the list is ready.

Summary

  • A MediaStream is a bundle of live tracks. getUserMedia() creates one from the camera or microphone and needs a secure context and the user's permission.
  • Use constraints with ideal values, handle each DOMException name (NotAllowedError, NotFoundError, NotReadableError, OverconstrainedError) with a clear message, and stop all tracks when finished.
  • Show a stream with video.srcObject. MediaRecorder delivers chunks through dataavailable; assemble them into a Blob in the stop event, and check which MIME types the browser supports.
  • Speech synthesis uses SpeechSynthesisUtterance and speechSynthesis.speak(). Wait for voiceschanged, split long text, and remember that voices differ per device.
  • Speech recognition is only partly supported and may send audio to a server; keep it optional.
  • Widgets that depend on patchy APIs (speech, translation) should talk to an adapter and fall back: a silent practice engine for speech, and a provider chain (built-in Translator, your server, an offline phrasebook) for translation, with debouncing and cancellation so only the newest answer is shown.

Next: Canvas Drawing from JavaScript - draw shapes, paths and animations on a <canvas>.