Skip to content
Tutorials Center
My learning

Topics

Lesson 51 of 54 ยท Projects

Project: Invisible Cloak

Build the invisible cloak effect: capture a clean background, mask a cloth color in HSV (including the red hue wrap), clean the mask and replace those pixels with the background.

In stories, an invisibility cloak makes whoever wears it disappear and shows whatever is behind them. With OpenCV you can fake the effect on video, and it is a great project because it is small, visual and uses ideas from several earlier lessons: color spaces, masks, morphology and compositing.

The trick is simple. Take a picture of the empty background first. Then, for every frame, find the pixels that have the color of the cloak, and replace exactly those pixels with the stored background pixels. Everything else shows the live camera. Because the cloth is replaced by what is behind it, it seems to be transparent.

You will build the project step by step: capture the background, choose a color range in HSV (red is special), clean the mask, composite, and then think about the weak points. Review Color Spaces, Arithmetic, Bitwise Ops and Masking and Morphological Operations first. The OpenCV program is read-only because it needs a webcam; NumPy models run the same masking logic on a synthetic scene.

The plan

  1. Capture the background with nobody in the frame, from a fixed camera. Average or take the median of several frames to reduce noise.
  2. Read each live frame and convert it to HSV.
  3. Make a mask of pixels whose hue, saturation and value fall in the cloak's color range.
  4. Clean the mask with morphological opening and dilation.
  5. Composite: where the mask is set take the background, elsewhere take the live frame.
  6. Show the result and stop on a key press.

The camera must not move, because the stored background has to line up with each frame pixel by pixel. A tripod or a laptop on a table is enough.

Which color for the cloak?

Pick a cloth that is strongly colored and not found elsewhere in the scene. Red and green are popular; blue is also easy. Avoid colors similar to your skin, hair or walls.

In HSV, a color is its hue (the type of color), saturation (how pure) and value (how bright). OpenCV scales hue to 0 to 179 (degrees divided by 2) and saturation and value to 0 to 255. Greens are around hue 35 to 85 and blues around 90 to 130. Red is special because the hue wheel wraps around: pure red sits at hue 0, and slightly purple-red is at 170 to 179. So a red cloak needs two ranges, one near 0 and one near 179, combined with OR:

Python
lower_red1, upper_red1 = (0, 120, 70), (10, 255, 255)
lower_red2, upper_red2 = (170, 120, 70), (180, 255, 255)

The minimum saturation (120) and value (70) stop gray, white and very dark pixels from counting as red. Tune these numbers with trackbars (Mouse Events, Trackbars and Interactive Windows): move six sliders and watch the mask until the cloak is white and everything else is black.

The program

Python
import cv2
import numpy as np
import time

cap = cv2.VideoCapture(0)
time.sleep(2)                                   # let the camera adjust its exposure

# 1. Capture the background: several frames, then the median.
frames = []
for _ in range(60):
    ok, frame = cap.read()
    if ok:
        frames.append(frame)
background = np.median(np.stack(frames), axis=0).astype(np.uint8)

kernel_open = np.ones((3, 3), np.uint8)
kernel_dilate = np.ones((7, 7), np.uint8)

while True:
    ok, frame = cap.read()
    if not ok:
        break

    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

    # 2. Two hue ranges for red, combined.
    mask1 = cv2.inRange(hsv, (0, 120, 70), (10, 255, 255))
    mask2 = cv2.inRange(hsv, (170, 120, 70), (180, 255, 255))
    mask = cv2.bitwise_or(mask1, mask2)

    # 3. Clean up: remove specks, then grow a little to cover the cloth's edge.
    mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel_open, iterations=2)
    mask = cv2.dilate(mask, kernel_dilate, iterations=1)
    inverse = cv2.bitwise_not(mask)

    # 4. Composite: background where the cloak is, live frame everywhere else.
    cloak_area = cv2.bitwise_and(background, background, mask=mask)
    rest = cv2.bitwise_and(frame, frame, mask=inverse)
    result = cv2.addWeighted(cloak_area, 1, rest, 1, 0)

    cv2.imshow("invisible", result)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

Walk through the key lines:

  • The median of 60 frames gives a clean background even if the picture flickers a little, which is better than a single frame.
  • inRange returns 255 where the pixel is in range and 0 elsewhere, so it is a ready-made mask. Two masks are OR-ed to cover both ends of the red hue.
  • MORPH_OPEN (erosion then dilation) deletes tiny isolated specks that happen to be reddish. dilate then enlarges the mask slightly, because the cloth's soft edge pixels are often a blend of cloth and background and fall just outside the exact range.
  • bitwise_and(..., mask=...) keeps pixels only where the mask is non-zero. The two kept parts do not overlap (one uses the mask, the other its inverse), so adding them with addWeighted(..., 1, ..., 1, 0) (or cv2.add) combines them without brightness change.

Making it better

  • Flicker at the edges. Increase the dilation or blur the mask with cv2.GaussianBlur(mask, (9, 9), 0) and blend with the mask as an alpha: result = frame * (1 - a) + background * a, where a = mask / 255.0.
  • Holes inside the cloak. The cloth has shadows (darker value) that fail the value range. Lower the minimum value, or close the mask with MORPH_CLOSE.
  • Lighting changes. The background reflects the room's light; auto-exposure shifts brightness when the cloak enters the frame. Lock the camera exposure if the driver allows it, and capture the background in the same light.
  • Static background only. If people or objects move in the background after you captured it, they will appear frozen inside the cloak. Update the background slowly in regions where the mask is 0.
  • The red problem. Skin, wood and some walls can fall in the red range. A cloak in a very unusual color (bright green, neon pink) avoids false positives, and you can restrict the effect to the largest contour: find contours of the mask and keep only the biggest (Finding and Drawing Contours).
  • Use the dnn module. A person-segmentation neural network (The dnn Module) would make a cloak of any color, or even no cloak at all, possible.

This is the same compositing idea as the green screens used in film: key out a color, and replace it with another image. Replace background with a different picture, and you have a virtual background effect.

Try it: HSV, the red wrap and the composite

OpenCV is not available in the browser, so this model converts RGB to OpenCV-style HSV in NumPy, builds the double red mask on a tiny scene (a gray-blue wall, a skin-tone person and a red cloak) and composites it with a stored background.

Python Mask a red cloak and swap in the background

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The cloak area becomes background and the person is untouched. The stray noise pixel is removed by the cleanup, which is what MORPH_OPEN does in OpenCV.

Try it: why red needs two ranges

The last model checks a sweep of hues. A single range around red misses half of the reds, and combining two ranges covers both ends of the hue wheel.

Python The hue wheel wraps around

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

What is real: the plan, the HSV ranges and wrap-around for red, inRange, morphological cleanup, mask compositing and the program structure are how this effect is built with OpenCV. What is the stand-in: the NumPy models use a tiny made-up scene, a hand-written HSV conversion that follows OpenCV's scaling and a simple neighbor test instead of MORPH_OPEN; the full program needs a webcam and a fixed camera position.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does a red cloak need two cv2.inRange masks?

    Red sits at both ends of OpenCV's 0 to 179 hue scale, so you need one range near 0 and one near 179, combined with OR.
  2. 2. Why capture the background as the median of many frames?

    A median across frames suppresses sensor noise and moving specks, giving a clean reference.
  3. 3. What does bitwise_and(frame, frame, mask=inverse) do?

    The mask argument keeps pixels where it is non-zero. With the inverse mask, you keep everything except the cloak.
  4. 4. Why is the mask dilated slightly?

    A little dilation hides the soft border of the cloth so no red fringe remains.
  5. 5. What must stay fixed for the effect to look right?

    The background is replaced pixel by pixel, so the camera cannot move after the background is captured.

Summary

  • The invisible cloak replaces the pixels of a colored cloth with pixels from a stored image of the empty background.
  • Capture the background from a fixed camera (use the median of many frames), convert frames to HSV and select the cloth color with cv2.inRange. Red needs two ranges (near 0 and near 179).
  • Clean the mask with MORPH_OPEN and a little dilation, then composite with bitwise_and and addWeighted.
  • Improve it with a soft alpha mask, a closed mask for shadows, locked exposure, background updating and contour filtering. The same idea gives green-screen and virtual-background effects.

Next: apply what you have learned to Practice Exercises with Solutions.