Skip to content
Tutorials Center
My learning

Topics

Lesson 38 of 54 ยท Detection with Machine Learning

Face Detection with Haar Cascades

Detect faces with OpenCV's CascadeClassifier and tune detectMultiScale. Learn how Haar features and integral images work, with NumPy examples and privacy notes.

A Haar cascade is one of the oldest practical object detectors in computer vision, and it is still shipped with OpenCV. It was designed in the early 2000s (the Viola-Jones method) to find faces in real time on very modest hardware. It is fast, needs no GPU and works well for frontal faces under decent light. It is also easy to fool, which makes it a good place to learn how sliding-window detectors work and why their parameters matter.

For anything demanding, such as faces at angles, small faces or many object classes, neural networks do better; they are the subject of The dnn Module: Running Neural Networks. But cascades remain useful as a quick, dependency-free baseline, and the ideas inside them (integral images, simple features, early rejection) show up everywhere.

The cv2 code below is read-only because opencv-python and its cascade XML files do not run in the browser. The live examples implement the integral image and a Haar-like feature in NumPy, and a toy sliding-window scan. They explain how the real detector works, but they are not a face detector.

Using CascadeClassifier

OpenCV ships trained cascade files in its data/haarcascades folder, and the Python package exposes the path as cv2.data.haarcascades:

Python
import cv2

path = cv2.data.haarcascades + "haarcascade_frontalface_default.xml"
face_cascade = cv2.CascadeClassifier(path)
if face_cascade.empty():
    raise RuntimeError("Could not load the cascade file: " + path)

img = cv2.imread("group.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gray = cv2.equalizeHist(gray)                # helps under uneven lighting

faces = face_cascade.detectMultiScale(
    gray,
    scaleFactor=1.1,
    minNeighbors=5,
    minSize=(40, 40),
)

for (x, y, w, h) in faces:                   # faces is an array of (x, y, w, h)
    cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)

print(len(faces), "face(s) found")

Details that trip people up:

  • CascadeClassifier(path) does not raise on a bad path; it just loads nothing. Check empty().
  • Detection runs on a grayscale image.
  • The result is an array of (x, y, w, h) rectangles, or an empty tuple when nothing is found, so len(faces) works either way.
  • Other bundled cascades include haarcascade_eye.xml, haarcascade_smile.xml, haarcascade_profileface.xml and haarcascade_fullbody.xml. Quality varies a lot; the frontal face one is the most reliable.

How detectMultiScale works

A cascade classifier looks at a fixed-size window (24 by 24 pixels for the default face model). To find faces of any size at any position, detectMultiScale scans the window over the image, then shrinks the image by scaleFactor and scans again, and repeats until the image is smaller than the window.

The detectMultiScale loop

Slide Test a small window at every position.
Rescale Shrink the image by scaleFactor, repeat.
Group Merge overlapping hits into one box.
Filter Keep boxes with minNeighbors hits.
1 / 2

Each window passes through a cascade of stages. Early stages use only a handful of features and reject the obvious non-faces (sky, wall, floor) immediately. Only windows that survive every stage count as detections. Because the great majority of windows are background, the average cost per window is tiny.

Tuning the parameters

Parameter What it controls Effect
scaleFactor How much the image shrinks between scans (default 1.1) Smaller (1.05) finds more sizes but is slower; larger (1.3) is faster but can miss faces between scales
minNeighbors How many overlapping detections a box needs to survive Higher means fewer false positives but more missed faces
minSize Smallest face to look for, in pixels Larger values skip tiny false hits and run faster
maxSize Largest face to look for Useful when you know the faces are small

True faces tend to be hit by many neighboring windows and scales, while false positives usually get one or two hits. That is why minNeighbors is the main knob for false positives. In practice you start with scaleFactor=1.1, minNeighbors=5, set minSize to the smallest face you care about, and adjust by looking at results.

Common reasons for false positives and misses:

  • False positives on textured backgrounds, such as bricks, foliage and patterned fabric. Raise minNeighbors and minSize.
  • Missed faces when the head is tilted, turned, partly hidden or in shadow. Haar cascades are trained on mostly frontal, upright faces. Rotating the image and detecting again is a hack, not a solution.
  • Nothing found because the image is huge. The window size is fixed, so a face of 600 pixels needs many rescales; resize the image first to a moderate size.

The integral image

The trick that makes cascades fast is the integral image (also called a summed-area table). Each pixel stores the sum of everything above and to the left of it. With it, the sum of any rectangle needs only four lookups, no matter how large the rectangle is:

TEXT
rect_sum = I[bottom, right] - I[top-1, right] - I[bottom, left-1] + I[top-1, left-1]

OpenCV computes it with cv2.integral(img), which returns an array one row and one column larger than the input (padded with a leading row and column of zeros).

Python Integral image and constant-time rectangle sums

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Once the integral image is built (one pass over the picture), every rectangle sum costs the same four additions, whether it covers 4 pixels or 4,000. That is what lets a cascade evaluate thousands of features per window at video speed.

Haar-like features

A Haar-like feature compares the sum of pixels in adjacent rectangles. A two-rectangle edge feature, for example, subtracts the sum of a dark rectangle from the sum of a bright one beside it. Faces have regular brightness structure that such features capture: the eye region is usually darker than the cheeks below it, and the bridge of the nose is brighter than the eyes on either side. Training (with the AdaBoost algorithm on thousands of face and non-face images) selects the few thousand most useful features and arranges them into the cascade stages.

The stand-in builds a toy 12 by 12 "face" with a dark eye band above a bright cheek area, plus a flat gray patch. It then evaluates a two-rectangle feature (upper rectangle minus lower rectangle) using the integral image.

Python A Haar-like feature on a toy face

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The toy face gives a strong positive value because the eye band is much darker than the cheek band. The flat wall gives exactly 0. A single feature like this is a weak classifier, and random noise may pass it by chance; the real cascade combines hundreds of them per stage, and noise has to pass all the stages. That is the power of boosting plus a cascade: many weak, cheap tests, with early rejection.

The sliding window and grouping

The last stand-in shows why minNeighbors exists. It slides a window over a scene with a large bright "object" and a small bright distraction, collects the windows that say "yes", groups the touching ones and keeps only groups with enough members.

Python Why minNeighbors filters false hits

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The big object makes 157 neighboring windows fire in one connected group, while the small distraction makes only 15. A minNeighbors of 20 removes the small group and keeps the object, which is the trade-off you tune in detectMultiScale: too low keeps false positives, too high starts removing real but weakly detected objects. (OpenCV groups rectangles by similarity of position and size rather than by flood fill, but the counting idea is the same.)

Faces in video: counting, blurring and landmarks

A cascade classifier is fast enough to run on every frame of a webcam. The loop is the one from Reading and Writing Video with a detection step in the middle:

Python
cap = cv2.VideoCapture(0)
while True:
    ok, frame = cap.read()
    if not ok:
        break
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = cascade.detectMultiScale(gray, scaleFactor=1.2, minNeighbors=5, minSize=(60, 60))

    for (x, y, w, h) in faces:
        face = frame[y:y + h, x:x + w]
        frame[y:y + h, x:x + w] = cv2.GaussianBlur(face, (51, 51), 30)   # privacy blur
        # or draw: cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    cv2.putText(frame, f"faces: {len(faces)}", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 255), 2)
    cv2.imshow("faces", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

len(faces) is the face count. For a more accurate count, raise minNeighbors, set minSize to ignore tiny false positives, and expect to miss turned or partly hidden faces, since the default model is for frontal faces. Blurring the region is the simplest anonymization; a pixelated block (resize small, then back up with INTER_NEAREST) looks more clearly "censored". A webcam stream is normally a different resolution from a photo, so resize frames before detection to keep it fast.

Facial landmarks (read-only). A box says where a face is; landmarks are 68 points on the eyes, brows, nose, mouth and jaw. OpenCV has cv2.face.createFacemarkLBF in the contrib package, and the dlib library offers a popular 68-point predictor:

Python
import dlib

detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")   # a separate download

for rect in detector(gray):
    shape = predictor(gray, rect)
    for i in range(68):
        p = shape.part(i)
        cv2.circle(frame, (p.x, p.y), 2, (0, 255, 0), -1)

Landmarks drive face alignment, expression analysis, virtual make-up and head-pose estimation with cv2.solvePnP. They need extra libraries and model files, so they cannot run in this course's browser examples; the cascade logic above is the OpenCV-only part.

Ethics and privacy

Face detection is technically simple and socially sensitive. Detecting that a face exists is different from identifying who it is, but both involve people's images, and in many places the law regulates it.

  • Consent and notice. Tell people when a camera analyzes them, and get consent where required. Rules such as the GDPR in Europe treat images of identifiable people as personal data, and some regions restrict biometric processing.
  • Data minimization. If you only need to count people or blur faces, do not store the images. Process on the device and keep results, not frames.
  • Bias. Detectors trained on unbalanced data work worse for some skin tones, ages and head poses. Test on varied, representative images and do not rely on a detector for decisions about people.
  • Purpose. Use detection for clear, limited purposes such as autofocus, privacy blurring or accessibility. Think twice about surveillance and profiling uses, and never use it to track individuals without their knowledge.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does face_cascade.detectMultiScale(gray, ...) return when no face is found?

    detectMultiScale returns an array of (x, y, w, h) rectangles, or an empty tuple when nothing is detected. len(faces) is 0 either way.
  2. 2. You get many false positives on a brick wall. Which change helps most?

    False hits usually have few overlapping detections. A higher minNeighbors discards them, and a larger minSize skips tiny false matches.
  3. 3. Why does an integral image make Haar features fast?

    Each integral image pixel holds the sum of everything above and to the left, so a rectangle sum is a combination of four entries.
  4. 4. Why is a cascade fast on typical photos?

    Most windows contain background and fail the first stage, so the average work per window is very small.
  5. 5. Which is the best description of what the frontal face cascade struggles with?

    The model was trained mostly on upright frontal faces, so poses far from that, occlusion and harsh lighting cause misses.

Summary

  • cv2.CascadeClassifier(path) loads a trained XML cascade; check empty(), and run detectMultiScale on a grayscale image.
  • detectMultiScale slides a fixed window over a pyramid of scales and merges overlapping hits.
  • scaleFactor, minNeighbors and minSize trade speed, false positives and missed detections.
  • Integral images make any rectangle sum a four-lookup operation, which Haar features exploit.
  • A cascade rejects most windows in early cheap stages, so it runs in real time on a CPU.
  • Haar cascades work best on upright frontal faces; deep networks are better for harder cases.
  • Treat face detection as sensitive: get consent, minimize stored data and test for bias.

Next: The dnn Module: Running Neural Networks