Skip to content
Tutorials Center
My learning

Topics

Lesson 39 of 54 ยท Detection with Machine Learning

The dnn Module: Running Neural Networks

Run neural networks with OpenCV's dnn module: load an ONNX model with readNet, build a blob, call forward, decode YOLO-style output and apply non-max suppression.

The Haar cascades in the previous lesson are fast but old. Modern detectors and classifiers are neural networks, and OpenCV can run them through its dnn module. It does inference only: you train a model elsewhere (PyTorch, TensorFlow, and so on), export it to a portable file such as ONNX, and OpenCV loads that file and runs it on the CPU (or a GPU backend if your build supports one).

The nice part is that you need no deep-learning framework at runtime. The work for you is three steps: turn an image into the exact tensor the network expects, call forward, and turn the raw numbers into boxes you can draw. The first and last steps are plain array math, and that is what you will run in this lesson.

What is real and what is a stand-in. cv2.dnn needs the opencv-python package and a model file, neither of which exists in this page's sandbox, so the OpenCV calls below are read-only. The runnable examples re-create the preprocessing and the post-processing with NumPy (in real Pyodide). The network itself is replaced by a hand-made output array.

Inference in four steps

Load readNet reads the model file.
Blob blobFromImage builds the input tensor.
Forward net.forward runs the network.
Decode Turn numbers into boxes, then NMS.

Steps 2 and 4 are ordinary array code.

1 / 2

Loading a network

cv2.dnn.readNet(path) looks at the file extension and picks the right reader. For a model exported to ONNX you can also call readNetFromONNX directly.

Python
import cv2

net = cv2.dnn.readNet("model.onnx")      # downloaded or exported by you
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_OPENCV)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU)

print(net.getLayerNames()[:5])           # names of the first layers
print(net.getUnconnectedOutLayersNames())  # names of the output layers

Model files are large (tens of megabytes is normal), so they are not part of the opencv-python wheel. You download them once from the model's publisher, keep them next to your script, and check the license before shipping them. That step is the only one that needs a network connection.

Not every ONNX operator is supported. If readNet raises an error that names an unsupported layer, the model was exported with an operator the dnn module does not know. Re-export with a lower opset, or pick a simpler architecture.

The blob: what the network expects

A network does not take a normal image. It takes a blob: a 4-dimensional float array in NCHW order: batch, channels, height, width. The model's documentation states four things you must match exactly:

  1. The input size (for example 224 by 224 for many classifiers, 640 by 640 for common detectors).
  2. A scale factor (often 1/255 to bring 0..255 into 0..1).
  3. A per-channel mean to subtract (some models), and sometimes a per-channel division.
  4. The channel order. OpenCV images are BGR; most models were trained on RGB, so swapRB=True is common.
Python
blob = cv2.dnn.blobFromImage(
    image,                 # BGR uint8, shape (H, W, 3)
    scalefactor=1 / 255.0,
    size=(640, 640),       # (width, height)
    mean=(0, 0, 0),
    swapRB=True,
    crop=False,
)
print(blob.shape)          # (1, 3, 640, 640)

net.setInput(blob)
outputs = net.forward()    # one array, or a list for multi-output nets

Notice that size is (width, height), the opposite order from array shapes. Getting a preprocessing detail wrong rarely raises an error. The network just returns confident nonsense, which makes this the most common dnn bug.

Here is that same preprocessing written with NumPy, so you can see exactly what blobFromImage does. The resize here is a simple nearest-neighbor pick to keep the code short; the real function uses bilinear interpolation.

Python blobFromImage in NumPy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The red half of the picture ends up in channel 0 (the R plane) because of swapRB. If you forgot the swap, a model trained on RGB would see a blue sky as orange.

Classification output

A classifier returns one score per class. For ImageNet-style models that is a vector of 1000 numbers. You usually apply softmax to turn raw scores (logits) into probabilities, then take the largest.

Python
scores = net.forward().flatten()
probs = np.exp(scores - scores.max())
probs /= probs.sum()
top = probs.argmax()
print(labels[top], probs[top])

The labels come from a separate text file that matches the model's training set. The network knows only indices.

Detection output and decoding

A detector returns many candidate boxes, far more than there are objects. A YOLO-style export typically gives an array of shape (1, 5 + classes, N) or (1, N, 5 + classes), depending on the version, where each candidate holds a box center cx, cy, size w, h, and class scores. Always print outputs.shape and read the model's page before you decode.

Decoding is three moves: drop low-confidence candidates, convert cx, cy, w, h into corner coordinates, and scale from the network's input size back to your image size.

Python Decode a fake detector output

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Boxes here use the [x, y, width, height] form, which is what cv2.dnn.NMSBoxes expects.

Non-max suppression

Look at the first two candidates above: they describe nearly the same cat. Real networks produce dozens of such overlapping boxes per object. Non-max suppression (NMS) keeps the best box and removes the ones that overlap it too much.

"Overlap" is measured by IoU, intersection over union: the area both boxes share divided by the area they cover together. IoU is 0 for disjoint boxes and 1 for identical boxes.

The algorithm:

  1. Sort boxes by score, highest first.
  2. Keep the top box.
  3. Discard every remaining box whose IoU with it exceeds a threshold (0.4 to 0.5 is typical).
  4. Repeat with what is left.
Python IoU and non-max suppression

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The second box overlaps the first with an IoU above the 0.5 threshold and has the lower score, so it is removed. The dog box does not overlap anything and survives. In OpenCV you would write:

Python
keep = cv2.dnn.NMSBoxes(boxes, scores, score_threshold=0.5, nms_threshold=0.45)
for i in np.array(keep).flatten():
    x, y, w, h = boxes[i]
    cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 0), 2)

Wrapping keep in np.array(...).flatten() is a habit worth having, because older OpenCV versions returned an awkward nested shape.

Practical advice

  • Start small. Pick a compact model first (a MobileNet-class classifier or a "nano" detector) and measure speed before you reach for a large one.
  • Match the training pipeline. Size, scale, mean and channel order must all match the model's documentation.
  • Fix the input size. Many exports are baked for one size. Use letterboxing (resize with padding) to keep aspect ratio instead of stretching.
  • Time it. CPU inference runs from a few milliseconds to over a second per frame. See Making OpenCV Fast for how to measure.
  • Mind licenses and privacy. Models have licenses, and detecting people in video has legal and ethical implications in many places.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What shape does cv2.dnn.blobFromImage produce for one 640x480 color image resized to 640x640?

    A blob is NCHW: batch, channels, height, width. That gives (1, 3, 640, 640).
  2. 2. Why is swapRB=True often needed?

    cv2.imread returns BGR. A model trained on RGB would see swapped red and blue channels unless you swap them.
  3. 3. What does non-max suppression do?

    NMS keeps the best-scoring box and drops others whose IoU with it is above a threshold.
  4. 4. Two boxes are identical. What is their IoU?

    Intersection equals union for identical boxes, so IoU is exactly 1.

Summary

  • The dnn module runs inference on exported models (ONNX and others); it does not train.
  • readNet loads the model, blobFromImage builds an NCHW float input, forward returns raw output.
  • Size, scale, mean and channel order must match what the model was trained with.
  • Decode detections by thresholding, converting center boxes to corners, and rescaling.
  • Use NMS (based on IoU) to remove duplicate boxes.

Next, read machine-made codes in QR Codes, ArUco Markers and Text.