Skip to content
Tutorials Center
My learning

Topics

Lesson 5 of 54 ยท Getting Started

Mouse Events, Trackbars and Interactive Windows

Make OpenCV windows interactive: setMouseCallback events, click coordinates and colors, drawing and cropping with the mouse, trackbars for live tuning, selectROI and waitKey.

So far a window has only shown an image. OpenCV's built-in GUI module, HighGUI, can also react to the mouse and the keyboard and can add sliders. That is enough to build small but genuinely useful tools: click a pixel to read its color, drag a box to crop an object, move sliders to find the right threshold for a color, or paint a mask for inpainting. These interactive tools are how most computer-vision developers tune their parameters.

This lesson covers cv2.setMouseCallback, the event and flag constants, trackbars (createTrackbar), cv2.selectROI and the details of cv2.waitKey. HighGUI needs a desktop and the GUI build of OpenCV (see Installing OpenCV), so the OpenCV code is read-only here, and runnable Python models show the logic.

Windows and the event loop

A window is created by name (explicitly with namedWindow, or implicitly by the first imshow). Nothing happens on screen, and no mouse or key events are delivered, unless your program calls cv2.waitKey() regularly, because that call also runs the GUI's event loop.

Python
import cv2

img = cv2.imread("photo.jpg")
cv2.namedWindow("view", cv2.WINDOW_NORMAL)     # allows the window to be resized by the user

while True:
    cv2.imshow("view", img)
    key = cv2.waitKey(20) & 0xFF               # wait up to 20 ms for a key
    if key == ord("q") or key == 27:           # q or Esc
        break

cv2.destroyAllWindows()

waitKey(0) waits forever for a key; waitKey(n) waits n milliseconds and returns -1 if no key was pressed. The & 0xFF keeps the low byte, which makes the key code the same on all platforms. ord("q") gives the code of a character. cv2.WINDOW_AUTOSIZE (the default) fixes the window to the image size.

Mouse callbacks

You register one function per window with cv2.setMouseCallback. OpenCV calls it for every mouse event, with five arguments:

Python
def on_mouse(event, x, y, flags, param):
    ...

cv2.setMouseCallback("view", on_mouse, param)
Argument Meaning
event What happened, one of the cv2.EVENT_* constants
x, y Pixel position of the cursor: x is the column, y is the row
flags Which buttons and modifier keys are held (cv2.EVENT_FLAG_*)
param The extra object you passed when registering, handy for sharing state

Common events:

Constant When
EVENT_MOUSEMOVE The pointer moved
EVENT_LBUTTONDOWN / EVENT_LBUTTONUP Left button pressed / released
EVENT_RBUTTONDOWN / EVENT_RBUTTONUP Right button
EVENT_MBUTTONDOWN Middle button
EVENT_LBUTTONDBLCLK Left button double-click
EVENT_MOUSEWHEEL Wheel scrolled (the sign of flags tells the direction)

Flags are bit masks you test with &: flags & cv2.EVENT_FLAG_LBUTTON is true while the left button is held, and flags & cv2.EVENT_FLAG_CTRLKEY while Ctrl is down. Combining a MOUSEMOVE event with that flag is how you detect dragging.

Example: show the coordinates and color of a click

Python
def on_click(event, x, y, flags, param):
    if event == cv2.EVENT_LBUTTONDOWN:
        b, g, r = img[y, x]                    # row first, then column!
        print(f"({x}, {y})  BGR=({b}, {g}, {r})")
        cv2.putText(img, f"{x},{y}", (x, y), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 255), 1)
        cv2.imshow("view", img)

The most common mistake is the order: the callback gives (x, y) but the array is indexed [y, x], exactly as described in Accessing and Modifying Pixels. Also, if the window was resized with WINDOW_NORMAL, the coordinates are still reported in image pixels.

Example: draw with the mouse

Keep state in a dictionary or a small class, because the callback has no other way to remember things between calls.

Python
state = {"drawing": False, "last": None}
canvas = np.zeros((400, 600, 3), np.uint8)

def paint(event, x, y, flags, state):
    if event == cv2.EVENT_LBUTTONDOWN:
        state["drawing"], state["last"] = True, (x, y)
    elif event == cv2.EVENT_MOUSEMOVE and state["drawing"]:
        cv2.line(canvas, state["last"], (x, y), (255, 255, 255), 3)
        state["last"] = (x, y)
    elif event == cv2.EVENT_LBUTTONUP:
        state["drawing"] = False

cv2.setMouseCallback("canvas", paint, state)

Example: drag a rectangle and cut out the object

Record the point where the button goes down, update the rectangle while it moves, and on release slice the image:

Python
def select(event, x, y, flags, state):
    if event == cv2.EVENT_LBUTTONDOWN:
        state["start"] = (x, y)
    elif event == cv2.EVENT_LBUTTONUP:
        x0, y0 = state["start"]
        x1, y1 = x, y
        x0, x1 = sorted((x0, x1))
        y0, y1 = sorted((y0, y1))
        state["crop"] = img[y0:y1, x0:x1].copy()     # cropping is slicing

Sorting the two corners lets the user drag in any direction. This is also the first step of selecting a region to track or match.

selectROI: a ready-made rectangle chooser

For the common "let me pick a region" task, OpenCV has a function that does the whole dragging interaction:

Python
x, y, w, h = cv2.selectROI("pick", img, showCrosshair=True, fromCenter=False)
cv2.destroyWindow("pick")
if w > 0 and h > 0:
    roi = img[y:y + h, x:x + w]

The user drags a box, then presses Enter or Space to confirm (or c to cancel). It returns (x, y, width, height), and zeros if cancelled. cv2.selectROIs returns several boxes. Note that the format is (x, y, w, h), not two corners.

Trackbars

A trackbar is a slider attached to a window. It gives you a live integer in the range 0 to count without any GUI toolkit.

Python
def nothing(value):
    pass

cv2.namedWindow("palette")
cv2.createTrackbar("B", "palette", 0, 255, nothing)
cv2.createTrackbar("G", "palette", 0, 255, nothing)
cv2.createTrackbar("R", "palette", 0, 255, nothing)
cv2.createTrackbar("0 : OFF \n1 : ON", "palette", 0, 1, nothing)   # a switch

swatch = np.zeros((300, 512, 3), np.uint8)

while True:
    cv2.imshow("palette", swatch)
    if cv2.waitKey(1) & 0xFF == 27:
        break
    b = cv2.getTrackbarPos("B", "palette")
    g = cv2.getTrackbarPos("G", "palette")
    r = cv2.getTrackbarPos("R", "palette")
    on = cv2.getTrackbarPos("0 : OFF \n1 : ON", "palette")
    swatch[:] = (b, g, r) if on else 0

Facts about trackbars:

  • createTrackbar(name, window, initial, maximum, onChange); the callback receives the new value, and you may pass a do-nothing function (it cannot be None in older Python builds).
  • Values are integers from 0 up, so for a ratio such as 0.0 to 1.0, use 0 to 100 and divide, and for a gamma value use an offset (gamma = pos / 10).
  • The name is the label, and a two-state trackbar (maximum 1) makes a switch, because HighGUI has no checkbox.
  • Poll with getTrackbarPos, or do the work inside the callback; set a value from code with setTrackbarPos.

Trackbars are the standard way to find good parameters for a color filter. Create six sliders for the lower and upper HSV bounds, call cv2.inRange in the loop, and watch the mask change; when it looks right, copy the numbers into your code. You will use this technique in Project: Color Object Detector.

Try it: a mouse-event simulator

HighGUI is just an event dispatcher. The model below delivers a scripted sequence of events to a callback, tracks dragging with the flag bit, and crops a NumPy "image" exactly the way the rectangle-selection example does.

Python Drag to select a region

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Dragging up and to the left still produces a correct crop, because the two corners are sorted. Note the final line: the click is at x=6, y=5 but the array is indexed [5, 6].

Try it: a trackbar palette and a threshold tuner

Trackbars deliver integers, and that is all you need to model them. The first part turns three slider values into a color swatch description and converts it to a hex code. The second part simulates tuning a threshold slider on a small grayscale image and shows how many pixels each position selects.

Python Trackbar values drive the result

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

What is real: the callback signature, event constants, flag bits, the [y, x] indexing rule, selectROI return format, integer-only trackbars and the waitKey loop are how HighGUI works. What is the stand-in: the simulator feeds scripted events to the same callback shape; real windows need a desktop and a GUI build of OpenCV, and key and mouse constants are looked up by name in real code.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. In a mouse callback you receive x=120 and y=40. Which pixel of the NumPy image is that?

    The callback gives (x, y) = (column, row), but NumPy indexes [row, column], so the pixel is img[40, 120].
  2. 2. Why must a program using OpenCV windows call cv2.waitKey regularly?

    Without waitKey the window never repaints and no events arrive, so the window appears frozen.
  3. 3. How do you detect that the mouse is being dragged with the left button held?

    Mouse-move events carry flags that tell which buttons are down, so checking the left-button flag tells you a drag is in progress.
  4. 4. What range of values can a trackbar produce?

    Trackbars are integer sliders starting at 0. Scale or offset the value in your code for decimals or negative ranges.
  5. 5. What does cv2.selectROI return?

    selectROI returns the rectangle as (x, y, w, h). Slice with img[y:y+h, x:x+w] to crop.

Summary

  • HighGUI windows are driven by cv2.waitKey: call it every loop, and mask the result with & 0xFF.
  • cv2.setMouseCallback(window, fn, param) calls fn(event, x, y, flags, param); test EVENT_* constants, use flags bits for held buttons or keys, and index images as [y, x].
  • Keep state between events in param; draw, crop or select regions inside the callback. cv2.selectROI does a rectangle selection for you and returns (x, y, w, h).
  • cv2.createTrackbar makes integer sliders (and switches); read with getTrackbarPos. Use them to tune thresholds and HSV ranges live.
  • These tools need a GUI build of OpenCV and a desktop, but the logic is ordinary Python.

Next: begin working with color on the pixel level in Color Spaces: BGR, RGB, Gray and HSV.