Skip to main content

OpenCV Object Detection and Tracking

Shenzhen, China

Github Repo

OpenCV Haarcascades​

To identify objects I am going to use the provided OpenCV Haarcascades from Github.com:

You only need a few lines of code to perform an object detection on an image (image source pexels.com):

import cv2

image = cv2.imread('resources/group_of_people.jpg')

detector = cv2.CascadeClassifier('cascades/haarcascade_fullbody.xml')
image_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

detections = detector.detectMultiScale(image_gray)
print('[INFO] People detected: ' + str(len(detections)))

for (x, y, w, h) in detections:
    cv2.rectangle(image, (x, y), (x+w, y+h), (0, 0, 255), 2)

cv2.imshow('People', image)

cv2.waitKey(0)
cv2.destroyAllWindows()

OpenCV Object Detection and Tracking

Using Haarcascades to find Objects​

In the previous Object Tracking Example I selected the object I wanted to track by hand. We can now use the OpenCV object detection feature to automatically find people in our video and return the coordinates for a bounding box:

import cv2
import sys

tracker = cv2.legacy.TrackerCSRT_create()

video = cv2.VideoCapture('resources/group_of_people_03.mp4')
if not video.isOpened():
    print('[ERROR] loading video')
    sys.exit()

ok, frame = video.read()
if not ok:
    print('[ERROR] loading frame')
    sys.exit()

detector = cv2.CascadeClassifier('cascades/haarcascade_fullbody.xml')


def detect():
    while True:
        ok, frame = video.read()
        frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        detections = detector.detectMultiScale(frame_gray, minSize=(300, 300))
        print('[INFO] people detected: ' + str(len(detections)))

        for (x, y, w, h) in detections:
            cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 0, 255), 2)
            cv2.imshow('Detections', frame)
            cv2.waitKey(0)
            cv2.destroyAllWindows()
            if x > 0:
                print('[INFO] bounding box calculated')
                return x, y, w, h


bbox = detect()
print(bbox)

This code will search for a person inside the first frame of the image and return the bounding box coordinates once it found one (all videos are free stock from pexels.com):

OpenCV Object Detection and Tracking

Those coordinates can then be fed into the tracking algorithm that allows us to track the person recognized by OpenCV:

# initialize tracker with detected coordinates
ok = tracker.init(frame, bbox)
if not ok:
    print('[ERROR] loading tracker')
# generate random colours for bounding box
colours = (randint(0, 255), randint(0, 255), randint(0, 255))

# loop through all frames and apply tracker on detected object
while True:
    ok, frame = video.read()
    if not ok:
        print('[INFO] reached end of video')
        break
    # update bounding box in new frame using tracker
    ok, bbox = tracker.update(frame)
    if ok:
        (x, y, w, h) = [int(v) for v in bbox]
        cv2.rectangle(frame, (x, y), (x+w, y+h), colours, 5)
        # show tracking / resize frame if necessary
        resized = cv2.resize(frame, (1280, 720), interpolation=cv2.INTER_NEAREST)
        cv2.imshow('Tracking', resized)
        # press 'q' to break loop and close window
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

    else:
        print('[WARNING] No Track')
        # re-run detection if track was lost
        bbox = detect()
        # restart tracker if person detected
        tracker = cv2.legacy.TrackerCSRT_create()
        tracker.init(frame, bbox)

With this code we are entirely dependent on the Haarcascade to select objects to track. We can adjust this by first selecting the initial object by hand and only handing the selection to the algorithm if the track fails:

# call detection function
# if you want OpenCV to detect the initial object automatically
# bbox = detect()
# print(bbox)
# or use the ROI selector to select initial object yourself
bbox = cv2.selectROI(frame)

Now the detection code is skipped and I am presented with the manual selector instead (all videos are free stock from pexels.com):

OpenCV Object Detection and Tracking

The tracking algorithm will then take those coordinates and start tracking the region of interest. If it looses track it will trigger the object detection function to reacquire it:

OpenCV Object Detection and Tracking