Skip to main content

OpenCV CAMshift Algorithm for Object Tracking

Shenzhen, China

Github Repo

CAMshift​

One of the problems with the Meanshift Algorithm was that the region of interest always stayed at the same size when the object came closer to the camera or moved farther away. The region needs to adapt it's size with size and rotation of the target. The solution is called CAMshift (Continuously Adaptive Meanshift) published by Gary Bradsky in his paper “Computer Vision Face Tracking for Use in a Perceptual User Interface” in 1988.

It applies meanshift first. Once meanshift converges, it updates the size of the window. It also calculates the orientation of best fitting ellipse to it. Again it applies the meanshift with new scaled search window and previous window location. The process is continued until required accuracy is met.

Get your Videostream​

Get your RTSP video stream input and define a region of interest for the Meanshift algorithm:

# get video stream from IP camera
print("[INFO] starting video stream")
vs = VideoStream(args["url"]).start()

# first frame from stream
frame = vs.read()
# select region of interest
bbox = cv2.selectROI(frame)
x, y, w, h = bbox
track_window = (x, y, w, h)
# define area of bounding box as area of interest
roi = frame[y:y+h, x:x+w]

Histogram Calculation in OpenCV​

The Meanshift algorithm is going to use the histogram of your region of interest to track the object you selected above. But we have to convert the frame to to the HSV colour space and normalize it first:

hsv_roi = cv2.cvtColor(roi, cv2.COLOR_BGR2HSV)
# get histogram for [0] blue, [1] green, [2] red channel
roi_hist = cv2.calcHist([hsv_roi], [0], None, [180], [0, 180])
# convert hist values 0-180 to a range between 0-1
roi_hist = cv2.normalize(roi_hist, roi_hist, 0, 255, cv2.NORM_MINMAX)

So now we use cv.calcHist() function to find the histogram. Let's familiarize with the function and its parameters :

cv.calcHist(images, channels, mask, histSize, ranges[, hist[, accumulate]])

  • images : it is the source image of type uint8 or float32. it should be given in square brackets, ie, [img].
  • channels : it is also given in square brackets. It is the index of channel for which we calculate histogram. For example, if input is grayscale image, its value is [0]. For color image, you can pass [0], [1] or [2] to calculate
  • histogram of blue, green or red channel respectively.
  • mask : mask image. To find histogram of full image, it is given as "None". But if you want to find histogram of particular region of image, you have to create a mask image for that and give it as mask. (I will show an example later.)
  • histSize : this represents our BIN count. Need to be given in square brackets. For full scale, we pass [256] for RGB and [180] for HSV. ranges : this is our RANGE. Normally, it is [0,256] for RGB and [0, 180] for HSV.

Apply the CAMshift Algorithm​

while True:
    # get next frame
    frame = vs.read()
    if True:
        # convert to hsv
        hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
        # compare blue channel of current with roi histogram
        # https://docs.opencv.org/3.4.15/da/d7f/tutorial_back_projection.html
        dst = cv2.calcBackProject([hsv], [0], roi_hist, [0, 180], 1)
        # call meanshift() to find match of histogram in current frame
        # and get the new coordinates
        ok, track_window = cv2.meanShift(dst, (x, y, w, h), parameter)
        if not ok:
            print('[WARNING] track lost')
        # now update the roi coordinates to new values
        x, y, w, h = track_window
        cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 255), 2)
        # display track
        cv2.imshow("Meanshift Track", frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    else:
        break

The generated coordinates can be used to draw polylines around the calculated new position of our selected object:

OpenCV Meanshift Algorithm for Object Tracking