Knowledge system of Python OpenCV

The purpose of this article is to give you a detailed list of the learning route and important knowledge points of Python OpenCV. The core is divided into 24 small chunks, all of which are mastered.

Photo by Nguyen Dang Hoang Nhu on Unsplash

OpenCV first understanding and installation

In this part, you need to understand the related introduction of OpenCV (Open Source Computer Vision Library). OpenCV can run on multiple platforms. It is lightweight and efficient. It consists of a series of C functions and a small number of C++ classes. It provides Python, Ruby, MATLAB, etc. Language interface, so when you study, you should pay attention to the language implementation-related issues of consulting materials.

In addition to installing OpenCV related libraries at this stage, it is recommended to go through the official website, the official manual, and the official introductory tutorial. These are the best learning materials.

After the module is installed, you need to focus on testing whether OpenCV is installed successfully, and you can query the installed version through Python.

Introduction to OpenCV Module

First, grasp which modules OpenCV consists of from a global perspective. For example, in the following modules, you need to find the application scenarios and introductions of the following modules.

core, imgproc, highgui, calib3d, features2d, contrib, flann, gpu, legacy, ml, objdetect, photo, stitching

Organize the core functions of each module, complete the first OpenCV case, and read the display picture.

OpenCV image read, display, save

After installing OpenCV, start learning from image acquisition, including local loading of pictures, camera acquisition of pictures, video acquisition, and image creation.

Only after the image is acquired, can the image be manipulated, processed, information extracted, output, image displayed, and image saved.

For an image, the steps for reading and displaying in OpenCV are as follows, and you can correspond to its code.

• Image reading;
• Window creation;
• Image display;
• Image save;
• Resource release;

Function involved have to learn cv2.imread(), cv2.namedWindow(), cv2.imshow(), cv2.imwrite(), cv2.destroyWindow(), cv2.destroyAllWindows(), cv2.imshow(), cv2.cvtColor(), cv2.imwrite(), cv2.waitKey().

Camera and video read and save

The first thing to focus on learning VideoCapture class, common methods are:

• open() function;
• isOpened() function;
• release() function;
• grab() function;
• retrieve() function;
• get() function;
• set() function;

In addition to reading the video, you also need to master Opencv provided VideoWriter class that holds video files.

After learning the relevant knowledge, you can carry out such an experiment to save a video as a picture frame by frame.

OpenCV commonly used data structure and color space

This part of the class has to master Point class, Rect class, Size class, Scalar class, in addition, in Python using NumPy the image operation, the NumPy relevant knowledge, it is recommended to learn in advance, the better.

The commonly used color spaces in OpenCV include BGR color space, HSV/HLS color space, and Lab color space. All of these need to be understood, and the BGR color space should be mastered first.

OpenCV commonly used drawing functions

Master the usage of the following functions, you can draw graphics in Opencv proficiently.

• cv2.line();
• cv2.circle();
• cv2.rectangle();
• cv2.ellipse();
• cv2.fillPoly();
• cv2.polylines();
• cv2.putText();

Mouse and slider for OpenCV interface event operation

The first function to be mastered is the mouse operation message callback function cv2.setMouseCallback(). The slider involves two functions, namely: cv2.createTrackbar()and cv2.getTrackbarPos().

After mastering the above content, two cases can be realized. One is to drag the frame selection area on a picture with the mouse to take a screenshot, and the other is to use the slide bar to make the video play at double speed.

Image pixel, channel separation and merging

Understand the image pixel matrix, familiar with the pixel composition of the picture, you can access the pixel value of the specified pixel and modify it.

Channel separation function cv2.split(), channel combination function cv2.merge().

Image logic operations

To master the calculation between images, the functions involved are as follows:

• cv2.add();
• cv2.addWeighted();
• cv2.subtract();
• cv2.absdiff();
• cv2.bitwise_and();
• cv2.bitwise_not();
• cv2.bitwise_xor();

You can also study image multiplication and division.

Image ROI and mask

This part belongs to the key knowledge in OpenCV. The first is the region of interest ROI, and the second is the mask operation. When learning the ROI part, you can also learn about the dark and shallow copies of the image.

Image geometric transformation

Image geometric transformation is still the learning and understanding of basic functions, and the contents are as follows:

• Image zoom cv2.resize();
• Image translation cv2.warpAffine();
• Image rotation cv2.getRotationMatrix2D();
• Image transpose cv2.transpose();
• Image mirroring cv2.flip();
• Image remap cv2.remap();

Image filtering

Understand what filtering is, high frequency and low frequency filtering, image filtering function.

Linear filtering: box filtering, mean filtering, Gaussian filtering,
non-linear filtering: median filtering, bilateral filtering,

• Box filter cv2.boxFilter();
• Mean filtering cv2.blur();
• Gaussian filtering cv2.GaussianBlur();
• Median filtering cv2.medianBlur();
• Bilateral Filter cv2.bilateralFilter();

Image fixed threshold and an adaptive threshold

Image thresholding is an important basic part of image processing. It is widely used. Different parts of the image can be segmented according to the difference in grayscale. The image processed by thresholding is generally a single-channel image (grayscale). Two functions must be mastered at the core:

• Fixed threshold: cv2.threshold();
• Adaptive threshold: cv2.adaptiveThreshold().

Image expansion and corrosion

Dilation and erosion are morphological operations and are a series of image processing operations based on the shape of the image.
Expansion and corrosion are based on the operation of the highlighted part (white). Expansion is the expansion of the highlighted part, which is similar to “field expansion”. Corrosion is the corrosion of the highlighted part, similar to “the field is eroded.”

Application and function of swelling corrosion:

• Eliminate noise
• Split independent elements or connect adjacent elements;
• Find obvious maximum and minimum areas in the image;
• Find the gradient of the image;

The core functions that need to be mastered are as follows:

• Dilate cv2.dilate();
• Corrosion cv2.erode();

Other morphological operations, opening operation, closing operation, the top-hat, black hat, morphological gradient are based on the basis of the expansion of corrosion by cv2.morphologyEx() function operation.

Edge detection

Edge detection can extract important contour information of an image, reduce image content, and can be used for operations such as image segmentation and feature extraction.

The general steps of edge detection:

• Filtering: Filter out the influence of noise and detection edge;
• Enhancement: The intensity change of the pixel neighbourhood can be highlighted-gradient operator;
• Detection: Threshold method to determine the edge;

Common edge detection operators:

• Canny operator, Canny edge detection function cv2.Canny();
• Sobel operator, Sobel edge detection function cv2.Sobel();
• Scharr operator, Scharr edge detection function cv2.Scahrr();
• Laplacian operator, Laplacian edge detection function cv2.Laplacian();

Hough Transform

Hough Transform (Hough Transform) is a feature extraction technology in image processing. This process calculates the local maximum of the cumulative result in parameter space to obtain a set conforming to the specific shape as the result of the Hough Transform.

Functions to be learned in this part:

• Standard Hough transform, multi-scale Hough transform cv2.HoughLines();
• Cumulative probability Hough transform cv2.HoughLinesP();
• Hough circle transform cv2.HoughCricles();

Image histogram calculation and drawing

Histogram first master concepts, in grasp the core function, and finally through matplotlib to draw the histogram module. The function used to calculate the histogram is cv2.calcHist().

Histogram related applications:

• Histogram equalization cv2.equalizeHist();
• Histogram comparison cv2.compareHist();
• Back projection cv2.calcBackProject().

Template matching

Template matching is a technique to find the most matching (similar) part of an image with another template image.

The functions used by the core are as follows:

• Template matching cv2.matchTemplate();
• Matrix normalization cv2.normalize();
• Find the maximum value cv2.minMaxLoc().

Contour search and drawing

The core is to understand that in OpenCV, finding contours is like finding white objects on a black background.

Commonly used functions:

• Find the contour cv2.findContours();
• Draw contours cv2.drawContours().

Finally, you should master the operation for each contour.

Outline feature attributes and applications

This part of the content is more important, and there are more knowledge points, the core content and functions are as follows:

• Find the convex hull cv2.convexHull() and convexity detection cv2.isContourConvex();
• The outline circumscribed rectangle cv2.boundingRect();
• The minimum enclosing rectangle of the contour cv2.minAreaRect();
• The minimum circumscribed circle of the contour cv2.minEnclosingCircle();
• Contour ellipse fitting cv2.fitEllipse();
• Approximate polygon curve cv2.approxPolyDP();
• Calculate the contour area cv2.contourArea();
• Calculate the contour length cv2.arcLength();
• Calculate the distance and position relationship between the point and the contour cv2.pointPolygonTest();
• Shape matching cv2.matchShapes().

Advanced Part-Watershed Algorithm and Image Inpainting

Grasp the principle of the watershed algorithm and master the core functions cv2.watershed().

Can be extended to supplement image repair technology and related functions cv2.inpaint(), after learning, you can try the portrait freckle application.

GrabCut & FloodFill image segmentation, corner detection

This part of the content requires some professional background knowledge of images, first master the knowledge of related concepts, and focus on learning-related functions.

• GrabCut algorithm cv2.grabCut();
• Flood fill algorithm cv2.floodFill();
• Harris corner detection cv2.cornerHarris();
• Shi-Tomasi corner detection cv2.goodFeaturesToTrack();
• Sub-pixel corner detection cv2.cornerSubPix().

Feature Detection and Matching

The detection and matching of feature points are one of the very important technologies in computer vision, and it has a wide range of applications in the fields of object recognition, visual tracking, and three-dimensional reconstruction.

OpenCV provides the following feature detection methods:

• “FAST” FastFeatureDetector;
• “STAR” StarFeatureDetector;
• “SIFT” SIFT (nonfree module) Opencv3 is removed, xfeature2d library needs to be called;
• “SURF” SURF (nonfree module) Opencv3 removed, need to call xfeature2d library;
• “ORB” ORB Opencv3 is removed, xfeature2d library needs to be called;
• “MSER” MSER;
• “GFTT” GoodFeaturesToTrackDetector;
• “HARRIS” (with Harris detector);
• “Dense” DenseFeatureDetector;
• “SimpleBlob” SimpleBlobDetector.

OpenCV application part of moving object tracking and face recognition

What is understood as moving object detection, the OpenCV commonly used method for detecting a moving object with background subtraction, the frame difference method, optical flow method, the tracking algorithm commonly used are meanShift, camShift, particle filter, optical flow and the like.

• meanShift tracking algorithm cv2.meanShift();
• CamShift tracking algorithm cv2.CamShift().

If you learn face recognition, the knowledge points involved are:

• Face detection: find the position of the face from the image and mark it;
• Face recognition: distinguish the person’s name or other information from the located face area;
• Machine learning.