The OpenCV function is based on the GitHub project flask-video-streaming, we changed the camera_opencv.py to perform OpenCV related operations.
37.1 Single Thread Processing of Video Frames
First, we introduce the process of single-thread processing of video frames. Let ’s start with a simple one, so that you will understand why OpenCV uses multiple threads to process video frames. The process of single-thread processing of video frames is as follows:
Process explanation: First obtain a frame from the camera, and then use OpenCV to analyze the content of this frame. After the analysis is completed, the information to be drawn is generated, such as the position of the center point of the target object, the text lamp information that needs to be generated on the screen, then Draw those elements on the screen according to the generated drawing information, and finally display the processed and drawn frame on the page.
Such a processing flow will cause each frame to be collected to wait for the OpenCV related process to be processed. After this frame is displayed, the second frame can be collected for processing and analysis to make it abnormally stuck.
37.2 Multi-thread Processing of Video Frames
Because single thread has problems in processing video frames, we chose to use multiple threads to process video frames. Next, we will introduce the process of multi-thread processing video frames:
Process explanation: In order to improve the frame rate, we separate the analysis task of the video frame from the process of acquisition and display, and place it in a background thread to execute and generate drawing information.
37.3 Multi-threaded code program for processing video frames
We change the code of multi-threading camera_opencv.py (this code only shows the principle of multi-threading and removes the function of OpenCV).Camera_opencv.py is stored in the adeept_alter/server directory.You need to find it in this directory. Let's learn the code program of camera_opencv.py together.
Import dependent libraries.
This class is used to handle the task of OpenCV analyzing video frames in the background.
In class CVThread, the mode (self, imgInput) method is used to pass in the video frame that needs to be processed.
This method is used to draw elements in the picture.
In this method, you can add content to be processed by OpenCV.
This method can block the thread and wait for the next frame to be processed.
Resume thread.
Thread processing video frames in the background.
Instantiate CVThread().
If OpenCV is processing video frames, skip.
If OpenCV is not processing the video frame, the thread that processes the video frame will process the new video frame and resume the processing thread at the same time.
Draw elements on the screen.
