¶ Introduction to OpenCV environment construction and camera usage
The development board supports one MIPI-CSI interface, and the default accessory is the gc8034 camera. The flow direction of image data is
GC8034 -> MIPI CSI2 -> ISP0
¶ Use of Gstreamer
Currently on the development board, the source of data obtained by the application layer is ISP0. Currently, all RK platforms support V4L2 interface to obtain data, but the V4L2 of the RK platform is not a complete version, so you cannot directly use opencv's v4l2 to pull data. If you use it directly, it will prompt that there is a V4L2 ioctl command that cannot be supported. But if you use a USB camera on this platform, it can support openCV to call V4L2 directly.
However, there is a compromise here. RK has adapted Gstreamer in their Debian and other systems. The Gstreamer in the Debian system on the board has been modified, and a plug-in called gstreamer-rockchip has been added to support RK's ISP component.
The default test command is
gst-launch-1.0 v4l2src device=/dev/video22 ! video/x-raw, format=NV12, width=1632, height=1224, framerate=30/1 ! autovideosink
The device name after "device=" needs to be filled in according to the actual situation. For these parameters, please refer to the camera tutorial
The development board is connected to the monitor. After using the application LX terminal to execute, you can see that the camera screen is displayed, indicating that the gstreamer function is available
¶ opencv-gstreamer installation
The board itself has opencv, but does not support gstreamer. Therefore, you need to recompile and install a version that supports gstreamer.
First, install the relevant software packages
sudo su
apt-get update
apt-get install gcc g++
apt-get install aptitude
aptitude install build-essential
apt-get install cmake libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev python3-dev python3-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev libunwind-dev
Next, download opencv. It doesn't matter which version you use. I use 4.5.1 here. The opencv source code download address is https://github.com/opencv/opencv/releases. After downloading, unzip it
tar -zxvf opencv-4.5.1.tar.gz
Then create a compilation directory
cd opencv-4.5.1
mkdir build
cd build
Next, build in this build directory. Opencv supports cmake.
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D WITH_GSTREAMER=ON \
-D PYTHON_DEFAULT_EXECUTABLE=$(python3 -c "import sys; print(sys.executable)") \
-D PYTHON3_DEFAULT_EXECUTABLE=$(python3 -c "import sys; print(sys.executable)") \
-D PYTHON3_EXECUTABLE=$(python3 -c "import sys; print(sys.executable)") \
-D PYTHON3_NUMPY_INCLUDE_DIRS=$(python3 -c "import numpy; print (numpy.get_include())") \
-D PYTHON3_PACKAGES_PATH=$(python3 -c "import site; print(site.getsitepackages()[0])") \
-DBUILD_DOCS=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_TESTS=OFF \
-DBUILD_PERF_TESTS=OFF \
..
Note that GStreamer must be YES in the output of this step, otherwise some packages are missing
In this way, you can get the makefile in the build directory and execute the compilation and installation
make -j4
make install
This completes the installation
¶ Test demo
Here, a python script is used to implement the demo of opencv gstreamer previewing the camera screen, named opencv_g.py. The script is as follows
import cv2 as cv
import os
import time
cap = cv.VideoCapture('v4l2src device=/dev/video22 ! video/x-raw, format=NV12, width=1632, height=1224, framerate=30/1 ! videoconvert ! appsink', cv.CAP_GSTREAMER)
if not cap.isOpened():
print("Cannot capture from camera. Exiting.")
os._exit()
last_time = time.time()
while(True):
ret, frame = cap.read()
this_time = time.time()
print (str((this_time-last_time)*1000)+'ms')
last_time = this_time;
cv.imshow('frame', frame)
if cv.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv.destroyAllWindows()
Use python3 opencv_g.py in the LX terminal on the board to see the camera image
python3 opencv_g.py