¶ OpenCV environment construction and use of camera method introduction
The development board supports one way MIPI-CSI interface, and the default accessory used is the gc8034 camera. The flow of image data is
GC8034 -> MIPI CSI2 -> ISP0
¶ Use of Gstreamer
Currently on the development board, the source of the application layer to get data is ISP0. Currently the RK platform all support V4L2 interface to get data, but the RK platform V4L2 is not the full version, so you can not directly use opencv's v4l2 to pull data. If you use it directly, you will be prompted that there is a V4L2 ioctl command that cannot be supported. However, if you use a USB camera on this platform, it is possible to support direct calls to V4L2 from openCV.
RK has adapted Gstreamer to their debian systems, and the Gstreamer on Debian is modified to add a plugin called gstreamer-rockchip, which supports RK's ISP component.
The default test command is
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, format=NV12, width=1632, height=1224, framerate=30/1 ! rkximagesink
After connecting the development board to the monitor and executing it with the LX terminal of the application, you can see that the camera screen is displayed, which means that the gstreamer function is available
¶ opencv-gstreamer installation
The board itself has opencv, but does not support gstreamer, so it needs to be recompiled and installed with a version that supports gstreamer.
First perform the installation of the relevant packages
apt-get update
apt-get install build-essential cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev python3-dev python3-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev libunwind-dev
apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer-1.0
There is another package, libjasper-dev, which cannot be found in the system's own sources, so you need to add two more sources. You can use vim or other tools to add the following two lines to /etc/apt/sources.list
deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-updates main restricted universe multiverse
deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ xenial main multiverse restricted universe
These two sources above need to add a public key
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 3B4FE6ACC0B21F32
Execute the following command to install libjasper-dev
apt-get update
apt-get install libjasper-dev
After installation, it is recommended to remove the above two sources to avoid affecting the installation of other packages
Next, download opencv, with which version are similar. Here use the 4.5.1 version as an example. opencv source code can be downloaded at https://github.com/opencv/opencv/releases. After downloading, execute the decompression
tar -zxvf opencv-4.5.1.tar.gz
Then create a compilation directory
cd opencv-4.5.1
mkdir build
cd build
Next, execute the build under this build directory. opencv supports cmake.
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D WITH_GSTREAMER=ON \
-D PYTHON_DEFAULT_EXECUTABLE=$(python3 -c "import sys; print(sys.executable)") \
-D PYTHON3_DEFAULT_EXECUTABLE=$(python3 -c "import sys; print(sys.executable)") \
-D PYTHON3_EXECUTABLE=$(python3 -c "import sys; print(sys.executable)") \
-D PYTHON3_NUMPY_INCLUDE_DIRS=$(python3 -c "import numpy; print (numpy.get_include())") \
-D PYTHON3_PACKAGES_PATH=$(python3 -c "import site; print(site.getsitepackages()[0])") \
-DBUILD_DOCS=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_TESTS=OFF \
-DBUILD_PERF_TESTS=OFF \
..
Note that GStreamer must be YES in the output of this step, otherwise some packages are missing
This will give you the makefile in the build directory, which can be compiled and installed
make -j4
make install
This completes the installation
¶ demo
Here a python script is used to implement the opencv gstreamer preview camera screen demo, named opencv_g.py The script is as follows
import cv2 as cv
import os
import time
cap = cv.VideoCapture('v4l2src device=/dev/video0 ! video/x-raw, format=NV12, width=800, height=600, framerate=30/1 ! videoconvert ! appsink', cv.CAP_GSTREAMER)
if not cap.isOpened():
print("Cannot capture from camera. Exiting.")
os._exit()
last_time = time.time()
while(True):
ret, frame = cap.read()
this_time = time.time()
print (str((this_time-last_time)*1000)+'ms')
last_time = this_time;
cv.imshow('frame', frame)
if cv.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv.destroyAllWindows()
Use python3 opencv_g.py in the LX terminal on the board to see the camera screen
