¶ OpenCV environment setup and use of the camera method introduced
The development board supports one way MIPI-CSI interface and the default accessory used is the gc8034 camera. The flow of image data is
GC8034 -> MIPI CSI2 -> ISP0
¶ Use of Gstreamer
On the development board, the source of the application layer to get data is ISP0. At present, all the RK platforms support the V4L2 interface to get data, but the V4L2 of the RK platforms is not a complete version, so you can't use the v4l2 of the opencv directly to pull the data. If you use it directly, you will be prompted that there is a V4L2 ioctl command that cannot be supported. But if you use a USB camera on this platform, it is possible to support openCV to call V4L2 directly.
However, there is a compromise: RK has already adapted Gstreamer for their debian systems, and the Debian system on the board has been modified to add a plugin called gstreamer-rockchip, which supports RK's ISP components.
The default test command is
gst-launch-1.0 v4l2src device=/dev/video22 ! video/x-raw, format=NV12, width=1632, height=1224, framerate=30/1 ! autovideosink
Where "device=" followed by the device name needs to be filled in according to the actual, these parameters can refer to the camera tutorials
After connecting the development board to the monitor and using the application LX terminal, you can see the camera screen displayed, indicating that the gstreamer function is available.
¶ opencv-gstreamer installation
The board itself has opencv, but does not support gstreamer, so you need to recompile and install a version that supports gstreamer.
First, install the relevant packages
sudo su
apt-get update
apt-get install gcc g++
apt-get install aptitude
aptitude install build-essential
apt-get install cmake libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev python3-dev python3-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev libunwind-dev
Next download opencv, with which version are almost the same, I use here is 4.5.1. opencv source code download address is https://github.com/opencv/opencv/releases. After downloading the execution of the unzipped
tar -zxvf opencv-4.5.1.tar.gz
Then create a compilation directory
cd opencv-4.5.1
mkdir build
cd build
Next, execute the build in this build directory. opencv supports cmake.
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D WITH_GSTREAMER=ON \
-D PYTHON_DEFAULT_EXECUTABLE=$(python3 -c "import sys; print(sys.executable)") \
-D PYTHON3_DEFAULT_EXECUTABLE=$(python3 -c "import sys; print(sys.executable)") \
-D PYTHON3_EXECUTABLE=$(python3 -c "import sys; print(sys.executable)") \
-D PYTHON3_NUMPY_INCLUDE_DIRS=$(python3 -c "import numpy; print (numpy.get_include())") \
-D PYTHON3_PACKAGES_PATH=$(python3 -c "import site; print(site.getsitepackages()[0])") \
-DBUILD_DOCS=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_TESTS=OFF \
-DBUILD_PERF_TESTS=OFF \
..
Note that GStreamer must be YES in the output of this step, otherwise some packages are missing!
This will give you the makefile in the build directory, so you can compile and install it.
make -j4
make install
¶ demo
Here we use a python script to implement a demo of the opencv gstreamer previewing the camera feed, named opencv_g.py The script looks like this
import cv2 as cv
import os
import time
cap = cv.VideoCapture('v4l2src device=/dev/video22 ! video/x-raw, format=NV12, width=1632, height=1224, framerate=30/1 ! videoconvert ! appsink', cv.CAP_GSTREAMER)
if not cap.isOpened():
print("Cannot capture from camera. Exiting.")
os._exit()
last_time = time.time()
while(True):
ret, frame = cap.read()
this_time = time.time()
print (str((this_time-last_time)*1000)+'ms')
last_time = this_time;
cv.imshow('frame', frame)
if cv.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv.destroyAllWindows()
Use python3 opencv_g.py in the LX terminal on the board to see the camera screen!
