¶ Introduction to the use of RK NPU
RK3568 has an NPU with 0.8T computing power. rk's NPU currently adopts a self-developed architecture and only supports the use of non-open source drivers and libraries to operate.
RK's NPU sdk is divided into two parts, the PC side uses rknn-toolkit2, which can be used for model conversion, inference and performance evaluation on the PC side. Specifically, it converts mainstream models such as Caffe, TensorFlow, TensorFlow Lite, ONNX, DarkNet, PyTorch, etc. to RKNN models and can use this RKNN model for inference simulation, computation time and memory overhead on the PC side. There is another part on the board side, the rknn runtime environment, which contains a set of C API libraries and driver modules to communicate with the NPU, executable programs, etc. This article describes how to use rk's npu sdk.
¶ Related resource download links
- rknn-toolkit2
- rknpu2
- Debian source code with toolchain insidehttp://dd.youyeetoo.cn:5000/sharing/2bkwX9vRn
¶ Install docker
The following command is executed on an x86 Ubuntu host instead of YY3568
Since rknn-toolkit2 runtime environment has more dependencies, it is recommended to install rknn PC environment directly by docker, the docker provided by rk already contains all the necessary environment.
1、Uninstall the old docker version
apt-get remove docker docker-engine docker.io containerd runc
2、Install the dependencies.
sudo apt-get install apt-transport-https ca-certificates curl gnupg2 software-properties-common
3、Trust docker's gpg public key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
To verify that the public key was added successfully, you can use the following command
apt-key fingerprint 0EBFCD88
- Add the software source and install it
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
apt-get update
apt-get install docker-ce
To verify if the installation is successful, you can use the following command
docker -v
¶ Download and run rknn docker
The following command is executed on an x86 Ubuntu host instead of YY3568
First, download rknn-toolkit2. The address is
https://github.com/rockchip-linux/rknn-toolkit2
Note that there is no docker in this place, there is a link to RK's Baidu website on this page, you can go to this link to get the docker
Open this link and find
Just download this file.
Then if you want to update rknn-toolkit, you can also find it inside this netdisk. I'm using version 1.3 here, which is the same as the full version of the sdk.
After downloading, open the directory where the docker is located, and execute
sudo docker load --input rknn-toolkit2-1.3.0-cp36-docker.tar.gz
Then execute
sudo docker images
You can see that this image has been loaded
Then use the command
sudo docker run -t -i --privileged -v /dev/bus/usb:/dev/bus/usb -v $(pwd)/rknn-toolkit2/examples/onnx/yolov5:/rknn_yolov5_demo rknn-toolkit2:1.3.0-cp36 /bin/bash
Where -v is to map the directory into the Docker environment. This is an example in rknn-toolkit2, but other directories can be mapped as well. Also /dev/bus/usb is needed for debugging with adb later. If the board you have has adb service, you can turn it on, if not, you can leave it on
(pwd)/rknn-toolkit2/ in the above command needs to be replaced with the actual directory of the rknn-toolkit2 project, where there is the examples/onnx/yolov5 folder.
After entering docker, check the files under the root directory with the ls command and you can see that there is indeed a folder rknn_yolov5_demo
¶ Run rknn-toolkit2 to generate the model and reason about it
The following command is executed on an x86 Ubuntu host instead of YY3568
Go to rknn_yolov5_demo in docker and execute
python3 ./test.py
You can get the following result
The converted model is then stored in the host's rknn-toolkit2/examples/onnx/yolov5, where
yolov5s.rknn is the model that supports rknn format
If you want to see a picture of the simulation results on the PC, you can add the following to test.py.
This will output the results to result.jpg as follows (the original image is on the left, the inference result is on the right)
¶ Run on development board
First open rknn-toolkit2, you need to regenerate the rknn model file suitable for board-side execution. In the way described before, the generated rknn library is simulated to run on PC, to generate it for the target platform, you need to change the test.py
If there is no adb, then just add the parameter target_platform='rk3568' to rknn.config. If you have adb and need to connect to adb for debugging, you need to add the parameter rknn.init_runtime.
Then execute python3 again in docker . /test.py to get the yolov5s.rknn file, which can be run on the rk3568 device
Then on top of the host, open the rknpu2 folder, and here you still have to choose the yolov5 demo.
cd rknpu2/examples/rknn_yolov5_demo
Modify the application compilation script build-android_RK356X.sh. The script used by the author can be downloaded from the link below, the compilation toolchain is all inside the prebuilts/gcc/linux-x86/aarch64 in the sdk directory. Users need to modify the PATH and TOOL_CHAIN variables according to the actual sdk installation path.
Note here that it depends on what compiler is used for the glibc library on this board, if you are using 6.3.1 use this 6.3.1 compiler path, otherwise use the 9.3.0 version of that one. Method: Copy libc-2.28.so from the board and strings libc-2.28.so | grep gcc, rk sdk usually has two compilers inside. debian system uses 6.3.1 compiler by default.
Execute build-android_RK356X.sh. After that, the following file will be generated
Put this executable, the rknn model, and the sample image together on the board under the same path, you can use ssh, adb, etc.
Then execute
./rknn_yolov5_demo ./model/yolov5s.rknn ./model/bus.jpg
The out.jpg will be generated under this directory, check this file
It works basically the same as the PC-side simulated inference, but the graph doesn't have the confidence level typed on it.
By this point, the whole process of an rknn demo from PC-side inference, to onboard execution is completed.
¶ Common error solutions
If the prompt is missing GLIBC-2.29, this is usually a problem with the cross-compiler configuration, and the solution is:
# Check if the system supports glibc-2.29
strings /lib/aarch64-linux-gnu/libm.so.6 | grep GLIBC_
# Update related packages
sudo apt-get install make gawk bison gcc
wget http://ftp.gnu.org/gnu/glibc/glibc-2.29.tar.gz
tar -zxvf glibc-2.29.tar.gz
cd glibc-2.29
mkdir build
cd build
../configure --prefix=/usr/local --disable-sanity-checks
make -j4
sudo make install
cp /usr/local/lib/libm-2.29.so /lib/aarch64-linux-gnu/
ln -sf libm-2.29.so libm.so.6
The full version of the RKNN API is available for reference rknpu2/doc/Rockchip_RKNPU_User_Guide_RKNN_API_V1.3.0_CN.pdf