¶ Compile and use the MPP library
RK3588S has powerful codec capabilities, and its codec supports up to 7680x4320@30fps decoding and 7680x4320@30fps encoding. The RK platform uniformly uses the MPP library as the development tool for its codec. Therefore, it can be developed once and ported between multiple platforms.
Since Hot Wheels Technology has open sourced the SDK, it is recommended to build the SDK on the PC side uniformly to ensure a consistent build environment. The cross-compilation environment in the SDK is consistent with the environment on the board.
¶ Obtain MPP source code
There are two ways to obtain the MPP library source code.
¶ Method 1
If you want to keep it consistent with the dynamic library on the board, you need to pull the entire SDK provided by Hot Wheels Technology. The mpp source code is in the root directory of the SDK
external/mpp
. It is recommended to copy this source code to a separate working directory for use
If you do not want to download the entire SDK, you can download the mpp package from the link below. This package is extracted from the complete version of the SDK
¶ Method 2
If you want to experience the latest version of mpp, you can directly pull it from rk's git, create a new working directory, and then execute
git clone https://github.com/rockchip-linux/mpp.git
¶ MPP source code compilation
First, configure the cross-compilation environment. In the SDK root directory , execute the following lines
DIR=$(pwd)
export PATH=$PATH:${DIR}/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin
export CROSS_COMPILE=aarch64-none-linux-gnu-
export ARCH=aarch64
In the mpp working directory, open the build/linux/aarch64 directory
Then modify the compiler in the arm.linux.cross.cmake file as follows
SET(CMAKE_C_COMPILER "aarch64-none-linux-gnu-gcc")
SET(CMAKE_CXX_COMPILER "aarch64-none-linux-gnu-g++")
Then execute in the build/linux/aarch64 directory
./make-Makefiles.bash
make -j8
Then the compilation can be performed. The information of the compilation completion is as follows
After the compilation is completed, you can see the following files in this directory
Then verify whether the compiled mpp environment can run on the board.
There is a compiled program under build/linux/aarch64/test in the mpp working directory. This mpp_info_test can be used to verify whether the mpp environment on the current board is normal. Put this file on the board and execute
./mpp_info_test
View the output: This program changes the printing interface to system log, you need to enter the following command to view the mpp output
tail -10f /var/log/messages
If you can see normal output, it means that the mpp operating environment is fine
¶ Compile MPP DEMO
MPP project comes with cmake, you can compile its own demo by configuring cmake. But in actual application, it is often necessary to integrate mpp function into your own project, and then you need to have the function of linking mpp library. The following will explain how to link mpp library by introducing how to compile a mpp demo separately.
First, copy a copy of mpp demo source code, for example, copy it to the mytest directory of mpp project (This directory needs to be created by yourself, or it can be changed to other directories). The encoding and decoding functions are commonly used here, and their demo source codes are respectively
test/mpi_dec_test.c
test/mpi_enc_test.c
Then cd to the mpp project directory and execute
aarch64-none-linux-gnu-gcc mytest/mpi_enc_test.c build/linux/aarch64/utils/libutils.a build/linux/aarch64/osal/libosal.a -Lbuild/linux/aarch64/mpp -lrockchip_mpp -I./osal/inc -I./osal/linux -I./inc -I./utils -o mytest/mpi_enc_test -lpthread -lstdc++
aarch64-none-linux-gnu-gcc mytest/mpi_dec_test.c build/linux/aarch64/utils/libutils.a build/linux/aarch64/osal/libosal.a -Lbuild/linux/aarch64/mpp -lrockchip_mpp -I./osal/inc -I./osal/linux -I./inc -I./utils -o mytest/mpi_dec_test -lpthread -lstdc++
You can compile mpi_enc_test and mpi_dec_test, which are the demos for encoding and decoding
libutils are some auxiliary functions, such as crc calculation, which will be used in the demo. If the user's own program does not use them, you can not quote them
libosal is some operating system APIs, such as memory allocation, etc., which can also be ignored if the user's own program does not use them
¶ MPP DEMO demonstration
The following uses the mpi_enc_test in the demo for demonstration, which is used for encoding. The so-called encoding is to compress the original RAW image data, such as YUV422 YUV420 NV12 NV16 data formats. The compression methods are usually H264 H265, etc. After compression, the volume is reduced, which is convenient for storage and transmission.
First, you need to capture a section of the original video of the camera. The method of capturing the original video can refer to the previous chapter.
Here, 10 frames of data are captured at a resolution of 1632x1224 and stored in /tmp/isp.out. After capturing, use the ls -ll command to see its size. It can be seen that this data format image takes up a lot of space
Then use the following command to encode
./mpi_enc_test -w 1632 -h 1224 -t 7 -i /tmp/isp.out -o /tmp/isp.h264 -n 10 -f 0
Where -w is the width -h is the height -t specifies the output format, 7 represents H264, -i -o are the input and output file names respectively, -n is the number of frames, -f specifies the input format, and 0 represents YUV420SP, NV12. The specific parameters of this encoding command must match the format of the original image, otherwise an error will be reported.
After encoding, the output file is /tmp/isp.h264, and it can be seen that its occupied space has been greatly reduced.
The output information of mpp can be viewed in syslog information
tail -20f /var/log/messages
Copy the /tmp/isp.h264 and /tmp/isp.out files to the virtual machine ubuntu on the PC, and then use the command
W=1632; H=1224; mplayer isp.out -loop 0 -demuxer rawvideo -fps 30 -rawvideo w=${W}:h=${H}:size=$((${W}*${H}*3/2)):format=NV12
You can play isp.out
mplayer isp.h264 -loop 0
Can play isp.h264
It can be seen that although the space occupied is greatly reduced, the picture quality is almost the same. This is the significance of using the board-side encoder.
Decoding is to restore the image compressed by h264 to RAW format. The command is
./mpi_dec_test -w 1632 -h 1224 -t 7 -i /tmp/isp.h264 -o /tmp/isp.h264.out -n 10 -f 0
In theory, the size of the content after decoding should be the same as that before encoding. Due to compression, some details will be lost and the content is not completely consistent, but the content displayed during playback is not much different
¶ MPP demo source code introduction
The following uses the mpi_enc_test source code in the demo as an example to introduce how to use the mpp dynamic library.
This source file is located in the test directory of the mpp project
First, let's look at the main function. It first uses mpi_enc_test_cmd_update_by_args to parse all the parameters, and then executes the encoding through the enc_test_multi function
int main(int argc, char **argv)
{
RK_S32 ret = MPP_NOK;
MpiEncTestArgs* cmd = mpi_enc_test_cmd_get();
// parse the cmd option
ret = mpi_enc_test_cmd_update_by_args(cmd, argc, argv);
if (ret)
goto DONE;
mpi_enc_test_cmd_show_opt(cmd);
ret = enc_test_multi(cmd, argv[0]);
DONE:
mpi_enc_test_cmd_put(cmd);
return ret;
}
The mpi_enc_test_cmd_update_by_args function parses parameters one by one, checks the parameters, and then completes the incomplete parameters.
MPP_RET mpp_opt_parse(MppOpt opt, int argc, char **argv);
The function is to parse the parameters when executing the mpi_enc_test command and put them into MppOpt opt.
MPP_RET mpi_enc_test_cmd_update_by_args(MpiEncTestArgs* cmd, int argc, char **argv)
{
MppOpt opts = NULL;
RK_S32 ret = -1;
RK_U32 i;
...
/* should change node count when option increases */
mpp_opt_setup(opts, cmd, 67, enc_opt_cnt);
for (i = 0; i < enc_opt_cnt; i++)
mpp_opt_add(opts, &enc_opts[i]);
/* mark option end */
mpp_opt_add(opts, NULL);
ret = mpp_opt_parse(opts, argc, argv);
...
if (!cmd->hor_stride)
cmd->hor_stride = mpi_enc_width_default_stride(cmd->width, cmd->format);
if (!cmd->ver_stride)
cmd->ver_stride = cmd->height;
...
if (cmd->rc_mode == MPP_ENC_RC_MODE_FIXQP) {
if (!cmd->qp_init) {
if (cmd->type == MPP_VIDEO_CodingAVC ||
cmd->type == MPP_VIDEO_CodingHEVC)
cmd->qp_init = 26;
}
}
if (cmd->trace_fps) {
fps_calc_init(&cmd->fps);
mpp_assert(cmd->fps);
fps_calc_set_cb(cmd->fps, show_enc_fps);
}
...
return ret;
}
Then main function calls
int enc_test_multi(MpiEncTestArgs* cmd, const char *name);
This function creates a thread and executes
void *enc_test(void *arg);
This function is to loop the encoding operation
void *enc_test(void *arg)
{
...
mpp_log_q(quiet, "%s start\n", info->name);
...
ret = test_ctx_init(info);
...
ret = mpp_buffer_group_get_internal(&p->buf_grp, MPP_BUFFER_TYPE_DRM);
...
ret = mpp_buffer_get(p->buf_grp, &p->frm_buf, p->frame_size + p->header_size);
...
ret = mpp_buffer_get(p->buf_grp, &p->pkt_buf, p->frame_size);
...
ret = mpp_buffer_get(p->buf_grp, &p->md_info, p->mdinfo_size);
...
// encoder demo
ret = mpp_create(&p->ctx, &p->mpi);
...
mpp_log_q(quiet, "%p encoder test start w %d h %d type %d\n",
p->ctx, p->width, p->height, p->type);
ret = p->mpi->control(p->ctx, MPP_SET_OUTPUT_TIMEOUT, &timeout);
...
ret = mpp_init(p->ctx, MPP_CTX_ENC, p->type);
...
ret = mpp_enc_cfg_init(&p->cfg);
...
ret = test_mpp_enc_cfg_setup(info);
...
t_s = mpp_time();
ret = test_mpp_run(info);
t_e = mpp_time();
ret = p->mpi->reset(p->ctx);
...
return NULL;
}
This function first initializes a buffer, then puts the info, data and other information of the input file into this buffer, then calls
mpp_create
to instantiate an MppCtx
then uses
mpp_init
to initialize it, then uses
mpp_enc_cfg_init
to set it, and finally calls
mpi->encode_put_frame(ctx, frame);
mpi->encode_get_packet(ctx, &packet);
in
test_mpp_run
to send the original data of a frame to mpp, and get the encoded data of a packet from mpp
This operation is repeated for multiple packets to obtain the final encoded file
Specific MPP For the API, please refer to the MPP documentation, which is in docs/Linux/Multimedia/Rockchip_Developer_Guide_MPP_CN.pdf.