¶ 03 SVP Video Processing
SVP (Smart Video Processing) is the core module for intelligent video processing. It gives developers a complete API set covering person, face, vehicle, fire/smoke, and DMS (driver monitoring) analytics.
This document uses YOLOv5 person detection as the example and walks through the whole intelligent vision pipeline, from model loading to inference output.
¶ 1 Architecture Overview
The GK7206 integrates a dedicated NPU (neural processing unit) that runs convolutional network inference efficiently. The data flow is:
Key hardware modules:
| Module | Full name | Function |
|---|---|---|
| VI | Video Input | Receives raw images captured by the sensor |
| VPSS | Video Processing Sub-System | Scales and crops images, outputting several resolutions |
| NPU | Neural Processing Unit | Neural network inference (convolution, pooling, activation, etc.) |
| VGS | Video Graphics Sub-System | Overlays rectangles, text, and other OSD elements on the image |
| VENC | Video Encoder | H.264 / H.265 video encoding |
| MMZ | Media Memory Zone | Physically contiguous memory management for media |
¶ 2 SVP Video Development Pipeline
Before the camera image reaches the NPU it passes through a chain of hardware modules:
Sensor -> VI -> VPSS -> channel0: full frame (1920x1080) -> VENC (main stream) / VO (display)
-> channel1: small frame (640x360) -> NPU inference
-> channel2: medium frame (1280x720) -> VENC (sub stream) / NPU (full-frame model)
Why scaling is needed
- NPU models are trained at a fixed resolution (for example 640x360)
- VPSS hardware performs the scaling automatically, with no CPU involvement
- The scaled frame is handed to the NPU directly in physical memory, which is zero-copy
Where the code lives: sample_svp_main.c:1108-1161
¶ 3 The Complete SVP Pipeline
¶ 3.1 Initializing the SVP Subsystem
SVP (Smart Vision Platform) is the GK7206 intelligent vision platform interface. It must be initialized before the NPU is used:
// file: sample_svp_main.c -> sample_svp_start()
xmedia_s32 ret;
// step 1: initialize the SVP subsystem
ret = xmedia_svp_init();
if (ret != XMEDIA_SUCCESS) {
printf("xmedia_svp_init error.\n");
return XMEDIA_FAILURE;
}
Important:
xmedia_svp_init()only needs to be called once for the whole application lifetime.
¶ 3.2 Loading the Model and Allocating Memory
The NPU needs dedicated physically contiguous memory for model weights and intermediate results.
¶ 3.2.1 Configuring Model Information
For person detection, the model parameters are configured as follows:
xmedia_svp_modules modules[8]; // up to 8 models
xmedia_svp_task_cfg task_cfg;
// configure model 0: person detection
modules[0].alg_type = XMEDIA_SVP_ALG_TYPE_PERSON; // algorithm type
modules[0].load_mode = XMEDIA_SVP_MODEL_FILE; // load from file
modules[0].format = XMEDIA_SVP_INPUTDATA_FORMAT_RGB888; // input format
modules[0].pathname = "./model/gnn_person_detect_640x360_rgb888hwc_v0103_20251203.bin";
task_cfg.module_num = 1; // this task uses 1 model
task_cfg.task_type = XMEDIA_SVP_TASK_DETECT; // detection task
task_cfg.modules = modules;
Model configuration fields:
| Field | Description | Valid values |
|---|---|---|
alg_type |
Algorithm category | XMEDIA_SVP_ALG_TYPE_PERSON, XMEDIA_SVP_ALG_TYPE_FACE, XMEDIA_SVP_ALG_TYPE_CAR, etc. |
load_mode |
Load method | XMEDIA_SVP_MODEL_FILE (from file), XMEDIA_SVP_MODEL_MEM (from memory) |
format |
Input image format | XMEDIA_SVP_INPUTDATA_FORMAT_RGB888, XMEDIA_SVP_INPUTDATA_FORMAT_YUV420SP |
pathname |
Model file path | NPU-specific model file in .bin format |
¶ 3.2.2 Querying Model Memory Requirements
// query the memory the model needs
xmedia_cl_mem_info model_mem_info;
ret = xmedia_cl_graph_query_model_info_from_file(
modules[0].pathname,
&model_mem_info,
XMEDIA_CL_MEM_INFO
);
// model_mem_info contains:
// worksize - NPU working buffer size (intermediate results)
// inputsize - input buffer size (holds the input image)
// outputsize- output buffer size (raw inference results)
¶ 3.2.3 Allocating NPU Memory
xmedia_svp_cfg svp_cfg;
xmedia_svp_get_config(&svp_cfg);
svp_cfg.reuse_type = XMEDIA_SVP_MEM_TYPE_BLOCK; // block reuse mode
// allocate the working buffer (used internally by the NPU; needs cache attributes)
sample_mmz_alloc_and_map_cache(
XMEDIA_NULL, "npu_work_mem",
&svp_cfg.workbuf_reuse_mem.phyaddr, // physical address
&svp_cfg.workbuf_reuse_mem.viraddr, // virtual address
svp_cfg.workbuf_reuse_mem.size // size
);
// allocate the input buffer (holds the input image data)
sample_mmz_alloc_and_map(
XMEDIA_NULL, "npu_input_mem",
&svp_cfg.input_reuse_mem.phyaddr,
&svp_cfg.input_reuse_mem.viraddr,
svp_cfg.input_reuse_mem.size
);
// allocate the output buffer (holds the raw inference results)
sample_mmz_alloc_and_map_cache(
XMEDIA_NULL, "npu_output_mem",
&svp_cfg.output_reuse_mem.phyaddr,
&svp_cfg.output_reuse_mem.viraddr,
svp_cfg.output_reuse_mem.size
);
// hand the memory configuration to SVP
xmedia_svp_set_config(&svp_cfg);
¶ 3.3 Creating an Inference Task
With the model configured and memory ready, create the SVP task (this is when the model is actually loaded into the NPU):
xmedia_s32 svp_handle;
ret = xmedia_svp_task_create(&svp_handle, task_cfg);
if (ret != XMEDIA_SUCCESS) {
printf("xmedia_svp_task_create failed!\n");
return XMEDIA_FAILURE;
}
// svp_handle is the handle for all later inference calls
Key concept: one
svp_handlerepresents one complete inference pipeline, including the loaded model, pre/post-processing, and all related resources.
¶ 3.4 Setting Inference Parameters
After creating the task you can adjust inference parameters (thresholds, tracking, and so on):
// YOLOv5 detection attributes
xmedia_svp_yolov5_attr yolov5_attr;
yolov5_attr.detect_threshold = 0.65f; // detection confidence threshold (lower scores are dropped)
yolov5_attr.classifier_threshold = 0.8f; // classifier confidence threshold
yolov5_attr.iou_threshold = 0.5f; // NMS IoU threshold (filters overlapping boxes)
yolov5_attr.max_target_num = 10; // max targets per frame (50 at most)
yolov5_attr.bytetrack_enable = XMEDIA_TRUE; // enable ByteTrack object tracking
yolov5_attr.motionless_filter_enable = XMEDIA_TRUE; // enable motion-state detection
yolov5_attr.stillness_thres = 0.9f; // stillness detection sensitivity
yolov5_attr.movement_fps_thres = 5; // consecutive-frame threshold
yolov5_attr.smart_venc_enable = XMEDIA_FALSE; // smart encoding
yolov5_attr.smart_ae_enable = XMEDIA_FALSE; // smart exposure
ret = xmedia_svp_task_set_attr(svp_handle, &yolov5_attr);
Tuning suggestions:
| Parameter | Suggested value | Effect of raising | Effect of lowering |
|---|---|---|---|
detect_threshold |
0.65 | Fewer false positives, more misses | Fewer misses, more false positives |
iou_threshold |
0.5 | Keeps more overlapping boxes | Merges more overlapping boxes |
max_target_num |
10 | Handles more targets, slightly slower | Caps target count, more stable |
bytetrack_enable |
TRUE | Cross-frame tracking with tracker_id | Per-frame detection only, no tracking |
¶ 3.5 Starting the Inference Thread
Once initialization is done, start a dedicated thread to run the inference loop:
// sample_svp_info holds all runtime state
sample_svp_info svp_info;
svp_info.detect_type = SAMPLE_SVP_ALG_TYPE_PERSON;
svp_info.svp_handle = svp_handle;
svp_info.vpss_pipe = vpss_pipe;
svp_info.vpss_ochn[1] = small_channel; // small-frame channel
svp_info.venc_chn[1] = venc_channel; // encoder channel
svp_info.big_stream = XMEDIA_FALSE;
g_svp_start_flag = XMEDIA_TRUE;
// create the inference thread
pthread_create(&g_svp_thread, NULL, sample_svp_proc, &svp_info);
¶ 4 Frame Acquisition and the Inference Loop
sample_svp_proc() is the main inference loop thread. It keeps taking frames from VPSS, submitting them to the NPU, and handling the results.
¶ 4.1 Acquiring a Camera Frame
// sample_svp_main.c → sample_svp_proc()
while (g_svp_start_flag == XMEDIA_TRUE) {
xmedia_video_frame_info video_frame;
xmedia_s32 milli_sec = 20000; // 20 second timeout
// acquire one small frame from VPSS (640x360)
ret = xmedia_vpss_acquire_ochn_frame(
svp_info->vpss_pipe, // VPSS pipe
svp_info->vpss_ochn[1], // output channel 1 (small frame)
&video_frame, // output frame info
milli_sec // timeout
);
if (ret != XMEDIA_SUCCESS) {
printf("get vpss small frame failed!\n");
continue; // frame acquisition failed, retry
}
Key information in video_frame:
video_frame (xmedia_video_frame_info)
├── frame.width = 640
├── frame.height = 360
+-- frame.addr.y_phy_addr -> physical address of the Y plane
+-- frame.addr.uv_phy_addr -> physical address of the UV plane
├── frame.pixel_format = YUV420SP
+-- pool_id -> VB pool ID
Important: the frame data lives in physical memory and needs no copy. The NPU accesses it through its physical address.
¶ 4.2 Submitting to the NPU
Package the frame into the NPU input format and submit it for inference:
// pack the input
xmedia_svp_task_input task_input;
xmedia_video_frame_info frame_info[2];
frame_info[0] = video_frame; // frame 0 = small frame
task_input.frame_num = 1; // single-frame input
task_input.frame = frame_info;
// if the full frame is needed (for example plate recognition needs 1920x1080)
if (svp_info->big_stream == XMEDIA_TRUE) {
xmedia_video_frame_info video_frame_big;
ret = xmedia_vpss_acquire_ochn_frame(
svp_info->vpss_pipe, svp_info->vpss_ochn[2],
&video_frame_big, milli_sec
);
frame_info[1] = video_frame_big; // frame 1 = full frame
task_input.frame_num = 2; // two-frame input
}
// *** core: submit inference (synchronous, blocking) ***
xmedia_svp_yolov5_output result = {0};
ret = xmedia_svp_task_process(
svp_info->svp_handle, // SVP task handle
&task_input, // input frames
&result // output detection results
);
if (ret != XMEDIA_SUCCESS) {
printf("xmedia_svp_task_process failed!\n");
}
¶ 4.3 Parsing the Detection Results
When inference returns, result already holds structured detection results:
if (result.target_num > 0) {
xmedia_video_rect target_rect[XMEDIA_SVP_MAX_TARGET_NUM];
for (xmedia_s32 i = 0; i < result.target_num; i++) {
// target bounding box (floating point, in 640x360 coordinates)
xmedia_float x1 = result.targets[i].rect.x1;
xmedia_float y1 = result.targets[i].rect.y1;
xmedia_float x2 = result.targets[i].rect.x2;
xmedia_float y2 = result.targets[i].rect.y2;
// aligned to even pixels (hardware requirement)
xmedia_s32 px1 = (xmedia_s32)roundf(x1 / 2) * 2;
xmedia_s32 py1 = (xmedia_s32)roundf(y1 / 2) * 2;
xmedia_s32 px2 = (xmedia_s32)roundf(x2 / 2) * 2;
xmedia_s32 py2 = (xmedia_s32)roundf(y2 / 2) * 2;
// target class
xmedia_svp_class_type cls = result.targets[i].class_type;
// for example: XMEDIA_SVP_CLASS_TYPE_PERSON
// confidence (0.0 ~ 1.0)
xmedia_float score = result.targets[i].detect_score;
// tracking ID (valid when ByteTrack is enabled)
xmedia_s32 tracker_id = result.targets[i].tracker_id;
// motion state
xmedia_svp_motion_state motion = result.targets[i].motion_state;
// XMEDIA_SVP_MOTION_STATE_STATIC = still
// XMEDIA_SVP_MOTION_STATE_MOVING = moving
// build the rectangle for drawing
target_rect[i].x = px1;
target_rect[i].y = py1;
target_rect[i].width = ABS(px2 - px1);
target_rect[i].height = ABS(py2 - py1);
}
¶ 4.4 Drawing Boxes and Sending to the Encoder
// step 1: draw detection boxes on the frame with VGS hardware (red rectangles)
sample_svp_draw(&target_rect[0], &video_frame, result);
// step 2 (optional): overlay OSD text
// requires defining USE_OSD at build time
#ifdef USE_OSD
xmedia_char info_text[64];
snprintf(info_text, sizeof(info_text),
"id[%d]scr[%.2f]cls[%.2f]mv[%d]",
result.targets[0].tracker_id,
result.targets[0].detect_score,
result.targets[0].classfier_score,
result.targets[0].motion_state);
sample_target_osd(&video_frame, px1, py1, info_text);
#endif
} // end if (result.target_num > 0)
// step 3: send the annotated frame to the encoder
ret = xmedia_venc_send_frame(
svp_info->venc_chn[1],
&video_frame,
milli_sec
);
// step 4: release the frame buffer (back to VPSS)
ret = xmedia_vpss_release_ochn_frame(
svp_info->vpss_pipe,
svp_info->vpss_ochn[1],
&video_frame
);
} // end while -> back to the top of the loop for the next frame
¶ 5 Key Data Structures
¶ 5.1 Model Configuration
// single-model configuration (xmedia_svp.h:173-180)
typedef struct {
xmedia_svp_model_type load_mode; // load method (file/memory)
xmedia_svp_inputdata_format format; // input format (RGB888/YUV420SP)
xmedia_char *pathname; // model file path
xmedia_u8 *buf; // model data for memory mode
xmedia_u32 len; // model length for memory mode
xmedia_svp_alg_type alg_type; // algorithm type
xmedia_void *priv; // private data
} xmedia_svp_modules;
// task configuration (xmedia_svp.h:182-187)
typedef struct {
xmedia_svp_task_type task_type; // task type
xmedia_svp_modules *modules; // model array
xmedia_u8 module_num; // number of models
xmedia_void *priv; // private data
} xmedia_svp_task_cfg;
¶ 5.2 Inference Input
// task input (xmedia_svp.h:167-170)
typedef struct {
xmedia_video_frame_info *frame; // frame array
xmedia_u8 frame_num; // frame count (1=single, 2=dual)
} xmedia_svp_task_input;
¶ 5.3 Detection Results
// YOLOv5 detection output (xmedia_svp.h:239-242)
typedef struct {
xmedia_u32 target_num; // number of detected targets
xmedia_svp_detect_result targets[XMEDIA_SVP_MAX_TARGET_NUM]; // target array (50 max)
} xmedia_svp_yolov5_output;
// single detection result (xmedia_svp.h:116-127)
typedef struct {
xmedia_svp_alg_type alg_type; // algorithm type
xmedia_svp_class_type class_type; // target class
xmedia_float detect_score; // detection confidence (0.0~1.0)
xmedia_float classfier_score; // classification confidence (0.0~1.0)
xmedia_s32 tracker_id; // tracking ID
xmedia_u32 tracker_age; // tracker age (frames alive)
xmedia_svp_rect rect; // bounding box (x1,y1,x2,y2)
xmedia_bool special_target; // whether this is a special target
xmedia_float distance; // target distance
xmedia_svp_motion_state motion_state; // motion state
} xmedia_svp_detect_result;
// bounding box coordinates (xmedia_svp.h:102-108)
typedef struct {
xmedia_float x1; // top-left X
xmedia_float y1; // top-left Y
xmedia_float x2; // bottom-right X
xmedia_float y2; // bottom-right Y
} xmedia_svp_rect;
¶ 5.4 Inference Parameters
// YOLOv5 detection parameters (xmedia_svp.h:225-237)
typedef struct {
xmedia_float detect_threshold; // confidence threshold, 0.65 recommended
xmedia_float classifier_threshold; // classifier threshold, 0.8 recommended
xmedia_float iou_threshold; // NMS IOU threshold, 0.5 recommended
xmedia_u32 max_target_num; // max targets, 50 at most
xmedia_bool bytetrack_enable; // object tracking switch
xmedia_bool motionless_filter_enable; // motion-state detection switch
xmedia_float stillness_thres; // stillness sensitivity, 0.9 recommended
xmedia_u8 movement_fps_thres; // consecutive-frame threshold, 5 recommended
xmedia_bool smart_venc_enable; // smart encoding switch
xmedia_bool smart_ae_enable; // smart exposure switch
} xmedia_svp_yolov5_attr;
¶ 6 Supported Algorithm Types
¶ 6.1 Single-Model Detection
| Algorithm | Enum value | Model file | Task type |
|---|---|---|---|
| Person detection | XMEDIA_SVP_ALG_TYPE_PERSON |
gnn_person_detect_640x360_rgb888hwc.bin |
XMEDIA_SVP_TASK_DETECT |
| Face detection | XMEDIA_SVP_ALG_TYPE_FACE |
gnn_face_detect_640x360_rgb888hwc.bin |
XMEDIA_SVP_TASK_DETECT |
| Vehicle detection | XMEDIA_SVP_ALG_TYPE_CAR |
gnn_car_detect_640x360_rgb888hwc.bin |
XMEDIA_SVP_TASK_DETECT |
| Pet detection | XMEDIA_SVP_ALG_TYPE_PET |
gnn_pet_detect_640x360_rgb888hwc.bin |
XMEDIA_SVP_TASK_DETECT |
| Head detection | XMEDIA_SVP_ALG_TYPE_HEAD |
gnn_head_detect_640x360_rgb888hwc.bin |
XMEDIA_SVP_TASK_DETECT |
| Non-motorized vehicle | XMEDIA_SVP_ALG_TYPE_NON_MOTORIZED_VEHICLE |
gnn_nocar_detect_640x360_rgb888hwc.bin |
XMEDIA_SVP_TASK_DETECT |
| Fire and smoke | XMEDIA_SVP_ALG_TYPE_FIREWORKS |
gnn_fireworks_detect_640x360_rgb888hwc.bin |
XMEDIA_SVP_TASK_DETECT |
| Package detection | XMEDIA_SVP_ALG_TYPE_PACKAGE |
gnn_package_detect_640x360_rgb888hwc.bin |
XMEDIA_SVP_TASK_DETECT |
¶ 6.2 Multi-Model Cascades
| Algorithm | Task type | Models | Description |
|---|---|---|---|
| Human keypoints | XMEDIA_SVP_TASK_DETECT_AND_KEYPOINT |
1 | Person detection + keypoints |
| Gesture recognition | XMEDIA_SVP_TASK_GESTURE |
2 | Hand detection + gesture classification |
| Facial expression | XMEDIA_SVP_TASK_EMOTION_CLASSIFITION |
3 | Face detection + keypoints + expression classification |
| Face recognition | XMEDIA_SVP_TASK_FACE_RECOGNITON |
3 | Face detection + keypoints + feature extraction |
| Two-stage detection | XMEDIA_SVP_TASK_RCNN |
2 | Coarse detection + refinement |
| ADAS | XMEDIA_SVP_TASK_ADAS |
5 | Vehicle + person + non-motorized vehicle + plate + lane |
| DMS | XMEDIA_SVP_TASK_DMS |
several | Face + fatigue + phone + smoking |
| Plate recognition | XMEDIA_SVP_TASK_PLATE |
several | Plate detection + character recognition |
| Vehicle recognition | XMEDIA_SVP_TASK_VEHICLE |
several | Vehicle detection + color/type recognition |
¶ 6.3 Output Classes
// target class enum (xmedia_svp.h:62-89)
typedef enum {
XMEDIA_SVP_CLASS_TYPE_PERSON, // person
XMEDIA_SVP_CLASS_TYPE_FACE, // face
XMEDIA_SVP_CLASS_TYPE_CAR, // vehicle
XMEDIA_SVP_CLASS_TYPE_PET, // pet
XMEDIA_SVP_CLASS_TYPE_HEAD, // head
XMEDIA_SVP_CLASS_TYPE_ELECTRIC_BICYCLE, // electric bicycle
XMEDIA_SVP_CLASS_TYPE_MASK, // face mask
XMEDIA_SVP_CLASS_TYPE_BIKE, // bicycle
XMEDIA_SVP_CLASS_TYPE_BIKER, // cyclist
XMEDIA_SVP_CLASS_TYPE_MOTOR, // motorcycle
XMEDIA_SVP_CLASS_TYPE_MOTORER, // motorcyclist
XMEDIA_SVP_CLASS_TYPE_TRICYCLE, // tricycle
XMEDIA_SVP_CLASS_TYPE_TRICYCLER, // tricycle rider
XMEDIA_SVP_CLASS_TYPE_FIREWORKS_FIRE, // flame
XMEDIA_SVP_CLASS_TYPE_FIREWORKS_SMOKE, // smoke
XMEDIA_SVP_CLASS_TYPE_PACKAGE, // package
} xmedia_svp_class_type;
¶ 7 Model Files
¶ 7.1 Model File Format
The NPU uses .bin model files with this naming convention:
gnn_<algorithm>_<input-resolution>_<input-format>_<version>_<date>.bin
Example:
gnn_person_detect_640x360_rgb888hwc_v0103_20251203.bin
│ │ │ │ │ │
| | | | | +- date: 2025-12-03
| | | | +- version: v01.03
| | | +- input format: RGB888 HWC layout
| | +- input resolution: 640x360
| +- function: person detection
+- prefix: GNN (generic neural network)
¶ 7.2 Model Conversion Flow
Important:
.binfiles are quantized and compiled for the NPU. You cannot run inference directly with a raw.ptor.onnxfile.
¶ 8 Multi-Model Cascade Example
Face recognition is used here to show how cascading works:
// face recognition needs a 3-model cascade
xmedia_svp_modules modules[3];
// model 1: face detection (locate faces)
modules[0].alg_type = XMEDIA_SVP_ALG_TYPE_FACE;
modules[0].load_mode = XMEDIA_SVP_MODEL_FILE;
modules[0].format = XMEDIA_SVP_INPUTDATA_FORMAT_RGB888;
modules[0].pathname = "./model/gnn_face_detect_640x360_rgb888hwc_v0103_20251209.bin";
// model 2: face landmarks (locate 5 facial points)
modules[1].load_mode = XMEDIA_SVP_MODEL_FILE;
modules[1].format = XMEDIA_SVP_INPUTDATA_FORMAT_RGB888;
modules[1].pathname = "./model/gnn_face_keypoint_48x48_rgb888hwc_v0101_20250818.bin";
// model 3: face feature extraction (512-d vector)
modules[2].load_mode = XMEDIA_SVP_MODEL_FILE;
modules[2].format = XMEDIA_SVP_INPUTDATA_FORMAT_RGB888;
modules[2].pathname = "./model/gnn_face_recognition_112x112_rgb888hwc_v0101_20250818.bin";
task_cfg.module_num = 3;
task_cfg.task_type = XMEDIA_SVP_TASK_FACE_RECOGNITON;
task_cfg.modules = modules;
// create the task
xmedia_svp_task_create(&handle, task_cfg);
The inference call is the same; the SDK handles the cascade internally:
// the inference call is identical to the single-model case
xmedia_svp_fr_output fr_output = {0};
ret = xmedia_svp_task_process(handle, &task_input, &fr_output);
// fr_output.face_num -> number of detected faces
// fr_output.fr_result[i] -> 512-d feature vector + box per face
¶ 9 Memory Management Strategy
¶ 9.1 Memory Reuse Modes
The SDK supports three memory reuse modes to save memory:
typedef enum {
XMEDIA_SVP_MEM_TYPE_BLOCK, // block reuse (recommended)
XMEDIA_SVP_MEM_TYPE_AINR_SHARE, // shared with AINR
XMEDIA_SVP_MEM_TYPE_COMPLETE, // fully independent
} xmedia_svp_mem_reuse_type;
¶ 9.2 Lifecycle Management
Application start
│
+-- xmedia_svp_init() // initialize SVP
+-- xmedia_cl_graph_query_model_info() // query memory requirements
+-- MMZ alloc (work/input/output) // allocate physically contiguous memory
+-- xmedia_svp_set_config() // configure memory
+-- xmedia_svp_task_create() // create task (load model)
│
| +-- while loop ---------------- +
| | acquire_frame -> task_process | // inference loop
│ │ → draw → venc → release_frame │
│ └────────────────────────────────┘
│
+-- xmedia_svp_task_destroy() // destroy task (unload model)
+-- MMZ free (work/input/output) // free memory
+-- xmedia_svp_uninit() // de-initialize SVP
¶ 9.3 MMZ Memory Operations
// allocate physically contiguous memory
xmedia_u64 phy_addr = xmedia_mmz_alloc("mmz_name", "buf_name", size);
// map into the user-space virtual address space
void *virt_addr = xmedia_mmz_map(phy_addr, size, cache_enabled);
// access the memory (read/write)
memcpy(virt_addr, src, size);
// unmap
xmedia_mmz_unmap(virt_addr);
// free the physical memory
xmedia_mmz_free(phy_addr);
¶ 10 Troubleshooting
¶ 10.1 Inference Failures
| Symptom | Likely cause | Fix |
|---|---|---|
xmedia_svp_task_process returns non-zero |
Model file damaged or missing | Check the .bin path and permissions |
| Intermittent inference failures | Frame acquisition timed out | Increase the milli_sec timeout |
| Memory allocation fails | Not enough MMZ space | Check the mmz configuration and shrink buffers |
| Zero targets detected | Confidence threshold too high | Lower detect_threshold |
¶ 10.2 Performance Tuning
// enable timing statistics (define SAMPLE_TIME_DEBUG at build time)
#define SAMPLE_TIME_DEBUG 1
// print inference time
TIME_COST_START();
xmedia_svp_task_process(handle, &input, &result);
TIME_COST_END();
TIME_COST_PRINT("svp process all");
// sample output: svp process all cost time: 25000 us
¶ 10.3 Offline Testing with a YUV File
When no camera is available, a YUV file can be used as input for offline testing:
// define READ_YUV at build time
#define READ_YUV
// the inference thread reads .yuv files from ./yuv_dir/ automatically
// the file resolution must match the model input (for example 640x360 YUV420SP)
¶ 10.4 OSD Text Overlay
Define the USE_OSD macro at build time and link the canvas and canvas_font libraries:
#ifdef USE_OSD
xmedia_char text[64];
snprintf(text, sizeof(text), "id[%d] score[%.2f]", id, score);
sample_target_osd(&video_frame, x, y, text);
#endif
¶ 11 API Quick Reference
¶ 11.1 SVP Lifecycle APIs
| API | Function | When to call |
|---|---|---|
xmedia_svp_init() |
Initialize the SVP subsystem | Once at application start |
xmedia_svp_uninit() |
De-initialize SVP | At application exit |
xmedia_svp_set_config() |
Set SVP memory configuration | Before creating a task |
xmedia_svp_get_config() |
Get SVP memory configuration | Before allocating memory |
xmedia_svp_task_create() |
Create an inference task | When loading the model |
xmedia_svp_task_destroy() |
Destroy an inference task | When unloading the model |
xmedia_svp_task_set_attr() |
Set task attributes | Any time after creation |
xmedia_svp_task_get_attr() |
Get task attributes | Any time |
xmedia_svp_task_process() |
Run inference | In a loop |
xmedia_svp_get_version() |
Get the SVP version | Any time |
¶ 11.2 Frame Management APIs
| API | Function |
|---|---|
xmedia_vpss_acquire_ochn_frame() |
Acquire a frame from VPSS (blocking) |
xmedia_vpss_release_ochn_frame() |
Release a frame back to the pool |
xmedia_venc_send_frame() |
Send a frame to the encoder |
¶ 11.3 Drawing APIs
| API | Function |
|---|---|
xmedia_vgs_init() |
Initialize VGS |
xmedia_vgs_create_job() |
Create a VGS job |
xmedia_vgs_add_task_cover() |
Add a rectangle-drawing task |
xmedia_vgs_add_task_line() |
Add a line-drawing task |
xmedia_vgs_add_task_osd() |
Add an OSD overlay task |
xmedia_vgs_add_task_scale() |
Add a scaling task |
xmedia_vgs_submit_job() |
Submit the VGS job |
xmedia_vgs_wait_job() |
Wait for the VGS job to finish |
xmedia_vgs_cancel_job() |
Cancel the VGS job |
¶ 11.4 Memory Management APIs
| API | Function |
|---|---|
xmedia_mmz_alloc() |
Allocate physically contiguous memory |
xmedia_mmz_map() |
Map to a virtual address |
xmedia_mmz_unmap() |
Unmap the virtual address |
xmedia_mmz_free() |
Free the physical memory |
¶ Appendix: Key Source Files
| File | Description |
|---|---|
source/gmp/include/xmedia_svp.h |
SVP API definitions (all structures and function declarations) |
sample/npu/demo_ai/sample_svp_main.c |
Complete SVP video development sample (main flow) |
sample/npu/demo_ai/sample_svp_main.h |
Sample header (type definitions) |
source/gmp/usr/svp/src/post_process/yolov5.c |
YOLOv5 post-processing |
source/gmp/usr/svp/src/post_process/yolov5.h |
YOLOv5 post-processing header |
sample/npu/demo_ai/model/ |
Model file directory |
source/gmp/include/xmedia_mmz.h |
MMZ memory management API |
source/gmp/include/xmedia_vgs.h |
VGS graphics drawing API |