.xmm is the proprietary neural network model format for the GK7206 NPU. It is a binary file containing:
| Content | Description |
|---|---|
| Model weights | Quantized network parameters |
| Compute graph structure | Operator arrangement and inter-layer connections |
| Private data | Input/output tensor descriptors, quantization parameters, and more |
| File header | Starts with the "ZZZZ" magic number |
Model source: usually a model exported from a training framework (PyTorch/ONNX), then quantized and compiled into a .xmm file with the XMTVM conversion tool from Goke.
Note
The .bin model format is what the higher-level SVP interface uses (internally it also loads a .xmm through the NPU).
GK7206 provides two API sets for different scenarios:
| Approach | API level | Header | Use case |
|---|---|---|---|
| Low-level CL interface | xmedia_cl_* |
xmedia_cl.h |
Load .xmm directly and fully control the inference flow |
| High-level SVP interface | xmedia_svp_* |
xmedia_svp.h |
Use .bin models with built-in pre/post-processing (NMS, etc.), ready to use |
Tip
For the high-level SVP interface, see SVP Video Processing.
| Stage | Key action | Description |
|---|---|---|
| Stage 1 | System initialization | Initialize the XMedia system module and the Compute Library (NPU runtime) |
| Stage 2 | Get the NPU device and create a context | Get the device count, then the device ID, then create the context as the resource base |
| Stage 3 | Query model memory needs | Use xmedia_cl_graph_querysize_from_file() to get the workspace and weight sizes |
| Stage 4 | Allocate NPU memory (MMZ) | Allocate physically contiguous memory for workspace and weights; ordinary malloc() cannot be used |
| Stage 5 | Load the model | Load the .xmm with xmedia_cl_graph_loadmodel_from_file_withmem() or the simplified variant |
| Stage 6 | Get input/output tensor info | Query the input and output counts and get the full tensor descriptors |
| Stage 7 | Prepare data and run inference | Allocate input/output buffers, set tensor addresses, flush the cache, then run inference |
| Stage 8 | Release resources | Unload the model, destroy the context, release the device, de-initialize, and free MMZ memory |
#include "xmedia_cl.h"
#include "xmedia_mmz.h"
#include "xmedia_sys.h"
// initialize the XMedia system module
ret = xmedia_sys_init(XMEDIA_NULL);
// initialize the Compute Library (NPU runtime)
ret = xmedia_cl_init();
xmedia_cl_u32 num_devices = 0;
xmedia_cl_device_id *devices = NULL;
xmedia_cl_context context = NULL;
// first call: get the device count
ret = xmedia_cl_get_device_ids(XMEDIA_CL_DEVICE_NPU, NULL, &num_devices);
// allocate the device array
devices = calloc(num_devices, sizeof(xmedia_cl_device_id));
// second call: get the device ID
ret = xmedia_cl_get_device_ids(XMEDIA_CL_DEVICE_NPU, devices, &num_devices);
// create the context (the basis of all resource management)
xmedia_cl_s32 err_code = 0;
context = xmedia_cl_create_context(num_devices, devices, &err_code);
xmedia_cl_u32 worksize, weightsize;
// query the workspace and weight sizes the model needs
ret = xmedia_cl_graph_querysize_from_file("data/neuron_network.xmm",
&worksize, &weightsize);
worksize: scratch buffer for intermediate results (workspace)weightsize: space taken by the model weightsxmedia_u64 phy_addr[4] = {0};
void *virt_addr[4] = {0};
// allocate workspace
if (worksize) {
XMEDIA_API_SYS_MmzAlloc_Cached(&phy_addr[0], &virt_addr[0],
"npu_workspace", NULL, worksize);
}
// allocate weights
if (weightsize) {
XMEDIA_API_SYS_MmzAlloc_Cached(&phy_addr[1], &virt_addr[1],
"npu_weight", NULL, weightsize);
}
Important
The NPU uses physically contiguous memory (MMZ). Ordinary
malloc()cannot be used; usexmedia_mmz_alloc()/xmedia_mmz_map().
xmedia_cl_graph graph = NULL;
// load from file using memory supplied by the caller
ret = xmedia_cl_graph_loadmodel_from_file_withmem(
&context,
"data/neuron_network.xmm", // .xmm file path
virt_addr[0], // workspace address
worksize, // workspace size
virt_addr[1], // weight address
weightsize, // weight size
&graph // output: graph handle
);
A simplified variant is also available (the SDK allocates memory for you):
ret = xmedia_cl_graph_loadmodel_from_file(&context, "data/neuron_network.xmm", &graph);
xmedia_cl_tensor_info_inout input = {0}, output = {0};
xmedia_cl_u32 input_num = 0, output_num = 0;
// first pass: get the input count
ret = xmedia_cl_graph_get_input(graph, input_num, &input);
input_num = input.num;
// allocate the tensor array
malloc_inout_tensor_mem(&input);
// second pass: get the full input tensor descriptors (shape, size, type, quant)
ret = xmedia_cl_graph_get_input(graph, input_num, &input);
// get the outputs the same way
ret = xmedia_cl_graph_get_output(graph, output_num, &output);
output_num = output.num;
malloc_inout_tensor_mem(&output);
ret = xmedia_cl_graph_get_output(graph, output_num, &output);
Each tensor contains:
typedef struct {
xmedia_cl_u32 tensor_id;
void *addr; // data address (set by the caller)
xmedia_cl_tensor_shape shape; // dimensions (N,C,H,W)
xmedia_cl_tensor_quant quant; // quantization parameters (scale, zero_point)
xmedia_cl_u32 size; // data size in bytes
xmedia_cl_s8 *name; // tensor name
} xmedia_cl_tensor;
// allocate input/output buffers
XMEDIA_API_SYS_MmzAlloc_Cached(&phy_addr[2], &virt_addr[2],
"npu_input", NULL, inputsize);
XMEDIA_API_SYS_MmzAlloc_Cached(&phy_addr[3], &virt_addr[3],
"npu_output", NULL, outputsize);
// set the input data address (write the pre-processed data here)
input.tensor[0].addr = virt_addr[2];
// ... copy the actual image data into input.tensor[i].addr ...
// set the output data address
output.tensor[0].addr = virt_addr[3];
// flush the cache (so the NPU sees the latest data)
XMEDIA_API_SYS_MmzFlushCache(phy_addr[2], virt_addr[2], inputsize);
// bind the inputs and outputs to the graph
ret = xmedia_cl_graph_set_inout(graph, &input, &output);
// run inference (synchronous)
ret = xmedia_cl_graph_process(graph);
After inference completes, the results are in output.tensor[i].addr.
// unload the model
xmedia_cl_graph_unload(graph);
// release the context
xmedia_cl_release_context(context);
// release the device
xmedia_cl_release_device_ids(devices, &num_devices);
free(devices);
// de-initialize
xmedia_cl_uninit();
xmedia_sys_exit();
// free the MMZ memory
for (i = 0; i < 4; i++) {
if (virt_addr[i]) XMEDIA_API_SYS_MmzFree(phy_addr[i], virt_addr[i]);
}
Tip
- Before using the NPU, load the kernel driver:
- Load the ko driver (in
out/xm7206xxx/ko):./load xm7206v11a -i- Or load the NPU module manually:
insmod xm_npu.ko
- For detailed .xmm usage see
sample/npu/xmm.