This plugin is currently in alpha status.

qtimlqairt is only available in qcom-multimedia-proprietary-image
For more information on QLI images refer to Qualcomm Linux release

Overview

qtimlqairt is a GStreamer inference element that executes neural network models using the Qualcomm AI Runtime (QAIRT), the unified runtime that consolidates the earlier SNPE and QNN runtimes into a single, runtime-independent API. It supersedes the runtime-specific qtimlsnpe and qtimlqnn elements: a single element loads the model and executes it on the selected hardware backend (CPU, GPU, or HTP), exposing one consistent set of properties regardless of the target. The element operates entirely in tensor mode: it accepts input tensors on its sink pad and produces output tensors on its source pad according to the model’s declared input and output specifications. qtimlqairt runs models prepared for QAIRT. It accepts a QAIRT/SNPE Deep Learning Container (.dlc) or a cached context binary (.bin). Note that QNN model.so model libraries are not supported by QAIRT and must be reconverted to DLC. The element is limited to model execution. It does not perform preprocessing, tensor reshaping, batching, layout conversion, or model-specific post-processing. These functions are expected to be handled by adjacent elements in the pipeline.

Key Responsibilities

qtimlqairt is responsible for:

Loading and executing a QAIRT model artifact — a .dlc container or a cached .bin binary
Selecting the execution backend (CPU, GPU, or HTP) through the backend property
Accepting preformatted input tensors from upstream elements
Producing output tensors that match the model output signature
Negotiating tensor data types and dimensions with adjacent pipeline elements
Dequantizing quantized model outputs to FLOAT32 for downstream elements
Propagating tensor metadata required by downstream elements
Managing DMA-backed buffers through GstMLBufferPool to reduce unnecessary memory copies

In practice, qtimlqairt serves as the inference stage in the pipeline, while tensor preparation and result interpretation are handled externally.

Example Pipeline

Download Required Files

File	Download	Save as
Yolov8 Detection W8A8 model	Export from Qualcomm AI Hub	`yolov8_det_w8a8.dlc`
Detection labels	yolov8.json	`yolov8.json`
Sample video	Input video	`ai_demo_sample.mp4`

Copy files to device

# Replace $HOME to the appropriate device path before running the commands.
# For QLI:    /root
# For Ubuntu: /home/ubuntu
# Modify this based on your platform and ensure files are copied to the correct location on the device.
# Run from your host machine — replace <user> and <device-ip>

ssh <user>@<device-ip> "mkdir -p $HOME/{models,labels,media,media/output}"
scp yolov8_det_w8a8.dlc          <user>@<device-ip>:$HOME/models/
scp yolov8.json                  <user>@<device-ip>:$HOME/labels/
scp ai_demo_sample.mp4    <user>@<device-ip>:$HOME/media/

Connect to device

# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip>

Set environment variables

Run below command on your device

mkdir -p $HOME/{models,labels,media,media/output}
export MODEL_NAME=yolov8_det_w8a8.dlc
export LABELS_NAME=yolov8.json
export SRC_VIDEO_NAME=ai_demo_sample.mp4

Run the pipeline

gst-launch-1.0 -e --gst-debug=2 \
filesrc location=$HOME/media/$SRC_VIDEO_NAME ! qtdemux ! h264parse ! \
v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12 ! queue ! \
tee name=t ! queue ! qtivcomposer name=mixer ! waylandsink fullscreen=true sync=false \
t. ! queue ! qtimlvconverter ! queue ! \
qtimlqairt model=$HOME/models/$MODEL_NAME backend=libQairtHtp.so tensors="<boxes,scores,class_idx>" ! queue ! \
qtimlpostprocess module=yolov8 labels=$HOME/labels/$LABELS_NAME settings="{\"confidence\": 51.0}" ! video/x-raw ! queue ! mixer.

Hierarchy

GObject
   GstObject
      GstElement
         GstBaseTransform
            qtimlqairt

Pad Templates

sink

Capabilities
`neural-network/tensors`	`type: { INT8, UINT8, INT16, UINT16, INT32, UINT32, INT64, UINT64, FLOAT16, FLOAT32 }`
Availability: Always
Direction: sink

src

Capabilities
`neural-network/tensors`	`type: { INT8, UINT8, INT16, UINT16, INT32, UINT32, INT64, UINT64, FLOAT16, FLOAT32 }`
Availability: Always
Direction: source

Element Properties

Property	Description
`backend`	Name of the QAIRT backend shared library to load. Determines which hardware accelerator is used for inference. Pass the library filename (e.g. `libQairtHtp.so`) or a full path if the library is not on the default library search path. `Type: String` `Default: libQairtCpu.so` `Flags: readable/writable/construct`
`layers`	List of output layer names. Should be set if the model has more than one output. `Type: Array of String` `Default: []` `Note: Mutually exclusive with tensors` `Flags: readable/writable`
`model`	Path to the model file. Either a QAIRT/SNPE `.dlc` container or a cached context `.bin` binary. This property is required. `Type: String` `Default: NULL` `Flags: readable/writable`
`priority`	Execution scheduling priority hint for the QAIRT runtime. Best-effort; ignored by backends that do not support it. `Type: Enum` `Default: normal` `Range:` `(0): normal - Normal priority` `(1): low - Lower priority` `(2): normal-high - Between normal and high` `(3): high - Higher than normal` `(4): critical - Critical priority` `Flags: readable/writable`
`tensors`	List of output tensor names. Alternative to the output layer list. Outputs are generated in the order defined in this list. `Type: Array of String` `Default: []` `Note: Mutually exclusive with layers` `Flags: readable/writable` `Example: tensors="<boxes,scores,class_idx>"`

Input and Output Behavior

Input Tensors

qtimlqairt exposes a single sink pad, but it supports both single-input and multi-input models. For multi-input models, all required tensors are delivered through the same sink pad as a tensor set. Input tensors must be fully prepared before they reach qtimlqairt. Expected tensor layout, shape, data type, and batch size are determined by:

The model input signature (read from the DLC or context binary at load time)
Caps negotiation with upstream elements

Typical upstream elements include:

qtimlvconverter — for scaling, color conversion, normalization, and quantization (if required)

qtimlqairt does not modify, reshape, batch, or reinterpret incoming tensors. It passes them to the QAIRT runtime as received.

Output Tensors

qtimlqairt exposes a single source pad and produces output tensors that follow the model’s declared output signature. Models with multiple output tensors are fully supported, and all outputs are emitted together on the source pad. Supported output behavior includes:

Single-tensor and multi-tensor outputs
Arbitrary tensor shapes and ranks, including batch and depth dimensions
Ordering and selection of output tensors using the layers or tensors property
Dequantization to FLOAT32: when a model produces quantized (fixed-point) outputs, qtimlqairt dequantizes them to FLOAT32 using the tensor’s quantization parameters (scale and offset) before emitting them. This lets downstream post-processing consume floating-point tensors without changing the model artifact.

The generated output tensors are intended for downstream post-processing stages, which are responsible for decoding model-specific results such as classification outputs, detection results, segmentation masks, landmark data, and other structured inference outputs.

Supported Data Types

qtimlqairt supports the tensor data types provided by the QAIRT runtime and the selected execution backend, subject to caps negotiation with adjacent elements. Supported data types include:

INT8
UINT8
INT16
UINT16
INT32
UINT32
INT64
UINT64
FLOAT16
FLOAT32

Quantized (fixed-point) model tensors are mapped to the corresponding integer type on the sink pad; model outputs are dequantized and emitted as FLOAT32 on the source pad. The element does not impose additional data-type restrictions beyond those required by the runtime, the selected backend, and negotiated pipeline caps.

Backends

A QAIRT backend defines the hardware target used to run a model. It is selected through the backend property, which accepts the name or path of a QAIRT backend shared library. Choosing a backend lets qtimlqairt offload inference from the CPU to an optimized hardware accelerator. The backend must be compatible with the model’s quantization. This is a property of the model artifact and is fixed at conversion time — the element cannot change it at runtime.

Library	Hardware	Typical use case
`libQairtHtp.so`	Hexagon Tensor Processor	Preferred backend where available. Best performance and power efficiency for quantized models.
`libQairtGpu.so`	GPU	Floating-point models and workloads that benefit from GPU parallelism.
`libQairtCpu.so`	CPU	Reference execution, bring-up, debugging, or systems without hardware acceleration.

libQairtHtp.so

Runs the model on the AI accelerator. This is the recommended backend for quantized models.

libQairtGpu.so

Runs the model on the GPU. The GPU backend is preferred for float models.

libQairtCpu.so

Runs the model on the CPU. This is the default when backend is not explicitly set. Suitable for reference execution and bring-up. Quantized-operation support on CPU is model-dependent.

Runtime Memory Behavior and GAP Handling

QAIRT Memory Model

qtimlqairt operates within the memory model of the QAIRT runtime. The element uses DMA buffers via GstMLBufferPool to minimize memory copies and maintain zero-copy transport where possible. The element discovers input/output tensor metadata (count, shape, type) at model load time and configures buffer pools accordingly. Mapped tensor memory is bound directly to the runtime’s tensors for execution.

GAP Buffer Handling

qtimlqairt is GAP-aware and correctly handles input buffers marked with GST_BUFFER_FLAG_GAP. When a GAP buffer is received, the element skips inference and forwards the buffer downstream. This preserves timing and synchronization while explicitly indicating that no valid inference input is available for that timestamp. GAP buffers commonly appear in conditional AI pipelines, such as cascaded workflows where later inference stages run only when earlier stages produce valid regions of interest.

Usage

Single-Stage AI Inference on Live Camera Stream (HTP)

This example demonstrates real-time inference on a live camera stream using a single instance of qtimlqairt with the HTP backend and a quantized model. Inference results are attached to each GstBuffer as MLMeta, allowing downstream elements to access synchronized metadata directly from the frame. An overlay stage then uses this metadata to render annotations such as bounding boxes, labels, or key-points before display or further processing.

Download Required Files

File	Download	Save as
Yolov8 Detection W8A8 model	Export from Qualcomm AI Hub	`yolov8_det_w8a8.dlc`
Detection labels	yolov8.json	`yolov8.json`

Copy files to device

# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip> "mkdir -p $HOME/{models,labels}"
scp yolov8_det_w8a8.dlc   <user>@<device-ip>:$HOME/models/
scp yolov8.json           <user>@<device-ip>:$HOME/labels/

Connect to device

# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip>

Set environment variables

Run below command on your device

mkdir -p $HOME/{models,labels}
export MODEL_NAME=yolov8_det_w8a8.dlc
export LABELS_NAME=yolov8.json

Run the pipeline

gst-launch-1.0 -e --gst-debug=2 \
qticamsrc name=camsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! \
tee name=t ! queue ! qtimetamux name=obj_mux ! qtivoverlay ! waylandsink fullscreen=true sync=false \
t. ! queue ! qtimlvconverter ! queue ! \
qtimlqairt model=$HOME/models/$MODEL_NAME backend=libQairtHtp.so tensors="<boxes,scores,class_idx>" ! queue ! \
qtimlpostprocess module=yolov8 labels=$HOME/labels/$LABELS_NAME settings="{\"confidence\": 51.0}" ! text/x-raw ! queue ! obj_mux.

Single-Stage AI Inference on Live Camera Stream (GPU)

This example demonstrates the same single-stage inference workflow using the GPU backend instead of HTP. The GPU backend is floating-point, so it requires a float model — use a float DLC.

Download Required Files

File	Download	Save as
Yolov8 Detection Float model	Export from Qualcomm AI Hub	`yolov8_det_float.dlc`
Detection labels	yolov8.json	`yolov8.json`

Copy files to device

# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip> "mkdir -p $HOME/{models,labels}"
scp yolov8_det_float.dlc   <user>@<device-ip>:$HOME/models/
scp yolov8.json            <user>@<device-ip>:$HOME/labels/

Connect to device

# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip>

Set environment variables

Run below command on your device

export MODEL_NAME=yolov8_det_float.dlc
export LABELS_NAME=yolov8.json

Run the pipeline

gst-launch-1.0 -e --gst-debug=2 \
qticamsrc name=camsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! \
tee name=t ! queue ! qtimetamux name=obj_mux ! qtivoverlay ! waylandsink fullscreen=true sync=false \
t. ! queue ! qtimlvconverter ! queue ! \
qtimlqairt model=$HOME/models/$MODEL_NAME backend=libQairtGpu.so tensors="<boxes,scores,class_idx>" ! queue ! \
qtimlpostprocess module=yolov8 labels=$HOME/labels/$LABELS_NAME settings="{\"confidence\": 51.0}" ! text/x-raw ! queue ! obj_mux.

Overview

Supported AI Models

Plugins

Advanced Topics

qtimlqairt

Overview

Key Responsibilities

Example Pipeline

Hierarchy

Pad Templates

sink

src

Element Properties

Input and Output Behavior

Input Tensors

Output Tensors

Supported Data Types

Backends

libQairtHtp.so

libQairtGpu.so

libQairtCpu.so

Runtime Memory Behavior and GAP Handling

QAIRT Memory Model

GAP Buffer Handling

Usage

Single-Stage AI Inference on Live Camera Stream (HTP)

Single-Stage AI Inference on Live Camera Stream (GPU)

​Overview

​Key Responsibilities

​Example Pipeline

​Hierarchy

​Pad Templates

​sink

​src

​Element Properties

​Input and Output Behavior

​Input Tensors

​Output Tensors

​Supported Data Types

​Backends

​libQairtHtp.so

​libQairtGpu.so

​libQairtCpu.so

​Runtime Memory Behavior and GAP Handling

​QAIRT Memory Model

​GAP Buffer Handling

​Usage

​Single-Stage AI Inference on Live Camera Stream (HTP)

​Single-Stage AI Inference on Live Camera Stream (GPU)

Overview

Key Responsibilities

Example Pipeline

Hierarchy

Pad Templates

sink

src

Element Properties

Input and Output Behavior

Input Tensors

Output Tensors

Supported Data Types

Backends

libQairtHtp.so

libQairtGpu.so

libQairtCpu.so

Runtime Memory Behavior and GAP Handling

QAIRT Memory Model

GAP Buffer Handling

Usage

Single-Stage AI Inference on Live Camera Stream (HTP)

Single-Stage AI Inference on Live Camera Stream (GPU)