> ## Documentation Index
> Fetch the complete documentation index at: https://imsdkdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# qtimlqairt

> GStreamer inference element that executes neural network models using the unified Qualcomm AI Runtime (QAIRT).

> [!NOTE]
>
> This plugin is currently in alpha status.

> [!NOTE]
>
> qtimlqairt is only available in `qcom-multimedia-proprietary-image`  \\
>
> For more information on QLI images refer to [Qualcomm Linux release](https://dragonwingdocs.qualcomm.com/Key-Documents/Yocto-Guide/add-image-recipes#qualcomm-reference-images)

# Overview

**qtimlqairt** is a GStreamer inference element that executes neural network models using the **Qualcomm AI Runtime (QAIRT)**, the unified runtime that consolidates the earlier **SNPE** and **QNN** runtimes into a single, runtime-independent API. It supersedes the runtime-specific [`qtimlsnpe`](qtimlsnpe) and [`qtimlqnn`](qtimlqnn) elements: a single element loads the model and executes it on the selected hardware backend (**CPU**, **GPU**, or **HTP**), exposing one consistent set of properties regardless of the target.

The element operates entirely in **tensor mode**: it accepts input tensors on its sink pad and produces output tensors on its source pad according to the model's declared input and output specifications.

`qtimlqairt` runs models prepared for QAIRT. It accepts a **QAIRT/SNPE Deep Learning Container (`.dlc`)** or a **cached context binary (`.bin`)**. Note that QNN `model.so` model libraries are not supported by QAIRT and must be reconverted to DLC.

The element is limited to model execution. It does not perform preprocessing, tensor reshaping, batching, layout conversion, or model-specific post-processing. These functions are expected to be handled by adjacent elements in the pipeline.

### Key Responsibilities

**qtimlqairt** is responsible for:

* Loading and executing a QAIRT model artifact — a `.dlc` container or a cached `.bin` binary
* Selecting the execution backend (CPU, GPU, or HTP) through the `backend` property
* Accepting preformatted input tensors from upstream elements
* Producing output tensors that match the model output signature
* Negotiating tensor data types and dimensions with adjacent pipeline elements
* Dequantizing quantized model outputs to `FLOAT32` for downstream elements
* Propagating tensor metadata required by downstream elements
* Managing DMA-backed buffers through `GstMLBufferPool` to reduce unnecessary memory copies

In practice, **qtimlqairt** serves as the inference stage in the pipeline, while tensor preparation and result interpretation are handled externally.

![](https://mintcdn.com/qimsdk/EQ4Q-kHTLfweoGjc/plugin-reference/images/qtimlqairt_overview.png?fit=max\&auto=format\&n=EQ4Q-kHTLfweoGjc\&q=85\&s=5a32e971bf8cc341ed42c03097df0f5f)

## Example Pipeline

![](https://mintlify.s3.us-west-1.amazonaws.com/qimsdk/plugin-reference/images/qtimlqairt_example_pipeline.png)

**1. Download Required Files**

| File | Download | Save as |
| - | - | - |
| Yolov8 Detection W8A8 model | [Export from Qualcomm AI Hub](https://aihub.qualcomm.com/iot/models/yolov8_det) | `yolov8_det_w8a8.dlc` |
| Detection labels | [yolov8.json](../labels/yolov8.json) | `yolov8.json` |
| Sample video | [Input video](https://github.com/qualcomm/sample-apps-for-qualcomm-linux/raw/refs/heads/main/qualcomm-linux/artifacts/videos/demo_samples/) | `ai_demo_sample.mp4` |

**2. Copy files to device**

```bash SCP (SSH) theme={null}
# Replace $HOME to the appropriate device path before running the commands.
# For QLI:    /root
# For Ubuntu: /home/ubuntu
# Modify this based on your platform and ensure files are copied to the correct location on the device.
# Run from your host machine — replace <user> and <device-ip>

ssh <user>@<device-ip> "mkdir -p $HOME/{models,labels,media,media/output}"
scp yolov8_det_w8a8.dlc          <user>@<device-ip>:$HOME/models/
scp yolov8.json                  <user>@<device-ip>:$HOME/labels/
scp ai_demo_sample.mp4    <user>@<device-ip>:$HOME/media/
```

**3. Connect to device**

```bash SCP (SSH) theme={null}
# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip>
```

**4. Set environment variables**

Run below command on your device

```bash theme={null}
mkdir -p $HOME/{models,labels,media,media/output}
export MODEL_NAME=yolov8_det_w8a8.dlc
export LABELS_NAME=yolov8.json
export SRC_VIDEO_NAME=ai_demo_sample.mp4
```

**5. Run the pipeline**

```bash theme={null}
gst-launch-1.0 -e --gst-debug=2 \
filesrc location=$HOME/media/$SRC_VIDEO_NAME ! qtdemux ! h264parse ! \
v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12 ! queue ! \
tee name=t ! queue ! qtivcomposer name=mixer ! waylandsink fullscreen=true sync=false \
t. ! queue ! qtimlvconverter ! queue ! \
qtimlqairt model=$HOME/models/$MODEL_NAME backend=libQairtHtp.so tensors="<boxes,scores,class_idx>" ! queue ! \
qtimlpostprocess module=yolov8 labels=$HOME/labels/$LABELS_NAME settings="{\"confidence\": 51.0}" ! video/x-raw ! queue ! mixer.
```

# Hierarchy

[GObject](https://docs.gtk.org/gobject/)\

   [GstObject](https://gstreamer.freedesktop.org/documentation/gstreamer/gstobject.html?gi-language=c)\

      [GstElement](https://gstreamer.freedesktop.org/documentation/gstreamer/gstelement.html?gi-language=c)\

         [GstBaseTransform](https://gstreamer.freedesktop.org/documentation/base/gstbasetransform.html?gi-language=c)\

            qtimlqairt

# Pad Templates

### sink

| Capabilities | |
| - | - |
| `neural-network/tensors` | `type: { INT8, UINT8, INT16, UINT16, INT32, UINT32, INT64, UINT64, FLOAT16, FLOAT32 }` |
| Availability: *Always* | |
| Direction: *sink* | |

### src

| Capabilities | |
| - | - |
| `neural-network/tensors` | `type: { INT8, UINT8, INT16, UINT16, INT32, UINT32, INT64, UINT64, FLOAT16, FLOAT32 }` |
| Availability: *Always* | |
| Direction: *source* | |

# Element Properties

| Property | Description |
| - | - |
| `backend` | Name of the QAIRT backend shared library to load. Determines which hardware accelerator is used for inference. Pass the library filename (e.g. `libQairtHtp.so`) or a full path if the library is not on the default library search path. `Type: String` `Default: libQairtCpu.so` `Flags: readable/writable/construct` |
| `layers` | List of output layer names. Should be set if the model has more than one output. `Type: Array of String` `Default: []` `Note: Mutually exclusive with tensors` `Flags: readable/writable` |
| `model` | Path to the model file. Either a QAIRT/SNPE `.dlc` container or a cached context `.bin` binary. This property is required. `Type: String` `Default: NULL` `Flags: readable/writable` |
| `priority` | Execution scheduling priority hint for the QAIRT runtime. Best-effort; ignored by backends that do not support it. `Type: Enum` `Default: normal` `Range:`    `(0): normal - Normal priority`    `(1): low - Lower priority`    `(2): normal-high - Between normal and high`    `(3): high - Higher than normal`    `(4): critical - Critical priority` `Flags: readable/writable` |
| `tensors` | List of output tensor names. Alternative to the output layer list. Outputs are generated in the order defined in this list. `Type: Array of String` `Default: []` `Note: Mutually exclusive with layers` `Flags: readable/writable`  `Example: tensors="<boxes,scores,class_idx>"` |

# Input and Output Behavior

### Input Tensors

**qtimlqairt** exposes a single sink pad, but it supports both single-input and multi-input models. For multi-input models, all required tensors are delivered through the same sink pad as a tensor set.

Input tensors must be fully prepared before they reach **qtimlqairt**. Expected tensor layout, shape, data type, and batch size are determined by:

* The model input signature (read from the DLC or context binary at load time)
* Caps negotiation with upstream elements

Typical upstream elements include:

* [`qtimlvconverter`](qtimlvconverter) — for scaling, color conversion, normalization, and quantization (if required)

**qtimlqairt** does not modify, reshape, batch, or reinterpret incoming tensors. It passes them to the QAIRT runtime as received.

### Output Tensors

**qtimlqairt** exposes a single source pad and produces output tensors that follow the model's declared output signature. Models with multiple output tensors are fully supported, and all outputs are emitted together on the source pad.

Supported output behavior includes:

* Single-tensor and multi-tensor outputs
* Arbitrary tensor shapes and ranks, including batch and depth dimensions
* Ordering and selection of output tensors using the `layers` or `tensors` property
* **Dequantization to `FLOAT32`**: when a model produces quantized (fixed-point) outputs, **qtimlqairt** dequantizes them to `FLOAT32` using the tensor's quantization parameters (scale and offset) before emitting them. This lets downstream post-processing consume floating-point tensors without changing the model artifact.

The generated output tensors are intended for downstream post-processing stages, which are responsible for decoding model-specific results such as classification outputs, detection results, segmentation masks, landmark data, and other structured inference outputs.

# Supported Data Types

**qtimlqairt** supports the tensor data types provided by the QAIRT runtime and the selected execution backend, subject to caps negotiation with adjacent elements.

Supported data types include:

* `INT8`
* `UINT8`
* `INT16`
* `UINT16`
* `INT32`
* `UINT32`
* `INT64`
* `UINT64`
* `FLOAT16`
* `FLOAT32`

Quantized (fixed-point) model tensors are mapped to the corresponding integer type on the sink pad; model outputs are dequantized and emitted as `FLOAT32` on the source pad. The element does not impose additional data-type restrictions beyond those required by the runtime, the selected backend, and negotiated pipeline caps.

# Backends

A QAIRT **backend** defines the hardware target used to run a model. It is selected through the `backend` property, which accepts the name or path of a QAIRT backend shared library. Choosing a backend lets **qtimlqairt** offload inference from the CPU to an optimized hardware accelerator.

The backend must be compatible with the model's quantization. This is a property of the model artifact and is fixed at conversion time — the element cannot change it at runtime.

| Library | Hardware | Typical use case |
| - | - | - |
| `libQairtHtp.so` | Hexagon Tensor Processor | Preferred backend where available. Best performance and power efficiency for quantized models. |
| `libQairtGpu.so` | GPU | Floating-point models and workloads that benefit from GPU parallelism. |
| `libQairtCpu.so` | CPU | Reference execution, bring-up, debugging, or systems without hardware acceleration. |

### libQairtHtp.so

Runs the model on the AI accelerator. This is the recommended backend for quantized models.

### libQairtGpu.so

Runs the model on the GPU. The GPU backend is preferred for float models.

### libQairtCpu.so

Runs the model on the CPU. This is the default when `backend` is not explicitly set. Suitable for reference execution and bring-up. Quantized-operation support on CPU is model-dependent.

# Runtime Memory Behavior and GAP Handling

### QAIRT Memory Model

**qtimlqairt** operates within the memory model of the QAIRT runtime. The element uses DMA buffers via `GstMLBufferPool` to minimize memory copies and maintain zero-copy transport where possible.

The element discovers input/output tensor metadata (count, shape, type) at model load time and configures buffer pools accordingly. Mapped tensor memory is bound directly to the runtime's tensors for execution.

### GAP Buffer Handling

**qtimlqairt** is GAP-aware and correctly handles input buffers marked with `GST_BUFFER_FLAG_GAP`.

When a GAP buffer is received, the element skips inference and forwards the buffer downstream. This preserves timing and synchronization while explicitly indicating that no valid inference input is available for that timestamp.

GAP buffers commonly appear in conditional AI pipelines, such as cascaded workflows where later inference stages run only when earlier stages produce valid regions of interest.

# Usage

### Single-Stage AI Inference on Live Camera Stream (HTP)

This example demonstrates real-time inference on a live camera stream using a single instance of **qtimlqairt** with the HTP backend and a quantized model. Inference results are attached to each `GstBuffer` as `MLMeta`, allowing downstream elements to access synchronized metadata directly from the frame. An overlay stage then uses this metadata to render annotations such as bounding boxes, labels, or key-points before display or further processing.

![](https://mintcdn.com/qimsdk/EQ4Q-kHTLfweoGjc/plugin-reference/images/qtimlqairt_usage1.png?fit=max\&auto=format\&n=EQ4Q-kHTLfweoGjc\&q=85\&s=3a3093d60a76eba9417818da7f8dd94c)

**1. Download Required Files**

| File | Download | Save as |
| - | - | - |
| Yolov8 Detection W8A8 model | [Export from Qualcomm AI Hub](https://aihub.qualcomm.com/iot/models/yolov8_det) | `yolov8_det_w8a8.dlc` |
| Detection labels | [yolov8.json](../labels/yolov8.json) | `yolov8.json` |

**2. Copy files to device**

```bash SCP (SSH) theme={null}
# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip> "mkdir -p $HOME/{models,labels}"
scp yolov8_det_w8a8.dlc   <user>@<device-ip>:$HOME/models/
scp yolov8.json           <user>@<device-ip>:$HOME/labels/
```

**3. Connect to device**

```bash SCP (SSH) theme={null}
# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip>
```

**4. Set environment variables**

Run below command on your device

```bash theme={null}
mkdir -p $HOME/{models,labels}
export MODEL_NAME=yolov8_det_w8a8.dlc
export LABELS_NAME=yolov8.json
```

**5. Run the pipeline**

```bash Run the pipeline theme={null}
gst-launch-1.0 -e --gst-debug=2 \
qticamsrc name=camsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! \
tee name=t ! queue ! qtimetamux name=obj_mux ! qtivoverlay ! waylandsink fullscreen=true sync=false \
t. ! queue ! qtimlvconverter ! queue ! \
qtimlqairt model=$HOME/models/$MODEL_NAME backend=libQairtHtp.so tensors="<boxes,scores,class_idx>" ! queue ! \
qtimlpostprocess module=yolov8 labels=$HOME/labels/$LABELS_NAME settings="{\"confidence\": 51.0}" ! text/x-raw ! queue ! obj_mux.
```

### Single-Stage AI Inference on Live Camera Stream (GPU)

This example demonstrates the same single-stage inference workflow using the GPU backend instead of HTP. The GPU backend is floating-point, so it requires a **float** model — use a float DLC.

![](https://mintcdn.com/qimsdk/EQ4Q-kHTLfweoGjc/plugin-reference/images/qtimlqairt_usage2.png?fit=max\&auto=format\&n=EQ4Q-kHTLfweoGjc\&q=85\&s=47facffb0cf620e698bea7efcd7b74b5)

**1. Download Required Files**

| File | Download | Save as |
| - | - | - |
| Yolov8 Detection Float model | [Export from Qualcomm AI Hub](https://aihub.qualcomm.com/iot/models/yolov8_det) | `yolov8_det_float.dlc` |
| Detection labels | [yolov8.json](../labels/yolov8.json) | `yolov8.json` |

**2. Copy files to device**

```bash SCP (SSH) theme={null}
# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip> "mkdir -p $HOME/{models,labels}"
scp yolov8_det_float.dlc   <user>@<device-ip>:$HOME/models/
scp yolov8.json            <user>@<device-ip>:$HOME/labels/
```

**3. Connect to device**

```bash SCP (SSH) theme={null}
# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip>
```

**4. Set environment variables**

Run below command on your device

```bash theme={null}
export MODEL_NAME=yolov8_det_float.dlc
export LABELS_NAME=yolov8.json
```

**5. Run the pipeline**

```bash Run the pipeline theme={null}
gst-launch-1.0 -e --gst-debug=2 \
qticamsrc name=camsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! \
tee name=t ! queue ! qtimetamux name=obj_mux ! qtivoverlay ! waylandsink fullscreen=true sync=false \
t. ! queue ! qtimlvconverter ! queue ! \
qtimlqairt model=$HOME/models/$MODEL_NAME backend=libQairtGpu.so tensors="<boxes,scores,class_idx>" ! queue ! \
qtimlpostprocess module=yolov8 labels=$HOME/labels/$LABELS_NAME settings="{\"confidence\": 51.0}" ! text/x-raw ! queue ! obj_mux.
```