> ## Documentation Index
> Fetch the complete documentation index at: https://imsdkdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# AI With Custom Preprocess

> Ready-to-run C native applications demonstrating QIM SDK AI with custom preprocessing

This section covers QIM SDK AI sample applications that demonstrate custom preprocessing.

***

## Prerequisites

> [accordion] Try me
>
> > [!NOTE]
> >
> > Some of the steps in the pre-requisties will be removed from future releases
> > once the necessary fixes are mainlined.
>
> **1. Set up Wi-Fi**
>
> Connect to the Wireless Access Point (Wi-Fi Router):
>
> ```bash theme={null}
> nmcli dev wifi connect <WiFi-SSID> password <WiFi-password>
> ```
>
> Check the connection and device status:
>
> ```bash theme={null}
> nmcli -p device
> ```
>
> **Login to the target device**
>
> Locate the IP address of the device according to the type of network connection,
> using the UART console on the Linux host:
>
> For Ethernet:
>
> ```bash theme={null}
> ip address show eth2
> ```
>
> For Wi-Fi:
>
> ```bash theme={null}
> ip address show wlp1s0
> ```
>
> Use the IP address from the Linux host to establish an SSH connection to the device:
>
> ```bash theme={null}
> ssh root@<ip-address>
> ```
>
> Example:
>
> ```bash theme={null}
> ssh root@192.168.0.222
> ```
>
> Connect to the SSH shell using the following password:
>
> ```bash theme={null}
> oelinux123
> ```
>
> **2. Download Models and Artifacts**
>
> On the target device, obtain the `download_artifacts.sh` script, set executable
> permissions, and run it to download the model, media, and label files:
>
> ```bash theme={null}
> cd /tmp/
> curl -L -O https://raw.githubusercontent.com/qualcomm/sample-apps-for-qualcomm-linux/refs/heads/main/qualcomm-linux/scripts/download_artifacts.sh
> chmod +x download_artifacts.sh
> ./download_artifacts.sh
> ```
>
> **3. Enable qticamsrc**
>
> In the terminal of the target device, run the following command to enable the
> `qticamsrc` on Config #2:
>
> ```bash theme={null}
> echo -n "camx" > /var/data
> efivar -n 882f8c2b-9646-435f-8de5-f208ff80c1bd-VendorDtbOverlays -w -f /var/data
> efivar -n 882f8c2b-9646-435f-8de5-f208ff80c1bd-VendorDtbOverlays -p
> sync
> reboot
> ```
>
> **4. Enable Audio and GPU Delegate**
>
> In the terminal of the target device, run the following command to enable audio:
>
> ```bash theme={null}
> systemctl stop pipewire wireplumber pipewire.socket pipewire-manager.socket
> chmod 777 /dev/dma_heap/system
> adsprpcd audiopd &
> systemctl start pipewire wireplumber
> wpctl status
> ```
>
> To set the default devices for sink and source, get the device numbers from
> `wpctl status` and run the following command:
>
> ```bash theme={null}
> wpctl set-default <device ID>
> ```
>
> In the terminal of the target device, run the following command to enable the
> GPU delegate and backend:
>
> ```bash theme={null}
> mount -o rw,remount /
> ```
>
> ```bash theme={null}
> export OCL_ICD_FILENAMES=/usr/lib/libOpenCL_adreno.so.1
> ```

***

## AI Custom Preprocessing Applications

***

### Object Detection (YOLOv8) with Custom Preprocessing

#### Common Description

`gst-custom-preprocess-yolov8` is a GStreamer sample application that demonstrates how to
run real-time object detection on Qualcomm platforms using a YOLOv8 model while substituting
the built-in `qtimlvconverter` preprocessing stage with an application-provided (custom)
preprocessing callback.

Instead of relying on the converter element's internal scale/format-conversion engine, the
application sets `engine=none` on `qtimlvconverter` and connects to its `process` signal.
The callback (`gst_app_preprocess`) manually converts the incoming NV12 video frame to an
8-bit RGB buffer, resizes it to the model's expected input dimensions, and writes it directly
into the tensor buffer consumed by inference. This pattern is useful when a model requires
input preparation that the stock converter does not support (e.g. custom color-space
conversion, non-standard resizing/letterboxing, or format packing).

Detected objects are annotated with bounding boxes and rendered on the output (display, file,
or an application-supplied sink).

##### Pipeline Diagram

![YOLOv8 Object Detection Pipeline](https://mintcdn.com/qimsdk/1m-QFLl35CbpkdgW/sample-application/images/ai-images/custom-preprocess-pipe.png?fit=max\&auto=format\&n=1m-QFLl35CbpkdgW\&q=85\&s=b52836bed79bc584a067edd467b9cd0c)

#### Input and Output Capabilities

##### Input

The video source branch is fully configurable via `--source` / `-s`. Supported inputs
(anything expressible as a GStreamer source bin) include:

| Input type | Example |
| - | - |
| Local video file | `filesrc location=/path/to/video.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12` |
| Camera | A `qtiqmmfsrc` / `libcamerasrc` based source bin producing `video/x-raw,format=NV12` |
| RTSP stream | An `rtspsrc` based source bin decoded down to `video/x-raw,format=NV12` |

The custom preprocessing callback expects an NV12 input frame (it reads the Y/luma and
interleaved U/V planes of the mapped `GstVideoBlit` to build the RGB model input), so any
source branch must ultimately deliver `video/x-raw,format=NV12`.

##### Output

The output branch is selected via `--output` / `-o`:

| Value | Behavior |
| - | - |
| `display` (default) | Renders annotated frames full-screen via `waylandsink`. |
| `video` | Encodes the annotated stream with `v4l2h264enc` and muxes it to `<model-base-path>/media/output.mp4`. |
| `appsink...` (prefix match) | Routes buffers to a named `appsink` (`emit-signals=true`, `sync=false`) for consumption by the surrounding application/process. |

Any other value causes the application to abort with `Unknown sink type`.

#### Sample Model and Label Files

| Item | Default value | Notes |
| - | - | - |
| Model | `yolov8_det_quantized.tflite` | 8-bit quantized TFLite YOLOv8 object-detection model, run through `qtimltflite` with the external QNN HTP delegate. Expected under `<model-base-path>/models/`. |
| Labels | `yolov8.json` | JSON label/class definition consumed by the `yolov8` postprocess module. Expected under `<model-base-path>/labels/`. |
| Postprocess module | `yolov8` | Built-in `qtimlpostprocess` module implementing the YOLOv8 decode (supports mono-, dual-, and triple-tensor output layouts, confidence threshold, NMS). |
| Sample media | `video.mp4` | Default input video, expected under `<model-base-path>/media/`. |

`--model-base-path` must point to a directory containing `models/`, `labels/`, and (for the
default source/output) `media/` subdirectories, e.g.:

```
<model-base-path>/
├── models/
│   └── yolov8_det_quantized.tflite
├── labels/
│   └── yolov8.json
└── media/
    ├── video.mp4
    └── output.mp4              (created when --output=video)
```

If `--model-base-path` is not supplied, it defaults to `$HOME`.

#### Run the Application on the Target Device

1. Copy (or ensure present on the target) the model, label, and sample media files in the
   layout described above.

2. Run with defaults (file source, display output, model-base-path=`$HOME`):

   ```
   gst-custom-preprocess-yolov8
   ```

3. Run with an explicit model base path:

   ```
   gst-custom-preprocess-yolov8 \
     --model-base-path=/opt/yolov8-detect
   ```

4. Run against a custom source and write encoded output to file:

   ```
   gst-custom-preprocess-yolov8 \
     --source="filesrc location=/path/to/video.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12" \
     --model-base-path=/path/to/models \
     --output=video
   ```

5. Stop the application with `Ctrl+C`; the pipeline receives EOS and shuts down cleanly, or
   it stops automatically at end-of-stream for finite file sources.

6. To display available options:
   ```
   gst-custom-preprocess-yolov8 -h
   ```

#### Configurations

Command line options (parsed with `GOption`):

| Option | Short | Type | Default | Description |
| - | - | - | - | - |
| `--source` | `-s` | string | `filesrc location=$HOME/media/video.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12` | GStreamer source pipeline description. |
| `--model-base-path` | | string | `$HOME/` | Base directory expected to contain `models/`, `labels/`, and `media/`. |
| `--output` | `-o` | string | `display` | GStreamer output pipeline description (`display`, `video`, or `appsink...`). |

Fixed, compile-time pipeline configuration (not exposed on the command line):

| Element | Property | Value | Purpose |
| - | - | - | - |
| `qtimlvconverter` (`preprocess`) | `engine` | `none` | Disables built-in conversion so the `process` signal/custom callback path is used. |
| `qtimltflite` (`inference`) | `delegate` | `external` | Runs TFLite inference via an external delegate. |
| `qtimltflite` (`inference`) | `external-delegate-path` | `libQnnTFLiteDelegate.so` | Shared library providing the external delegate implementation. |
| `qtimltflite` (`inference`) | `external-delegate-options` | `QNNExternalDelegate,backend_type=htp;` | Configures the QNN delegate to use the Hexagon Tensor Processor (HTP) backend. |
| `qtimltflite` (`inference`) | `model` | `<model-base-path>/models/yolov8_det_quantized.tflite` | Path to the YOLOv8 object detection model. |
| `qtimlpostprocess` (`postprocess`) | `module` | `yolov8` | Selects the YOLOv8 object-detection postprocess module. |
| `qtimlpostprocess` (`postprocess`) | `labels` | `<model-base-path>/labels/yolov8.json` | Label file for detected classes. |
| `qtimlpostprocess` (`postprocess`) | `results` | `1` | Number of results emitted per inference. |

#### Expected Output

![Expected output](https://mintcdn.com/qimsdk/Q1KUiYOU5DcCFxl7/sample-application/images/ai-images/yolov8-expected-output.png?fit=max\&auto=format\&n=Q1KUiYOU5DcCFxl7\&q=85\&s=824dc57eb08124cd40c7f99dadd7aa67)

#### Pipeline Flow

1. **Source**: The configured source branch decodes/produces NV12 video frames.
2. **Tee**: Each frame is duplicated — one copy goes to the ML preprocessing branch, the
   other is held for later metadata attachment (`qtimetamux`).
3. **Custom preprocess (`qtimlvconverter`, `engine=none`)**: Because no built-in engine is
   configured, the element emits the `process` signal instead of transforming the frame
   itself. The connected callback, `gst_app_preprocess`:
   * Validates the ML tensor contract (`UINT8` type, 4 dimensions, 3 channels).
   * Reads the Y (luma) and interleaved U/V (chroma) planes of the input `GstVideoBlit`.
   * Nearest-neighbor scales the frame to the model's input width/height.
   * Converts each sampled NV12 pixel to 8-bit RGB (`nv12_to_rgb`).
   * Writes the resulting 3-channel RGB image into the output ML tensor buffer.
4. **Inference (`qtimltflite`)**: Runs `yolov8_det_quantized.tflite` through the external QNN
   delegate on the Hexagon HTP backend, producing raw bounding-box/score/class tensors.
5. **Postprocess (`qtimlpostprocess`, `module=yolov8`)**: Decodes the raw tensors into object
   bounding boxes using the `yolov8` module — automatically dispatches to a mono-, dual-, or
   triple-tensor parsing path depending on the model's output layout, applies a confidence
   threshold, and runs non-max suppression. Labels come from `yolov8.json`.
6. **Metadata mux (`qtimetamux`)**: Merges the postprocess results (as GStreamer metadata)
   back onto the original (tee'd) video frame.
7. **Overlay (`qtivoverlay`)**: Draws the bounding boxes/labels on top of the video frame.
8. **Sink**: The annotated stream is sent to the selected output (`display`, `video` file,
   or `appsink`).
9. **Shutdown**: `SIGINT` (Ctrl+C) or an `EOS`/`ERROR` bus message stops the GLib main loop;
   the pipeline transitions to `NULL` and all resources are released.

### Palm Detection with Custom Preprocessing

#### Common Description

`gst-custom-preprocess-palm-detect` is a GStreamer sample application that
demonstrates how to run real-time, palm detection on Qualcomm platforms while
substituting the built-in `qtimlvconverter` preprocessing stage with an
application-provided (custom) preprocessing callback.

Instead of relying on the converter element's internal scale/format-conversion engine, the
application sets `engine=none` on `qtimlvconverter` and connects to its `process` signal.
The callback (`gst_app_preprocess`) manually converts the incoming NV12 video frame to a
float32 RGB buffer, resizes it to the model's expected input dimensions, and
writes it directly into the tensor buffer consumed by inference. This pattern is useful when
a model requires input preparation that the stock converter does not support (e.g. custom
color-space reduction, non-standard resizing/letterboxing, or format packing).

It identifies the presence, location and rotation angle of any hands in the scene and
attaches metadata, visualizes results with bounding boxes and rendered on the output
(display, file, or an application-supplied sink).

##### Pipeline Diagram

![Palm Detection Pipeline](https://mintcdn.com/qimsdk/Q1KUiYOU5DcCFxl7/sample-application/images/ai-images/palm-detect-pipe.png?fit=max\&auto=format\&n=Q1KUiYOU5DcCFxl7\&q=85\&s=9ddf9bdea6fffd13c64dbb5fc621959c)

#### Input and Output Capabilities

##### Input

The video source branch is fully configurable via `--source` / `-s`. Supported inputs
(anything expressible as a GStreamer source bin) include:

| Input type | Example |
| - | - |
| Camera (default) | A `qtiqmmfsrc` / `libcamerasrc` based source bin producing `video/x-raw,format=NV12` |
| Local video file | `filesrc location=/path/to/video.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12` |
| RTSP stream | An `rtspsrc` based source bin decoded down to `video/x-raw,format=NV12` |

The custom preprocessing callback expects an NV12 input frame (it reads only the Y/luma
plane of the mapped `GstVideoBlit` to build the grayscale model input), so any source branch
must ultimately deliver `video/x-raw,format=NV12`.

##### Output

The output branch is selected via `--output` / `-o`:

| Value | Behavior |
| - | - |
| `display` (default) | Renders annotated frames full-screen via `waylandsink`. |
| `video` | Encodes the annotated stream with `v4l2h264enc` and muxes it to `<model-base-path>/media/output.mp4`. |
| `appsink...` (prefix match) | Routes buffers to a named `appsink` (`emit-signals=true`, `sync=false`) for consumption by the surrounding application/process. |

Any other value causes the application to abort with `Unknown sink type`.

#### Sample Model and Label Files

| Item | Default value | Notes |
| - | - | - |
| Model | `palm_detection.tflite` | 8-bit quantized TFLite face-detection model, run through `qtimltflite` with `delegate=gpu`. Expected under `<model-base-path>/models/`. |
| Labels | `palmd_labels.json` | JSON label/class definition consumed by the `palmd` postprocess module. Expected under `<model-base-path>/labels/`. |
| Settings | `palmd_settings.json` | JSON file containing settings for `confidence` and `landmarks` used by  the `palmd` postprocess module. Expected under `<model-base-path>/labels/`. |
| Postprocess module | `palmd` | Built-in `qtimlpostprocess` module implementing the palm-detector decode (score/bbox/landmark tensor parsing, NMS, bbox squaring). |
| Sample media | `access_control_all.mp4` | Default input video, expected under `<model-base-path>/media/`. |

`--model-base-path` must point to a directory containing `models/`, `labels/`, and (for the
default source/output) `media/` subdirectories, e.g.:

```
<model-base-path>/
├── models/
│   └── palm_detection.tflite
├── labels/
│   │── palmd_labels.json
|   └── palmd_settings.json
└── media/
    └── output.mp4              (created when --output=video)
```

If `--model-base-path` is not supplied, it defaults to `$HOME`.

#### Run the Application on the Target Device

1. Copy (or ensure present on the target) the model, label, and sample media files in the
   layout described above.

2. Run with defaults (camera source, display output, model-base-path=`$HOME`):

   ```
   gst-custom-preprocess-palm-detect
   ```

3. Run with an explicit model base path:

   ```
   gst-custom-preprocess-palm-detect \
     --model-base-path=/opt/face-detect
   ```

4. Run against a custom source and write encoded output to file:

   ```
   gst-custom-preprocess-palm-detect \
     --source="filesrc location=/path/to/video.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12" \
     --model-base-path=/path/to/models \
     --output=video
   ```

5. Stop the application with `Ctrl+C`; the pipeline receives EOS and shuts down cleanly, or
   it stops automatically at end-of-stream for finite file sources.

6. To display available options:
   ```
   gst-custom-preprocess-palm-detect -h
   ```

#### Configurations

Command line options (parsed with `GOption`):

| Option | Short | Type | Default | Description |
| - | - | - | - | - |
| `--source` | `-s` | string | `qtiqmmfsrc ! video/x-raw,format=NV12` | GStreamer source pipeline description. |
| `--model-base-path` | | string | `$HOME/` | Base directory expected to contain `models/`, `labels/`, and `media/`. |
| `--output` | `-o` | string | `display` | GStreamer output pipeline description (`display`, `video`, or `appsink...`). |

Fixed, compile-time pipeline configuration (not exposed on the command line):

| Element | Property | Value | Purpose |
| - | - | - | - |
| `qtimlvconverter` (`preprocess`) | `engine` | `none` | Disables built-in conversion so the `process` signal/custom callback path is used. |
| `qtimltflite` (`inference`) | `delegate` | `gpu` | Runs TFLite inference on the GPU delegate. |
| `qtimltflite` (`inference`) | `model` | `<model-base-path>/models/palm_detection.tflite` | Path to the face detection model. |
| `qtimlpostprocess` (`postprocess`) | `module` | `palmd` | Selects the palm-detector postprocess module. |
| `qtimlpostprocess` (`postprocess`) | `labels` | `<model-base-path>/labels/palmd_labels.json` | Label file for detected classes. |
| `qtimlpostprocess` (`postprocess`) | `settings` | `<model-base-path>/labels/palmd_settings.json` | Settings file. |
| `qtimlpostprocess` (`postprocess`) | `results` | `1` | Number of results emitted per inference. |
| `qtimlpostprocess` (`postprocess`) | `bbox-stabilization` | `TRUE` | Smooths bounding boxes across frames. |

#### Expected Output

![Palm Detection expected output](https://mintcdn.com/qimsdk/Q1KUiYOU5DcCFxl7/sample-application/images/ai-images/palmd-expected-output.png?fit=max\&auto=format\&n=Q1KUiYOU5DcCFxl7\&q=85\&s=cff5acbc05f52fc01d788c0f1d950c1f)

#### Pipeline Flow

1. **Source**: The configured source branch decodes/produces NV12 video frames.
2. **Tee**: Each frame is duplicated — one copy goes to the ML preprocessing branch, the
   other is held for later metadata attachment (`qtimetamux`).
3. **Custom preprocess (`qtimlvconverter`, `engine=none`)**: Because no built-in engine is
   configured, the element emits the `process` signal instead of transforming the frame
   itself. The connected callback, `gst_app_preprocess`:
   * Validates the ML tensor contract (`UINT8` type, 4 dimensions, 1 channel).
   * Reads the Y (luma) plane of the input `GstVideoBlit`.
   * Nearest-neighbor scales it to the model's input width/height.
   * Writes the resulting single-channel grayscale image into the output ML tensor buffer.
4. **Inference (`qtimltflite`)**: Runs `palm_detection.tflite` on the GPU delegate,
   producing raw score/bbox/landmark tensors.
5. **Postprocess (`qtimlpostprocess`, `module=palmd`)**: Decodes the raw tensors into face
   bounding boxes using the `palmd` module — applies a confidence threshold, reconstructs
   boxes from center/landmark offsets, squares them with margin, runs non-max suppression,
   and stabilizes boxes across frames. Labels come from `palmd_labels.json`.
6. **Metadata mux (`qtimetamux`)**: Merges the postprocess results (as GStreamer metadata)
   back onto the original (tee'd) video frame.
7. **Overlay (`qtivoverlay`)**: Draws the bounding boxes/labels on top of the video frame.
8. **Sink**: The annotated stream is sent to the selected output (`display`, `video` file,
   or `appsink`).
9. **Shutdown**: `SIGINT` (Ctrl+C) or an `EOS`/`ERROR` bus message stops the GLib main loop;
   the pipeline transitions to `NULL` and all resources are released.

### Lightweight Face Detection with Custom Preprocessing

#### Common Description

`gst-custom-preprocess-lightweight-face-detect` is a GStreamer sample application that
demonstrates how to run real-time, lightweight face detection on Qualcomm platforms while
substituting the built-in `qtimlvconverter` preprocessing stage with an
application-provided (custom) preprocessing callback.

Instead of relying on the converter element's internal scale/format-conversion engine, the
application sets `engine=none` on `qtimlvconverter` and connects to its `process` signal.
The callback (`gst_app_preprocess`) manually converts the incoming NV12 video frame to a
single-channel grayscale buffer, resizes it to the model's expected input dimensions, and
writes it directly into the tensor buffer consumed by inference. This pattern is useful when
a model requires input preparation that the stock converter does not support (e.g. custom
color-space reduction, non-standard resizing/letterboxing, or format packing).

Detected faces are annotated with bounding boxes and rendered on the output (display, file,
or an application-supplied sink).

##### Pipeline Diagram

![Lightweight Face Detection Pipeline](https://mintcdn.com/qimsdk/1m-QFLl35CbpkdgW/sample-application/images/ai-images/custom-preprocess-pipe.png?fit=max\&auto=format\&n=1m-QFLl35CbpkdgW\&q=85\&s=b52836bed79bc584a067edd467b9cd0c)

#### Input and Output Capabilities

##### Input

The video source branch is fully configurable via `--source` / `-s`. Supported inputs
(anything expressible as a GStreamer source bin) include:

| Input type | Example |
| - | - |
| Local video file | `filesrc location=/path/to/video.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12` |
| Camera | A `qtiqmmfsrc` / `libcamerasrc` based source bin producing `video/x-raw,format=NV12` |
| RTSP stream | An `rtspsrc` based source bin decoded down to `video/x-raw,format=NV12` |

The custom preprocessing callback expects an NV12 input frame (it reads only the Y/luma
plane of the mapped `GstVideoBlit` to build the grayscale model input), so any source branch
must ultimately deliver `video/x-raw,format=NV12`.

##### Output

The output branch is selected via `--output` / `-o`:

| Value | Behavior |
| - | - |
| `display` (default) | Renders annotated frames full-screen via `waylandsink`. |
| `video` | Encodes the annotated stream with `v4l2h264enc` and muxes it to `<model-base-path>/media/output.mp4`. |
| `appsink...` (prefix match) | Routes buffers to a named `appsink` (`emit-signals=true`, `sync=false`) for consumption by the surrounding application/process. |

Any other value causes the application to abort with `Unknown sink type`.

#### Sample Model and Label Files

| Item | Default value | Notes |
| - | - | - |
| Model | `face_det_lite_w8a8.tflite` | 8-bit quantized TFLite face-detection model, run through `qtimltflite` with `delegate=gpu`. Expected under `<model-base-path>/models/`. |
| Labels | `qfd-labels.json` | JSON label/class definition consumed by the `qfd` postprocess module. Expected under `<model-base-path>/labels/`. |
| Postprocess module | `qfd` | Built-in `qtimlpostprocess` module implementing the lightweight-face-detector decode (score/bbox/landmark tensor parsing, NMS, bbox squaring). |
| Sample media | `access_control_all.mp4` | Default input video, expected under `<model-base-path>/media/`. |

`--model-base-path` must point to a directory containing `models/`, `labels/`, and (for the
default source/output) `media/` subdirectories, e.g.:

```
<model-base-path>/
├── models/
│   └── face_det_lite_w8a8.tflite
├── labels/
│   └── qfd-labels.json
└── media/
    ├── access_control_all.mp4
    └── output.mp4              (created when --output=video)
```

If `--model-base-path` is not supplied, it defaults to `$HOME`.

#### Run the Application on the Target Device

1. Copy (or ensure present on the target) the model, label, and sample media files in the
   layout described above.

2. Run with defaults (file source, display output, model-base-path=`$HOME`):

   ```
   gst-custom-preprocess-lightweight-face-detect
   ```

3. Run with an explicit model base path:

   ```
   gst-custom-preprocess-lightweight-face-detect \
     --model-base-path=/opt/face-detect
   ```

4. Run against a custom source and write encoded output to file:

   ```
   gst-custom-preprocess-lightweight-face-detect \
     --source="filesrc location=/path/to/video.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12" \
     --model-base-path=/path/to/models \
     --output=video
   ```

5. Stop the application with `Ctrl+C`; the pipeline receives EOS and shuts down cleanly, or
   it stops automatically at end-of-stream for finite file sources.

6. To display available options:
   ```
   gst-custom-preprocess-lightweight-face-detect -h
   ```

#### Configurations

Command line options (parsed with `GOption`):

| Option | Short | Type | Default | Description |
| - | - | - | - | - |
| `--source` | `-s` | string | `filesrc location=$HOME/media/access_control_all.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12` | GStreamer source pipeline description. |
| `--model-base-path` | | string | `$HOME/` | Base directory expected to contain `models/`, `labels/`, and `media/`. |
| `--output` | `-o` | string | `display` | GStreamer output pipeline description (`display`, `video`, or `appsink...`). |

Fixed, compile-time pipeline configuration (not exposed on the command line):

| Element | Property | Value | Purpose |
| - | - | - | - |
| `qtimlvconverter` (`preprocess`) | `engine` | `none` | Disables built-in conversion so the `process` signal/custom callback path is used. |
| `qtimltflite` (`inference`) | `delegate` | `gpu` | Runs TFLite inference on the GPU delegate. |
| `qtimltflite` (`inference`) | `model` | `<model-base-path>/models/face_det_lite_w8a8.tflite` | Path to the face detection model. |
| `qtimlpostprocess` (`postprocess`) | `module` | `qfd` | Selects the lightweight-face-detector postprocess module. |
| `qtimlpostprocess` (`postprocess`) | `labels` | `<model-base-path>/labels/qfd-labels.json` | Label file for detected classes. |
| `qtimlpostprocess` (`postprocess`) | `results` | `1` | Number of results emitted per inference. |
| `qtimlpostprocess` (`postprocess`) | `bbox-stabilization` | `TRUE` | Smooths bounding boxes across frames. |

#### Expected Output

![Lightweight Face Detection expected output](https://mintcdn.com/qimsdk/Q1KUiYOU5DcCFxl7/sample-application/images/ai-images/lightweight-face-detect-expected-output.png?fit=max\&auto=format\&n=Q1KUiYOU5DcCFxl7\&q=85\&s=d218a088c2cd9cbac59fc00e0c0f6c26)

#### Pipeline Flow

1. **Source**: The configured source branch decodes/produces NV12 video frames.
2. **Tee**: Each frame is duplicated — one copy goes to the ML preprocessing branch, the
   other is held for later metadata attachment (`qtimetamux`).
3. **Custom preprocess (`qtimlvconverter`, `engine=none`)**: Because no built-in engine is
   configured, the element emits the `process` signal instead of transforming the frame
   itself. The connected callback, `gst_app_preprocess`:
   * Validates the ML tensor contract (`UINT8` type, 4 dimensions, 1 channel).
   * Reads the Y (luma) plane of the input `GstVideoBlit`.
   * Nearest-neighbor scales it to the model's input width/height.
   * Writes the resulting single-channel grayscale image into the output ML tensor buffer.
4. **Inference (`qtimltflite`)**: Runs `face_det_lite_w8a8.tflite` on the GPU delegate,
   producing raw score/bbox/landmark tensors.
5. **Postprocess (`qtimlpostprocess`, `module=qfd`)**: Decodes the raw tensors into face
   bounding boxes using the `qfd` module — applies a confidence threshold, reconstructs
   boxes from center/landmark offsets, squares them with margin, runs non-max suppression,
   and stabilizes boxes across frames. Labels come from `qfd-labels.json`.
6. **Metadata mux (`qtimetamux`)**: Merges the postprocess results (as GStreamer metadata)
   back onto the original (tee'd) video frame.
7. **Overlay (`qtivoverlay`)**: Draws the bounding boxes/labels on top of the video frame.
8. **Sink**: The annotated stream is sent to the selected output (`display`, `video` file,
   or `appsink`).
9. **Shutdown**: `SIGINT` (Ctrl+C) or an `EOS`/`ERROR` bus message stops the GLib main loop;
   the pipeline transitions to `NULL` and all resources are released.

***

## Troubleshooting

> [accordion] Model not available after running the download script?
>
> If any model isn’t available after downloading the script file, you can download the model manually from [IoT — Qualcomm AI Hub](https://aihub.qualcomm.com)
> and push it to the target device:
>
> ```bash theme={null}
> scp <model filename> root@<ip addr of the target device>:/etc/models
> ```
>
> For example:
>
> ```bash theme={null}
> scp mobilenet_v2_quantized.tflite root@<ip addr of the target device>:/etc/models
> ```

> [accordion] Running sample applications from the UART shell
>
> Remount the file system with read/write permissions:
>
> For Qualcomm Linux:
>
> ```bash theme={null}
> mount -o remount,rw /
> ```

> [accordion] Cannot locate the qticamsrc plugin?
>
> If you cannot locate the qticamsrc plugin, ensure that the camera server is
> running and clear the GStreamer cache using the following commands:
>
> ```bash theme={null}
> ps -ef | grep cam-server 
> ```
>
> ```bash theme={null}
> rm ~/.cache/gstreamer-1.0/registry.aarch64.bin 
> ```

> [accordion] GStreamer logging and debugging
>
> To enable basic GStreamer logging, run the following before launching the
> application:
>
> ```bash theme={null}
> export GST_DEBUG=2 
> ```
>
> To increase verbosity for specific
> plugins, use a comma-separated list with log levels (1–7):
>
> ```bash theme={null}
> export GST_DEBUG=3,qticamsrc:5,qtimlvconverter:5,qtimltflite:5 
> ```
>
> To redirect logs to a file for offline analysis:
>
> ```bash theme={null}
> export GST_DEBUG=3 
> export GST_DEBUG_FILE=/tmp/gst_debug.log 
> ```