Reference Applications - Qualcomm Intelligent Multimedia SDK

Multimedia Pipeline

Before running any of the applications below, create the sample directory tree on the target device:

# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{media,models,labels}"

This is required even for apps that don’t download any media, model, or label files (such as the camera capture and recording apps), since they still write their output under this path.

USB Camera Preview

USB (UVC) cameras are handled by the v4l2src node. Each USB camera comes with its own format and frame constraints. Because of this, we always place qtivcomposer right after the USB camera to ensure that the rest of the pipeline receives a hardware-friendly NV12 (Semi-planar YUV420) video format. However, qtivcomposer will automatically operate in passthrough mode if the USB camera already supports NV12, so you don’t need to worry about any performance overhead.

Some USB webcams (e.g. Logitech Brio 4K) may render a green/garbled frame at 30fps depending on platform/build. If this happens, lower the pipeline’s framerate (e.g. to 15fps) as a workaround.

Try me

Explicit coding style
Implicit coding style
YAML config style

Check application source code on GitHub: qimsdk_ref_usb_camera.pyPre-built application on device: /usr/bin/qimsdk_ref_usb_camera.py

"""USB camera preview using explicit linking."""

import argparse

from qimsdk import Element, Pipeline, VideoFilter

#  Example pipeline:
#
#    source -> transform -> [videofilter] -> display
#
#  The pipeline reads frames from a USB (V4L2) camera, rotates them, restricts
#  the stream to NV12/1080p/30fps, and displays the result through Wayland.


def create_and_execute_pipeline(device: str) -> None:

    # Captures frames from the USB camera source.
    source = (
        Element("v4l2src", "source")
        .set("device", device)
    )

    # Applies geometric transforms to video frames.
    transform = Element("qtivtransform", "transform")

    # Stream filters used in branch links.
    # They define specific stream characteristics from the supported options.
    videofilter = VideoFilter().format("NV12").resolution(1920, 1080).framerate(30)

    # Render video stream on display.
    #
    # sync=false disables strict rendering synchronization to the pipeline clock.
    # fullscreen=true renders the output fullscreen on the target display.
    display = (
        Element("waylandsink", "display")
        .set("sync", False)
        .set("fullscreen", True)
    )

    # Creates the pipeline, adds elements, links them explicitly, and executes it.
    pipeline = (
        Pipeline("usb-cam-pipeline")
        .add(source)
        .add(transform)
        .add_stream_filter("videofilter", videofilter)
        .add(display)
        .link("source", "transform", "videofilter", "display")
    )

    pipeline.execute()


def main() -> None:

    from qimsdk import ImsdkGstLogMode, ImsdkLogLevel, SetImsdkGstLogMode, SetImsdkLogLevel
    SetImsdkGstLogMode(ImsdkGstLogMode.ImsdkLog)
    SetImsdkLogLevel(ImsdkLogLevel.Debug)

    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument(
        "-d", "--device", default="/dev/video2",
        help="V4L2 device node to capture from (default: %(default)s)",
    )
    args = parser.parse_args()

    create_and_execute_pipeline(args.device)


if __name__ == "__main__":
    main()

Connect a USB camera

Connect a USB (UVC) camera to the target device and verify it is exposed as /dev/video2 (update the pipeline’s device property if your camera is exposed on a different node).

Run the application

/usr/bin/qimsdk_ref_usb_camera.py --input-config /dev/video2

Expected Output

The live USB camera feed is rendered fullscreen on the display, rotated and scaled to 1080p/30fps NV12 as configured by the pipeline’s stream filter.

To stop the application, press CTRL + C.

Check application source code on GitHub: qimsdk_ref_usb_camera_impl.pyPre-built application on device: /usr/bin/qimsdk_ref_usb_camera_impl.py

"""USB camera preview using auto-linking."""

import argparse

from qimsdk import Pipeline, VideoFilter

#  Example pipeline:
#
#    source -> transform -> [videofilter] -> display
#
#  The pipeline reads frames from a USB (V4L2) camera, rotates them, restricts
#  the stream to NV12/1080p/30fps, and displays the result through Wayland.


def create_and_execute_pipeline(device: str) -> None:

    # Creates the pipeline, adds and links elements, and executes it.
    #
    # Elements are created on the fly as they are added, and linking is
    # implicit, following the order in which elements are added.
    #
    # sync=false disables strict rendering synchronization to the pipeline clock.
    # fullscreen=true renders the output fullscreen on the target display.
    pipeline = (
        Pipeline("usb-cam-pipeline")
        # Captures frames from the USB camera source.
        .add("v4l2src", "source", "device", device)
        # Applies geometric transforms to video frames.
        .add("qtivtransform", "transform")
        # Stream filter used in the branch link.
        # It defines specific stream characteristics from the supported options.
        .add_stream_filter(
            "videofilter",
            VideoFilter().format("NV12").resolution(1920, 1080).framerate(30),
        )
        # Render video stream on display.
        .add("waylandsink", "display", "sync", False, "fullscreen", True)
    )

    pipeline.execute()


def main() -> None:

    from qimsdk import ImsdkGstLogMode, ImsdkLogLevel, SetImsdkGstLogMode, SetImsdkLogLevel
    SetImsdkGstLogMode(ImsdkGstLogMode.ImsdkLog)
    SetImsdkLogLevel(ImsdkLogLevel.Debug)

    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument(
        "-d", "--device", default="/dev/video2",
        help="V4L2 device node to capture from (default: %(default)s)",
    )
    args = parser.parse_args()

    create_and_execute_pipeline(args.device)


if __name__ == "__main__":
    main()

Connect a USB camera

Connect a USB (UVC) camera to the target device and verify it is exposed as /dev/video2 (update the pipeline’s device property if your camera is exposed on a different node).

Run the application

/usr/bin/qimsdk_ref_usb_camera_impl.py --input-config /dev/video2

Expected Output

The live USB camera feed is rendered fullscreen on the display, rotated and scaled to 1080p/30fps NV12 as configured by the pipeline’s stream filter.

To stop the application, press CTRL + C.

Check application source code on GitHub: qimsdk_ref_usb_camera.yamlPre-built application on device: qimsdk_ref_yml.py

pipeline:
  elements:
    - type: v4l2src
      name: source
      device: /dev/video2

    - type: qtivtransform
      name: transform

    - type: filter
      name: videostream
      video:
        format: NV12
        width: 1920
        height: 1080
        framerate: 30

    - type: waylandsink
      name: display
      sync: false
      fullscreen: true

Connect a USB camera

Connect a USB (UVC) camera to the target device and verify it is exposed as /dev/video2 (update the pipeline’s device property if your camera is exposed on a different node).

Run the application

qimsdk_ref_yml.py /etc/qimsdk/qimsdk_ref_usb_camera.yaml

Expected Output

The live USB camera feed is rendered fullscreen on the display, rotated and scaled to 1080p/30fps NV12 as configured by the YAML pipeline description.

To stop the application, press CTRL + C.

Built-in Camera Preview

This pipeline includes a built-in camera node, a stream filter, and a display node. If the device has more than one built-in camera, you can specify which one to use by providing the camera’s unique ID. The stream filter defines the camera’s output format, resolution, and framerate. You can also configure the display settings, such as enabling fullscreen rendering.

IQ9 has no onboard ISP camera. You can attach an external IMX camera (e.g. from RB3 Gen2) instead — see ISP Camera (Config #2 / qticamsrc) for the procedure to switch from libcamera to qticamsrc.

Try me

Explicit coding style
Implicit coding style

Check application source code on GitHub: qimsdk_ref_camera.pyPre-built application on device: /usr/bin/qimsdk_ref_camera.py

#!/usr/bin/env python3

"""Basic camera preview using explicit linking."""

from qimsdk import Element, Pipeline, VideoFilter

#  Example pipeline:
#
#    source -> [videofilter] -> display
#
#  The pipeline reads camera frames and displays the result through Wayland.


def create_and_execute_pipeline() -> None:

    # Captures frames from the camera source.
    source = Element("qtiqmmfsrc", "source")

    # Stream filters used in branch links.
    # They define specific stream characteristics from the supported options.
    videofilter = VideoFilter().format("NV12").resolution(1920, 1080).framerate(30)

    # Render video stream on display.
    #
    # async=false enforce state transition to ensure the buffers are returned on time.
    # sync=false disables strict rendering synchronization to the pipeline clock.
    # fullscreen=true renders the output fullscreen on the target display.
    display = (
        Element("waylandsink", "display")
        .set("sync", False)
        .set("fullscreen", True)
    )

    # Creates the pipeline, adds elements, links them explicitly, and executes it.
    pipeline = (
        Pipeline("cam-pipeline")
        .add(source)
        .add_stream_filter("videofilter", videofilter)
        .add(display)
        .link("source", "videofilter", "display")
    )

    pipeline.execute()


def main() -> None:

    from qimsdk import ImsdkGstLogMode, ImsdkLogLevel, SetImsdkGstLogMode, SetImsdkLogLevel
    SetImsdkGstLogMode(ImsdkGstLogMode.ImsdkLog)
    SetImsdkLogLevel(ImsdkLogLevel.Debug)

    create_and_execute_pipeline()


if __name__ == "__main__":
    main()

Check application source code on GitHub: qimsdk_ref_camera_impl.pyPre-built application on device: /usr/bin/qimsdk_ref_camera_impl.py

#!/usr/bin/env python3

"""Basic camera preview using auto-linking."""

from qimsdk import Pipeline, VideoFilter

#  Example pipeline:
#
#    source -> [videofilter] -> display
#
#  The pipeline reads camera frames and displays the result through Wayland.


def create_and_execute_pipeline() -> None:

    # Creates the pipeline, adds and links elements, and executes it.
    #
    # Elements are created on the fly as they are added, and linking is
    # implicit, following the order in which elements are added.
    #
    # async=false enforce state transition to ensure the buffers are returned on time.
    # sync=false disables strict rendering synchronization to the pipeline clock.
    # fullscreen=true renders the output fullscreen on the target display.
    pipeline = (
        Pipeline("cam-pipeline")
        # Captures frames from the camera source.
        .add("qtiqmmfsrc", "source")
        # Stream filter used in the branch link.
        # It defines specific stream characteristics from the supported options.
        .add_stream_filter(
            "videofilter",
            VideoFilter().format("NV12").resolution(1920, 1080).framerate(30),
        )
        # Render video stream on display.
        .add("waylandsink", "display", "sync", False, "fullscreen", True)
    )

    pipeline.execute()


def main() -> None:

    from qimsdk import ImsdkGstLogMode, ImsdkLogLevel, SetImsdkGstLogMode, SetImsdkLogLevel
    SetImsdkGstLogMode(ImsdkGstLogMode.ImsdkLog)
    SetImsdkLogLevel(ImsdkLogLevel.Debug)

    create_and_execute_pipeline()


if __name__ == "__main__":
    main()

Run the application

python3 /usr/bin/qimsdk_ref_camera.py

Expected Output

The live camera feed is rendered fullscreen on the display at 1080p/30fps NV12 as configured by the pipeline’s stream filter.To stop the application, press CTRL + C.

Built-in Camera Preview with Still Capture

The application captures a still image a few seconds after starting and then stops automatically — this is expected behavior, not a hang.

Try me

Check application source code on GitHub: qimsdk_ref_camera_and_capture.pyPre-built application on device: /usr/bin/qimsdk_ref_camera_and_capture.py

#!/usr/bin/env python3

"""Camera preview + still image capture using CamSrc and explicit links."""

import time
import os

from qimsdk import CamSrc, Element, ImageFilter, Pipeline, VideoFilter

#  Example pipeline:
#
#    source -> [vf] -> display
#    source -> [if] -> imagesink
#
#  The pipeline reads camera frames, displays them through Wayland, and on
#  request captures a still image to a JPEG file.


def create_and_execute_pipeline() -> None:

    # Captures frames from the camera source.
    source = CamSrc("source")

    # Restricts the preview stream to NV12/1080p/30fps before display.
    vf = VideoFilter().format("NV12").resolution(1920, 1080).framerate(30)

    # Render video stream on display.
    display = (
        Element("waylandsink", "display")
        .set("sync", False)
        .set("async", False)
        .set("fullscreen", True)
    )

    # Restricts the still capture stream to a 4K JPEG format.
    imagefilter = ImageFilter().format("JPEG").resolution(3840, 2160)

    # Writes output buffers to files.
    imagesink = (
        Element("multifilesink", "imagesink")
        .set("enable-last-sample", False)
        .set("location", f"{os.environ['HOME']}/Downloads/qimsdk_samples/media/image_%d.jpeg")
    )

    # Creates the pipeline, adds and links elements, and executes it.
    #
    # Explicit linking is applied.
    pipeline = (
        Pipeline("cam-capture-pipeline")
        .add(source)
        .add_stream_filter("vf", vf)
        .add(display)
        .add_stream_filter("if", imagefilter)
        .add(imagesink)
        .link("source", "vf", "display")
        .link("source", "if", "imagesink")
    )

    pipeline.start()
    time.sleep(2)

    # image capture trigger
    cam = pipeline.get("source", CamSrc)
    cam.image_capture()

    time.sleep(2)
    pipeline.stop().wait()


def main() -> None:

    if "HOME" not in os.environ:
        raise EnvironmentError("Error: HOME environment variable is not set.")

    from qimsdk import ImsdkGstLogMode, ImsdkLogLevel, SetImsdkGstLogMode, SetImsdkLogLevel
    SetImsdkGstLogMode(ImsdkGstLogMode.ImsdkLog)
    SetImsdkLogLevel(ImsdkLogLevel.Debug)

    create_and_execute_pipeline()


if __name__ == "__main__":
    main()

Run the application

python3 /usr/bin/qimsdk_ref_camera_and_capture.py

The application displays a live preview and captures a still JPEG image a few seconds after starting, saving it under ~/media/image_%d.jpeg.

Expected Output

The live camera preview is rendered briefly on the display, and a still JPEG image is saved to ~/Downloads/qimsdk_samples/media/image_%d.jpeg. The application then exits automatically.

Camera Recording to MP4

This pipeline captures frames from the built-in camera, encodes them into an H.264 stream using the hardware encoder, parses the bitstream, muxes it into an MP4 container, and writes the resulting file to disk.

Try me

Check application source code on GitHub: qimsdk_ref_camera_encoder.pyPre-built application on device: /usr/bin/qimsdk_ref_camera_encoder.py

#!/usr/bin/env python3

"""Camera preview + still image capture using CamSrc and explicit links."""

import time
import os

from qimsdk import CamSrc, Element, Pipeline, VideoFilter

#  Example pipeline:
#
#    source -> [vf] -> encoder -> parser -> muxer -> sink
#
#  The pipeline reads camera frames, encodes them with H.264, muxes into MP4,
#  and writes the output to disk.


def create_and_execute_pipeline() -> None:

    # Captures frames from the camera source.
    source = CamSrc("source")

    # Stream filters used in branch links.
    # They define specific stream characteristics from the supported options.
    vf = VideoFilter().format("NV12").resolution(1920, 1080).framerate(30)

    # Encodes raw video frames into H.264 stream.
    encoder = (
        Element("v4l2h264enc", "encoder")
        .set("output-io-mode", "dmabuf-import")
        .set("capture-io-mode", "dmabuf")
    )

    # Parses H.264 bitstream for downstream muxing.
    parser = Element("h264parse", "parser")

    # Muxes encoded stream into MP4 container.
    muxer = Element("mp4mux", "muxer")

    # Writes output stream to a file.
    sink = (
        Element("filesink", "sink")
        .set("location", f"{os.environ['HOME']}/Downloads/qimsdk_samples/media/encoder_output.mp4")
    )

    # Creates the pipeline, adds and links elements, and executes it.
    #
    # Explicit linking is applied.
    pipeline = (
        Pipeline("cam-encoder-pipeline")
        .add(source)
        .add_stream_filter("vf", vf)
        .add(encoder)
        .add(parser)
        .add(muxer)
        .add(sink)
    )

    pipeline.eos(True).execute()


def main() -> None:

    if "HOME" not in os.environ:
        raise EnvironmentError("Error: HOME environment variable is not set.")

    from qimsdk import ImsdkGstLogMode, ImsdkLogLevel, SetImsdkGstLogMode, SetImsdkLogLevel
    SetImsdkGstLogMode(ImsdkGstLogMode.ImsdkLog)
    SetImsdkLogLevel(ImsdkLogLevel.Debug)

    create_and_execute_pipeline()


if __name__ == "__main__":
    main()

Run the application

python3 /usr/bin/qimsdk_ref_camera_encoder.py

The application records camera frames encoded as H.264 into an MP4 file saved to ~/Downloads/qimsdk_samples/media/encoder_output.mp4.

Expected Output

The application records silently with no display output. An MP4 file is written to ~/Downloads/qimsdk_samples/media/encoder_output.mp4, which you can copy off the device and play back to verify the recording.To stop the application, press CTRL + C.

MP4 File Playback

Try me

Check application source code on GitHub: qimsdk_ref_qtdemux_decode_display.pyPre-built application on device: /usr/bin/qimsdk_ref_qtdemux_decode_display.py

#!/usr/bin/env python3

"""Transport stream demux sample that exercises H264Filter."""

from qimsdk import Element, Pipeline, VideoFilter
import os

#  Example pipeline:
#
#    src -> demux -> parse -> decoder -> [videofilter] -> display
#
#  The pipeline reads an MP4/H.264 file, decodes it through the hardware decoder,
#  and displays the result through Wayland.


def create_and_execute_pipeline() -> None:

    # Reads the input media file as raw bytes.
    src = (
        Element("filesrc", "src")
        .set("location", f"{os.environ['HOME']}/Downloads/qimsdk_samples/media/ai_demo_sample.mp4")
    )

    # Extracts elementary streams from the MP4 container.
    demux = Element("qtdemux", "demux")

    # Prepares the H.264 bitstream for the decoder.
    parse = Element("h264parse", "parse")

    # Decodes the compressed H.264 stream into raw video frames.
    #
    # The I/O mode is configured to enforce DMA buffer usage,
    # avoiding unnecessary buffer copies.
    decoder = (
        Element("v4l2h264dec", "decoder")
        .set("output-io-mode", 4)
        .set("capture-io-mode", 4)
    )

    # Restricts the decoded stream to NV12 before it reaches the display.
    videofilter = VideoFilter().format("NV12")

    # Render video stream on display.
    display = (
        Element("waylandsink", "display")
        .set("fullscreen", True)
    )

    # Creates the pipeline, adds and links elements, and executes it.
    #
    # Linking is implicit and follows the order in which elements are added.
    pipeline = (
        Pipeline("video-pipeline")
        .add(src)
        .add(demux)
        .add(parse)
        .add(decoder)
        .add_stream_filter("videofilter", videofilter)
        .add(display)
    )

    pipeline.execute()


def main() -> None:

    if "HOME" not in os.environ:
        raise EnvironmentError("Error: HOME environment variable is not set.")

    from qimsdk import ImsdkGstLogMode, ImsdkLogLevel, SetImsdkGstLogMode, SetImsdkLogLevel
    SetImsdkGstLogMode(ImsdkGstLogMode.ImsdkLog)
    SetImsdkLogLevel(ImsdkLogLevel.Debug)

    create_and_execute_pipeline()


if __name__ == "__main__":
    main()

Download Required Files

File	Download	Save as
Sample video	ai_demo_sample.mp4	`ai_demo_sample.mp4`

You can also use your own MP4 (H.264) file, or update the pipeline’s location property to point to it.

Copy the video file to the device

# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/media"
scp ai_demo_sample.mp4 <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_qtdemux_decode_display.py

The application decodes the video file and renders it fullscreen on the display.

Expected Output

The MP4 file’s video track is decoded and rendered fullscreen on the display until the end of the file is reached.

To stop the application, press CTRL + C.

Transport Stream (.ts) File Playback

This pipeline is similar to the Offline Video Source pipeline, but ingests an MPEG transport stream (.ts) file instead of an MP4 container. It demultiplexes the transport stream using tsdemux to extract the video track, decodes the compressed H.264 stream into raw frames, converts (or directly negotiates) the output to NV12, and renders the frames to the display.

Try me

Check application source code on GitHub: qimsdk_ref_tsdemux_decode_display.pyPre-built application on device: /usr/bin/qimsdk_ref_tsdemux_decode_display.py

#!/usr/bin/env python3

"""Transport stream demux sample that exercises H264Filter."""

from qimsdk import Element, H264Filter, Pipeline, VideoFilter
import os

#  Example pipeline:
#
#    src -> demux -> [vf1] -> parse -> decoder -> [vf2] -> display
#
#  The pipeline reads a sequence of transport-stream files, extracts the H.264
#  elementary stream, decodes it through the hardware decoder, and displays
#  the result through Wayland.


def create_and_execute_pipeline() -> None:

    # Reads a sequence of input files as stream data.
    src = (
        Element("multifilesrc", "src")
        .set("location", f"{os.environ['HOME']}/Downloads/qimsdk_samples/media/ai_demo_sample.ts")
        .set("stop-index", 0)
    )

    # Extracts elementary streams from the transport stream.
    demux = Element("tsdemux", "demux")

    # Restricts the extracted stream to a fixed framerate before parsing.
    vf1 = H264Filter().framerate(30)

    # Prepares the H.264 bitstream for the decoder.
    parse = Element("h264parse", "parse")

    # Decodes the compressed H.264 stream into raw video frames.
    #
    # The I/O mode is configured to enforce DMA buffer usage,
    # avoiding unnecessary buffer copies.
    decoder = (
        Element("v4l2h264dec", "decoder")
        .set("output-io-mode", 4)
        .set("capture-io-mode", 4)
    )

    # Restricts the decoded stream to NV12 before it reaches the display.
    vf2 = VideoFilter().format("NV12")

    # Render video stream on display.
    display = (
        Element("waylandsink", "display")
        .set("fullscreen", True)
    )

    # Creates the pipeline, adds and links elements, and executes it.
    #
    # Linking is implicit and follows the order in which elements are added.
    pipeline = (
        Pipeline("video-pipeline")
        .add(src)
        .add(demux)
        .add_stream_filter("vf1", vf1)
        .add(parse)
        .add(decoder)
        .add_stream_filter("vf2", vf2)
        .add(display)
    )

    pipeline.execute()


def main() -> None:

    if "HOME" not in os.environ:
        raise EnvironmentError("Error: HOME environment variable is not set.")

    from qimsdk import ImsdkGstLogMode, ImsdkLogLevel, SetImsdkGstLogMode, SetImsdkLogLevel
    SetImsdkGstLogMode(ImsdkGstLogMode.ImsdkLog)
    SetImsdkLogLevel(ImsdkLogLevel.Debug)

    create_and_execute_pipeline()


if __name__ == "__main__":
    main()

Download Required Files

File	Download	Save as
Sample transport stream	ai_demo_sample.ts	`ai_demo_sample.ts`

Copy the transport stream file to the device

# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/media"
scp ai_demo_sample.ts <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_tsdemux_decode_display.py

The application decodes the transport stream file and renders it fullscreen on the display.

Expected Output

The transport stream file’s video track is decoded and rendered fullscreen on the display until the end of the file is reached.

To stop the application, press CTRL + C.

Synthetic Test Pattern (VideoFilter API)

This pipeline generates synthetic test video frames using videotestsrc (a bouncing ball pattern), instead of reading from a camera or file. The frames are restricted to NV12/1080p/30fps, rotated 180 degrees by qtivtransform, downscaled to 1280x720 by a second stream filter, and rendered to the display. This is a useful pipeline for testing display and transform stages without any camera or media file dependency.

Try me

Check application source code on GitHub: qimsdk_ref_videotestsrc_videofilter_display.pyPre-built application on device: /usr/bin/qimsdk_ref_videotestsrc_videofilter_display.py

#!/usr/bin/env python3

"""Test video test source with stream filter."""

from qimsdk import Element, Pipeline, VideoFilter

#  Example pipeline:
#
#    src -> [vf1] -> transform -> [vf2] -> display
#
#  The pipeline generates synthetic test video frames, restricts them to NV12
#  at 1920x1080/30fps, rotates the frames, restricts the output to 1280x720,
#  and displays the result through Wayland.


def create_and_execute_pipeline() -> None:

    # Generates synthetic test video frames.
    src = (
        Element("videotestsrc", "src")
        .set("pattern", "ball")
    )

    # Restricts the generated stream to NV12 at 1920x1080, 30fps.
    vf1 = VideoFilter().format("NV12").resolution(1920, 1080).framerate(30)

    # Applies geometric transforms to video frames.
    transform = (
        Element("qtivtransform", "transform")
        .set("rotate", "180")
    )

    # Restricts the transformed stream to 1280x720.
    vf2 = VideoFilter().resolution(1280, 720)

    # Render video stream on display.
    display = (
        Element("waylandsink", "display")
        .set("fullscreen", True)
    )

    # Creates the pipeline, adds and links elements, and executes it.
    #
    # Linking is implicit and follows the order in which elements are added.
    pipeline = (
        Pipeline("video-pipeline")
        .add(src)
        .add_stream_filter("vf1", vf1)
        .add(transform)
        .add_stream_filter("vf2", vf2)
        .add(display)
    )

    pipeline.execute()


def main() -> None:

    from qimsdk import ImsdkGstLogMode, ImsdkLogLevel, SetImsdkGstLogMode, SetImsdkLogLevel
    SetImsdkGstLogMode(ImsdkGstLogMode.ImsdkLog)
    SetImsdkLogLevel(ImsdkLogLevel.Debug)

    create_and_execute_pipeline()


if __name__ == "__main__":
    main()

Run the application

python3 /usr/bin/qimsdk_ref_videotestsrc_videofilter_display.py

The application generates a synthetic bouncing-ball test pattern, rotates and rescales it, and renders it fullscreen on the display.

Expected Output

A synthetic bouncing-ball test pattern is rendered fullscreen on the display, rotated 180 degrees and rescaled to 1280x720 — with no camera or media file required.

To stop the application, press CTRL + C.

Synthetic Test Pattern (Raw Caps StreamFilter)

This pipeline is functionally identical to the previous example, but it defines its stream constraints using raw GStreamer caps strings via StreamFilter (e.g. "video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1"), instead of the structured VideoFilter builder API. This is useful when you need to express filter capabilities that aren’t covered by the structured builder, or when porting caps strings directly from existing GStreamer pipelines.

Try me

Check application source code on GitHub: qimsdk_ref_videotestsrc_streamfilter_display.pyPre-built application on device: /usr/bin/qimsdk_ref_videotestsrc_streamfilter_display.py

#!/usr/bin/env python3

"""Test video test source with stream filter."""

from qimsdk import Element, Pipeline, StreamFilter

#  Example pipeline:
#
#    src -> [vf1] -> transform -> [vf2] -> display
#
#  The pipeline generates synthetic test video frames, applies a caps-string-based
#  stream filter, rotates the frames, applies a second caps-string-based stream
#  filter, and displays the result through Wayland.


def create_and_execute_pipeline() -> None:

    # Generates synthetic test video frames.
    src = (
        Element("videotestsrc", "src")
        .set("pattern", "ball")
    )

    # Restricts the generated stream to NV12 at 1920x1080, 30fps.
    vf1 = StreamFilter("video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1")

    # Applies geometric transforms to video frames.
    transform = (
        Element("qtivtransform", "transform")
        .set("rotate", "180")
    )

    # Restricts the transformed stream to 1280x720.
    vf2 = StreamFilter("video/x-raw,width=1280,height=720")

    # Render video stream on display.
    display = (
        Element("waylandsink", "display")
        .set("fullscreen", True)
    )

    # Creates the pipeline, adds and links elements, and executes it.
    #
    # Linking is implicit and follows the order in which elements are added.
    pipeline = (
        Pipeline("video-pipeline-streamfilter-string")
        .add(src)
        .add_stream_filter("vf1", vf1)
        .add(transform)
        .add_stream_filter("vf2", vf2)
        .add(display)
    )

    pipeline.execute()


def main() -> None:

    from qimsdk import ImsdkGstLogMode, ImsdkLogLevel, SetImsdkGstLogMode, SetImsdkLogLevel
    SetImsdkGstLogMode(ImsdkGstLogMode.ImsdkLog)
    SetImsdkLogLevel(ImsdkLogLevel.Debug)

    create_and_execute_pipeline()


if __name__ == "__main__":
    main()

Run the application

python3 /usr/bin/qimsdk_ref_videotestsrc_streamfilter_display.py

The application generates a synthetic bouncing-ball test pattern, rotates and rescales it, and renders it fullscreen on the display.

Expected Output

A synthetic bouncing-ball test pattern is rendered fullscreen on the display, rotated 180 degrees and rescaled to 1280x720, identical in appearance to the VideoFilter API example above.

To stop the application, press CTRL + C.

RTSP Camera Stream Playback

This pipeline connects to a network RTSP camera stream instead of a local (USB/built-in) camera or file. rtspsrc establishes the RTSP session and negotiates the H.264 RTP stream, a caps filter restricts it to H.264 video, rtph264depay extracts the H.264 payload from the RTP packets, h264parse prepares the bitstream for the hardware decoder, and the decoded frames are converted to NV12 and rendered to the display.

Try me

See the full YAML configuration here: qimsdk_ref_rtsp_camera.yamlCheck application source code on GitHub: qimsdk_ref_yml.pyPre-built application on device: qimsdk_ref_yml.pyYAML configuration saved on device: /etc/qimsdk/qimsdk_ref_rtsp_camera.yaml

pipeline:
  elements:
    - type: rtspsrc
      name: source
      location: rtsp://username:password@192.168.1.188:554/Streaming/Channels/101
      latency: 0

    - type: capsfilter
      name: rtp_caps
      caps: application/x-rtp,media=video,encoding-name=H264

    - type: rtph264depay
      name: depay

    - type: h264parse
      name: parse

    - type: v4l2h264dec
      name: decoder
      capture-io-mode: 4
      output-io-mode: 4

    - type: filter
      name: videostream
      video:
        format: NV12

    - type: waylandsink
      name: display
      sync: false
      fullscreen: true

Update the RTSP camera credentials

On the device, edit /etc/qimsdk/qimsdk_ref_rtsp_camera.yaml and update the location property with your RTSP camera’s username, password, IP address, and stream path.

Run the application

qimsdk_ref_yml.py /etc/qimsdk/qimsdk_ref_rtsp_camera.yaml

The application connects to the RTSP camera, decodes the H.264 stream, and renders it fullscreen on the display.

Expected Output

The RTSP camera’s live H.264 stream is decoded and rendered fullscreen on the display, for as long as the network stream remains available.To stop the application, press CTRL + C.

AI Pipeline

Video-to-Tensor Dump (Offline Preprocessing)

This pipeline ingests an MP4 file, demultiplexes it to extract the video track, decodes the compressed stream (e.g., H.264/H.265) into raw frames, converts (or directly negotiates) the output to NV12 — a hardware-friendly YUV 4:2:0 semi-planar format — and then goes through qtimlvconverter (ML preprocessing). The TensorFilter stream filter describes the tensor format and shape as UINT8 with dimensions [1, 520, 520, 3] (batch, height, width, channels). Finally, multifilesink writes the preprocessed tensors out as sequential .rgb files to ~/Downloads/qimsdk_samples/media/tensor_520_520_%d.rgb, rolling after every 10 files.

Try me

Check application source code on GitHub: qimsdk_ref_tensor_dump.pyPre-built application on device: /usr/bin/qimsdk_ref_tensor_dump.py

Download Required Files

File	Download	Save as
Sample video	ai_demo_sample.mp4	`ai_demo_sample.mp4`

You can also use your own MP4 (H.264) file, or update the pipeline’s location property to point to it.

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/media"
scp ai_demo_sample.mp4 <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_tensor_dump.py

The application writes preprocessed tensors to ~/Downloads/qimsdk_samples/media/tensor_520_520_%d.rgb.

Expected Output

There is no display output. Sequential .rgb tensor files are written to ~/Downloads/qimsdk_samples/media/, rolling after every 10 files, which you can inspect offline for use in other ML pipelines.To stop the application, press CTRL + C.

Live Camera YOLOv8 Detection (Manual Chain)

This pipeline captures live video from the on-device camera, then drives a full detection-and-visualization loop in real time: the raw NV12 stream is normalized and preprocessed into one or more tensors, fed to a TFLite inference stage (running on the external QNN delegate for efficient hardware execution), and the resulting tensors are postprocessed into semantic detections (classes, boxes, confidences). Those results are converted into QIM SDK-friendly metadata and attached back to the original video timeline, so the visual stream and its ML annotations stay synchronized. A parallel branch from the early tee ensures the original frames are available to a metadata muxer, which merges the detections with the video. Finally, the overlay stage renders the boxes/labels directly on the live frames, and the composed output is presented fullscreen to the display. In short, the graph turns a live camera feed into actionable on-screen insights by (1) creating a tensor view of the stream, (2) running detection, (3) translating inference outputs into QIM SDK metadata, (4) fusing metadata with the original frames, and (5) drawing the results on top of the video for low-latency, on-screen visualization.

Try me

Explicit coding style
YAML config style

Check application source code on GitHub: qimsdk_ref_camera_yolov8.pyPre-built application on device: /usr/bin/qimsdk_ref_camera_yolov8.py

Download Required Files

File	Save as
Model	yolov8_det_quantized.tflite
yolov8.json	yolov8.json

If the downloaded model file is a .zip archive, extract it on your host machine before copying: unzip filename.zipSome AI Hub models require running the AI Hub export/optimize step for your target runtime before the downloaded file is usable — download alone isn’t always sufficient.

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels}"
scp yolov8_det_quantized.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp yolov8.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/

Run the application

python3 /usr/bin/qimsdk_ref_camera_yolov8.py

The application overlays detected bounding boxes and labels on the live camera feed and renders it fullscreen on the display.

Expected Output

The live camera feed is rendered fullscreen on the display with YOLOv8 detection bounding boxes and class labels overlaid on each frame in real time.

To stop the application, press CTRL + C.

See the full YAML configuration here: qimsdk_ref_camera_yolov8.yamlCheck application source code on GitHub: qimsdk_ref_yml.pyPre-built application on device: qimsdk_ref_yml.pyYAML configuration saved on device: /etc/qimsdk/qimsdk_ref_camera_yolov8.yaml

pipeline:
  elements:
    - type: qtiqmmfsrc
      name: source
      camera: 0

    - type: filter
      name: videostream
      video:
        format: NV12
        width: 1920
        height: 1080
        framerate: 30

    - type: tee
      name: split

    - type: queue
      name: q1

    - type: qtimlvconverter
      name: preprocessing

    - type: queue
      name: q3

    - type: qtimltflite
      name: inferencing
      delegate: external
      external-delegate-path: libQnnTFLiteDelegate.so
      external-delegate-options: "QNNExternalDelegate,backend_type=htp;"
      model: ~/Downloads/qimsdk_samples/models/yolov8_det_quantized.tflite

    - type: queue
      name: q4

    - type: qtimlpostprocess
      name: postprocessing
      results: 5
      module: yolov8
      labels: ~/Downloads/qimsdk_samples/labels/yolov8.json
      settings: '{"confidence": 70.0}'

    - type: filter
      name: mlf
      text: {}

    - type: qtimetamux
      name: mlmuxer

    - type: queue
      name: q5

    - type: qtivoverlay
      name: overlay

    - type: waylandsink
      name: display
      sync: false
      fullscreen: true

  links:
    - [split, mlmuxer]
    - [source, videostream, split, q1, preprocessing, q3, inferencing, q4, postprocessing, mlf, mlmuxer, q5, overlay, display]

Download Required Files

File	Save as
Model	yolov8_det_quantized.tflite
yolov8.json	yolov8.json

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels}"
scp yolov8_det_quantized.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp yolov8.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/

Run the application

qimsdk_ref_yml.py /etc/qimsdk/qimsdk_ref_camera_yolov8.yaml

Expected Output

The live camera feed is rendered fullscreen on the display with YOLOv8 detection bounding boxes and class labels overlaid on each frame in real time, identical to the explicit coding style example above.

To stop the application, press CTRL + C.

Live Camera YOLOv8 Detection (ML Bin)

This solution is identical to the previous one. The only difference is that here we use qtimlvideotflitebin instead of manually linking the preprocessing, inference, postprocessing, and ML muxer components, which simplifies the pipeline.

Try me

Explicit coding style
YAML config style

Check application source code on GitHub: qimsdk_ref_camera_yolov8_mlbin.pyPre-built application on device: /usr/bin/qimsdk_ref_camera_yolov8_mlbin.py

Download Required Files

File	Save as
Model	yolov8_det_quantized.tflite
yolov8.json	yolov8.json

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels}"
scp yolov8_det_quantized.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp yolov8.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/

Run the application

python3 /usr/bin/qimsdk_ref_camera_yolov8_mlbin.py

The application overlays detected bounding boxes and labels on the live camera feed and renders it fullscreen on the display.

Expected Output

The live camera feed is rendered fullscreen on the display with YOLOv8 detection bounding boxes and class labels overlaid on each frame in real time.

To stop the application, press CTRL + C.

See the full YAML configuration here: qimsdk_ref_mlbin_yolov8.yamlCheck application source code on GitHub: qimsdk_ref_yml.pyPre-built application on device: qimsdk_ref_yml.pyYAML configuration saved on device: /etc/qimsdk/qimsdk_ref_mlbin_yolov8.yaml

pipeline:
  elements:
    - type: filesrc
      name: src
      location: ~/Downloads/qimsdk_samples/media/ai_demo_sample.mp4

    - type: qtdemux
      name: demux

    - type: h264parse
      name: parse

    - type: v4l2h264dec
      name: decoder
      output-io-mode: 4
      capture-io-mode: 4

    - type: filter
      name: videofilter
      video:
        format: NV12

    - type: qtimlvideotflitebin
      name: mlbin
      inference-delegate: "external"
      inference-external-delegate-path: "libQnnTFLiteDelegate.so"
      inference-external-delegate-options: "QNNExternalDelegate,backend_type=htp;"
      inference-model: "~/Downloads/qimsdk_samples/models/yolov8_det_quantized.tflite"
      postprocess-module: "yolov8"
      postprocess-labels: "~/Downloads/qimsdk_samples/labels/yolov8.json"

    - type: qtivoverlay
      name: overlay

    - type: waylandsink
      name: display
      fullscreen: true

Download Required Files

File	Save as
Model	yolov8_det_quantized.tflite
yolov8.json	yolov8.json

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels}"
scp yolov8_det_quantized.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp yolov8.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/

Run the application

qimsdk_ref_yml.py /etc/qimsdk/qimsdk_ref_mlbin_yolov8.yaml

The application overlays detected bounding boxes and labels on the live camera feed and renders it fullscreen on the display.

Expected Output

The live camera feed is rendered fullscreen on the display with YOLOv8 detection bounding boxes and class labels overlaid on each frame in real time, produced by the all-in-one ML bin instead of a manually chained pipeline.

To stop the application, press CTRL + C.

Offline Video YOLOv8 Detection (Manual Chain, Metamux Overlay)

This pipeline is the offline-video counterpart to Live Camera YOLOv8 Detection (Manual Chain): instead of a live camera feed, an MP4 file is demuxed and decoded, then split by a tee into a display branch and an ML branch. The ML branch runs preprocessing, inference, and postprocessing manually, and the resulting detections are attached back onto the original frame via qtimetamux before being overlaid and rendered. This Metamux-based approach is more flexible than the video composer approach (it allows chaining additional ML models or streaming metadata separately), whereas the composer overlay approach used in the next example offers better performance.

Try me

Check application source code on GitHub: qimsdk_ref_yolov8_metamux_overlay.pyPre-built application on device: /usr/bin/qimsdk_ref_yolov8_metamux_overlay.py

Download Required Files

File	Save as
Model	yolov8_det_quantized.tflite
yolov8.json	yolov8.json
ai_demo_sample.mp4	ai_demo_sample.mp4

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp yolov8_det_quantized.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp yolov8.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp ai_demo_sample.mp4          <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_yolov8_metamux_overlay.py

The application decodes the video file, runs YOLOv8 detection, attaches the detections to the original frame via qtimetamux, and overlays the results, rendering them fullscreen.

Expected Output

The video plays back with YOLOv8 bounding boxes and class labels overlaid on each detected object, rendered fullscreen on the display.

To stop the application, press CTRL + C.

Customize application

Python applications run directly from source, so no build or compilation step is required. Edit the script directly and rerun it to apply your changes.

Offline Video YOLOv8 Detection (ML Bin, Composer Overlay)

The AI output can be overlaid directly on top of the frame instead of attaching AI metadata to the main frame. This approach provides better performance, whereas the Metamux solution is more flexible and can be used for chaining AI models or streaming AI metadata separately.

Try me

Check application source code on GitHub: qimsdk_ref_yolov8_composer_overlay.pyPre-built application on device: /usr/bin/qimsdk_ref_yolov8_composer_overlay.py

Download Required Files

File	Save as
Model	yolov8_det_quantized.tflite
yolov8.json	yolov8.json
ai_demo_sample.mp4	ai_demo_sample.mp4

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp yolov8_det_quantized.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp yolov8.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp ai_demo_sample.mp4          <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_yolov8_composer_overlay.py

The application composes the AI detection overlay directly on top of the decoded video frames and displays it fullscreen.

Expected Output

The video plays back with YOLOv8 bounding boxes and class labels composited directly onto each frame, rendered fullscreen on the display.

To stop the application, press CTRL + C.

Customize application

Python applications run directly from source, so no build or compilation step is required. Edit the script directly and rerun it to apply your changes.

Gesture Recognition

This pipeline demonstrates a daisy-chained, multi-model gesture recognition graph built entirely from standard SDK elements (qtimlvconverter, qtimltflite, qtimlpostprocess, qtimetamux, qtimetatransform) rather than a custom postprocessing callback. Palm detection locates the hand region in the live camera frame; that region is transformed and fed into hand landmark detection; the resulting landmarks are passed through a gesture embedder followed by a canned gesture classifier to decode the recognized gesture. Two parallel qtimetamux merge points combine metadata from each stage back onto the original frame before the final overlay is rendered to the display.

Try me

Explicit coding style
YAML config style

Check application source code on GitHub: qimsdk_ref_gesture_recognition.pyPre-built application on device: /usr/bin/qimsdk_ref_gesture_recognition.py

See the full YAML configuration here: qimsdk_ref_gesture_recognition.yamlCheck application source code on GitHub: qimsdk_ref_yml.pyPre-built application on device: qimsdk_ref_yml.pyYAML configuration saved on device: /etc/qimsdk/qimsdk_ref_gesture_recognition.yaml

Download Required Files

Download the gesture recognizer task bundle from Google MediaPipe to obtain the palm detection, hand landmark, gesture embedder, and canned gesture classifier models:

# Download the gesture recognizer task bundle
wget https://storage.googleapis.com/mediapipe-models/gesture_recognizer/gesture_recognizer/float16/latest/gesture_recognizer.task

# Extract the top-level task
unzip gesture_recognizer.task

# Extract hand landmarker models
unzip hand_landmarker.task
# → hand_detector.tflite, hand_landmarks_detector.tflite

# Extract gesture recognizer models
unzip hand_gesture_recognizer.task
# → gesture_embedder.tflite, canned_gesture_classifier.tflite

File	Save as
hand_detector.tflite (see steps above)	palm_detection_full.tflite
hand_landmarks_detector.tflite (see steps above)	hand_landmark_full.tflite
gesture_embedder.tflite (see steps above)	gesture_embedder.tflite
canned_gesture_classifier.tflite (see steps above)	canned_gesture_classifier.tflite
palmd_labels.json	palmd_labels.json
palmd_settings.json	palmd_settings.json
hlandmarks.json	hlandmarks.json
hlandmark_settings.json	hlandmark_settings.json
gesture_rec.json	gesture_rec.json

If a downloaded model file is a .zip archive, extract it on your host machine before copying: unzip filename.zip

Copy Files to Device

This pipeline’s source is the on-device camera, so no video file needs to be copied.

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels}"
scp palm_detection_full.tflite       <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp hand_landmark_full.tflite        <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp gesture_embedder.tflite          <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp canned_gesture_classifier.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp palmd_labels.json                <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp palmd_settings.json              <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp hlandmarks.json                  <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp hlandmark_settings.json          <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp gesture_rec.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/

Run the application

python3 /usr/bin/qimsdk_ref_gesture_recognition.py

The application captures live video from the on-device camera, runs the daisy-chained palm detection → hand landmark → gesture embedder → canned gesture classifier models, and overlays the recognized gesture on the display fullscreen.

Expected Output

The live camera feed is rendered fullscreen on the display with the recognized gesture label overlaid on each frame in real time.

To stop the application, press CTRL + C.

PPE & Person/Foot Detection (ML Bin)

This pipeline demonstrates a two-model ML bin pipeline for personal protective equipment (PPE) and person/foot detection: foot_track_net detects persons and their feet in the frame, while gear_guard_net detects PPE gear. Both models run through qtimlvideotflitebin, which internally handles preprocessing, inference, and postprocessing, simplifying the pipeline compared to manually chaining each stage.

Try me

Explicit coding style
YAML config style

Check application source code on GitHub: qimsdk_ref_mlbin_ppe.pyPre-built application on device: /usr/bin/qimsdk_ref_mlbin_ppe.py

See the full YAML configuration here: qimsdk_ref_mlbin_ppe.yamlCheck application source code on GitHub: qimsdk_ref_yml.pyPre-built application on device: qimsdk_ref_yml.pyYAML configuration saved on device: /etc/qimsdk/qimsdk_ref_mlbin_ppe.yaml

Download Required Files

File	Save as
Model	gear_guard_net-ppe-detection-w8a8.tflite
Model	foot_track_net-person-foot-detection-w8a8.tflite
gear_guard_net.json	gear_guard_net.json
foot_track_net.json	foot_track_net.json
foot_track_net_settings.json	foot_track_net_settings.json
ppe_sample.mp4	ppe_sample.mp4

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp gear_guard_net-ppe-detection-w8a8.tflite      <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp foot_track_net-person-foot-detection-w8a8.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp gear_guard_net.json                           <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp foot_track_net.json                           <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp foot_track_net_settings.json                  <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp ppe_sample.mp4                                <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_mlbin_ppe.py

The application runs inference on the video file, detecting persons/feet with foot_track_net and PPE gear with gear_guard_net, overlaying the resulting detections on the video and rendering it fullscreen.

Expected Output

The video plays back with person/foot and PPE-gear detection bounding boxes overlaid, rendered fullscreen on the display.

To stop the application, press CTRL + C.

YOLOv8 Detection (ML Bin)

This pipeline reads an MP4/H.264 file, decodes it through the hardware decoder, and runs YOLOv8 object detection using qtimlvideotflitebin, which internally handles preprocessing, inference, and postprocessing. The detected objects are then overlaid on the video and displayed.

Try me

Explicit coding style
YAML config style

Check application source code on GitHub: qimsdk_ref_mlbin_yolov8.pyPre-built application on device: /usr/bin/qimsdk_ref_mlbin_yolov8.py

Download Required Files

File	Save as
Model	yolov8_det_quantized.tflite
yolov8.json	yolov8.json
ai_demo_sample.mp4	ai_demo_sample.mp4

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp yolov8_det_quantized.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp yolov8.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp ai_demo_sample.mp4          <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_mlbin_yolov8.py

The application decodes the video file, runs YOLOv8 detection through qtimlvideotflitebin, and overlays the detected objects on the video, rendering it fullscreen.

Expected Output

The video plays back with YOLOv8 bounding boxes and class labels overlaid on each detected object, rendered fullscreen on the display.

To stop the application, press CTRL + C.

Customize application

Python applications run directly from source, so no build or compilation step is required. Edit the script directly and rerun it to apply your changes.

AppSink and AppSrc

AppSink and AppSrc are special elements because they expose buffers directly to and from the application. Because of this, we create and configure these elements separately, using their dedicated wrapper classes. Once configured, we insert them into the pipeline as fully constructed element objects.

Try me

Check application source code on GitHub: qimsdk_ref_appsrc_and_appsink.pyPre-built application on device: /usr/bin/qimsdk_ref_appsrc_and_appsink.py

Run the application

python3 /usr/bin/qimsdk_ref_appsrc_and_appsink.py

The application generates synthetic test frames, pushes them through an AppSink/AppSrc bridge, and renders the resulting stream fullscreen on the display.

Expected Output

The synthetic test frames pushed through the AppSink/AppSrc bridge are rendered fullscreen on the display, confirming the buffers passed through application code unmodified.To stop the application, press CTRL + C.

Custom Pre and Post Processing

Custom Preprocessing for YOLOv8 (Detection)

QIM SDK plugin support different backend: GLES, OpenCV, etc. This level of interface could be exposed to application. This API takes all input and output at once in case of batching or daisy for example. It can be simplified by calling application implementation for every input. Advantage of this approach is that all complexity of batching, depth, daisy chain support remains hidden in the plugin and application will do only pre-processing on a single frame/tensor at once.

Try me

Check application source code on GitHub: qimsdk_ref_external_preprocess_detection_yolov8.pyPre-built application on device: /usr/bin/qimsdk_ref_external_preprocess_detection_yolov8.py

Download Required Files

File	Save as
Model	yolov8_det_quantized.tflite
yolov8.json	yolov8.json
ai_demo_sample.mp4	ai_demo_sample.mp4

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp yolov8_det_quantized.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp yolov8.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp ai_demo_sample.mp4          <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_external_preprocess_detection_yolov8.py

Expected Output

The video plays back with YOLOv8 bounding boxes and class labels overlaid on each detected object, rendered fullscreen on the display.

To stop the application, press CTRL + C.

Customize application

Python applications run directly from source, so no build or compilation step is required. Edit the script directly and rerun it to apply your changes.

Custom Preprocessing for YOLOv8 (ML Bin variant)

This example demonstrates external preprocessing support inside MLVideoTFLiteBin. Registering a callback via MLVideoTFLiteBin.set_preprocess_handler(...) disables the bin’s internal preprocessing path (the SDK sets preprocess-engine=none on the bin’s internal mlpreprocess element automatically) and hands frame-to-tensor conversion off to an application-defined Python callback. The callback receives the decoded NV12 blit(s) as NumPy-backed buffers and writes directly into the bin’s output tensor (a writable, in-place NumPy view), converting each pixel to the quantized int8 NHWC layout expected by the YOLOv8 model, with letterbox padding for any area the blit doesn’t cover. Inference and postprocessing then run inside the same bin, and the detected objects are overlaid on the video and displayed.

Try me

Check application source code on GitHub: qimsdk_ref_external_preprocess_detection_mlbin_yolov8.pyPre-built application on device: /usr/bin/qimsdk_ref_external_preprocess_detection_mlbin_yolov8.py

Download Required Files

File	Save as
Model	yolov8_det_quantized.tflite
yolov8.json	yolov8.json
ai_demo_sample.mp4	ai_demo_sample.mp4

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp yolov8_det_quantized.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp yolov8.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp ai_demo_sample.mp4          <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_external_preprocess_detection_mlbin_yolov8.py

The application decodes the video file, runs the custom, application-defined Python preprocessing callback inside mlbin to convert NV12 frames into the quantized int8 tensor expected by YOLOv8, then runs inference and postprocessing, overlaying the detected objects and rendering the result fullscreen.

Expected Output

The video plays back with YOLOv8 bounding boxes and class labels overlaid on each detected object, rendered fullscreen on the display.

To stop the application, press CTRL + C.

Customize application

Python applications run directly from source, so no build or compilation step is required. Edit the script directly and rerun it to apply your changes.

Custom Preprocessing with Lightweight Face Detection

This example demonstrates external preprocessing with MLVConverter (rather than MLVideoTFLiteBin), so the pipeline links preprocessing, inference, and postprocessing manually through explicit queues, and merges the detections back onto the original frame via qtimetamux before overlaying. Setting engine to "none" on the MLVConverter element disables its internal preprocessing path and hands frame-to-tensor conversion off to an application-defined callback registered via MLVConverter.set_handler(...). Unlike the YOLOv8 example, the face_det_lite_w8a8.tflite model consumes raw uint8 grayscale, so the callback resizes only the NV12 luma (Y) plane into a single-channel [1, H, W, 1] NumPy tensor, skipping color conversion and normalization entirely and leaving the chroma plane unread. A tee right after decoding splits the stream into a display branch and an ML branch, and a queue on each branch (including the display one) keeps the two branches from stalling each other on a shared thread.

Try me

Check application source code on GitHub: qimsdk_ref_external_preprocess_lightweight_face_detect.pyPre-built application on device: /usr/bin/qimsdk_ref_external_preprocess_lightweight_face_detect.py

Download Required Files

File	Save as
Model	face_det_lite_w8a8.tflite
qfd-labels.json	qfd-labels.json
ai_demo_sample.mp4	face_sample.mp4

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp face_det_lite_w8a8.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp qfd-labels.json            <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp face_sample.mp4            <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_external_preprocess_lightweight_face_detect.py

The application decodes the video file, runs the custom, application-defined Python preprocessing callback to convert the NV12 luma plane into the grayscale tensor expected by the lightweight face detector, then runs inference and postprocessing, merging the detected faces onto the original frame via qtimetamux, overlaying them, and rendering the result fullscreen.

Expected Output

The video plays back with face-detection bounding boxes decoded by the custom grayscale preprocessing callback, overlaid and rendered fullscreen on the display.To stop the application, press CTRL + C.

Custom Preprocessing with Palm Detection

This example is the live-camera counterpart to the lightweight face detection pipeline: CamSrc captures frames directly instead of decoding a file, but the branching structure is otherwise identical — a tee right after the stream filter splits the pipeline into a display branch and an ML branch, each guarded by its own queue, and qtimetamux merges the ML branch’s detections back onto the original frame before overlay. Setting engine to "none" on the MLVConverter element disables its internal preprocessing path and hands frame-to-tensor conversion off to an application-defined callback registered via MLVConverter.set_handler(...). Unlike the grayscale face-detection example, palm_detection_full.tflite expects a [1, H, W, 3] float32 RGB tensor normalized to [0, 1], so the callback converts the full NV12 image (both luma and chroma planes) to RGB using NumPy and scales each channel into that range, with letterbox padding for any area the blit doesn’t cover.

Try me

Check application source code on GitHub: qimsdk_ref_external_preprocess_palm_detect.pyPre-built application on device: /usr/bin/qimsdk_ref_external_preprocess_palm_detect.py

Download Required Files

Download the gesture recognizer task bundle from Google MediaPipe to obtain the palm detection model:

# Download the gesture recognizer task bundle
wget https://storage.googleapis.com/mediapipe-models/gesture_recognizer/gesture_recognizer/float16/latest/gesture_recognizer.task

# Extract the top-level task
unzip gesture_recognizer.task

# Extract hand landmarker models
unzip hand_landmarker.task
# → hand_detector.tflite, hand_landmarks_detector.tflite

File	Save as
hand_detector.tflite (see steps above)	palm_detection_full.tflite
palmd_labels.json	palmd_labels.json
palmd_settings.json	palmd_settings.json

If a downloaded model file is a .zip archive, extract it on your host machine before copying: unzip filename.zip

Copy Files to Device

This pipeline’s source is the on-device camera, so no video file needs to be copied.

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels}"
scp palm_detection_full.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp palmd_labels.json          <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp palmd_settings.json        <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/

Run the application

python3 /usr/bin/qimsdk_ref_external_preprocess_palm_detect.py

The application captures live video from the on-device camera, runs the custom, application-defined Python preprocessing callback to convert NV12 frames into the normalized float32 RGB tensor expected by the palm detector, then runs inference and postprocessing, merging the detected palms onto the original frame via qtimetamux, overlaying them, and rendering the result fullscreen.

Expected Output

The live camera feed is rendered fullscreen on the display with palm-detection bounding boxes decoded by the custom float32 RGB preprocessing callback overlaid on each frame in real time.

To stop the application, press CTRL + C.

Custom Postprocessing with YOLOv5 (Detection)

This example demonstrates the object detection postprocessing callback (ObjectDetectionPostprocessCallback) using a YOLOv5 model. Preprocessing and inference are handled by the SDK, and the raw output tensors are decoded into bounding boxes, class names, and confidence scores by an application-defined callback, which are then overlaid on the video and displayed.

Try me

Check application source code on GitHub: qimsdk_ref_external_postprocess_detection_yolov5.pyPre-built application on device: /usr/bin/qimsdk_ref_external_postprocess_detection_yolov5.py

Download Required Files

File	Save as
Model	yolov5m-320x320-int8.tflite
yolov5m.json	yolov5m.json
ai_demo_sample.mp4	ai_demo_sample.mp4

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp yolov5m-320x320-int8.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp yolov5m.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp ai_demo_sample.mp4           <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_external_postprocess_detection_yolov5.py

The application runs inference on the video file and uses the custom, application-defined postprocessing callback to decode detections, which are then overlaid and displayed fullscreen.

Expected Output

The video plays back with YOLOv5 bounding boxes and class labels decoded by the custom postprocessing callback, overlaid and rendered fullscreen on the display.

To stop the application, press CTRL + C.

Customize application

Python applications run directly from source, so no build or compilation step is required. Edit the script directly and rerun it to apply your changes.

Custom Post processing with ML Bin

Try me

Check application source code on GitHub: qimsdk_ref_external_postprocess_mlbin_yolov8.pyPre-built application on device: /usr/bin/qimsdk_ref_external_postprocess_mlbin_yolov8.py

Download Required Files

File	Save as
Model	yolov8_det_quantized.tflite
yolov8.json	yolov8.json
ai_demo_sample.mp4	ai_demo_sample.mp4

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp yolov8_det_quantized.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp yolov8.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp ai_demo_sample.mp4          <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_external_postprocess_mlbin_yolov8.py

The application runs preprocessing, inference, and the custom, application-defined postprocessing callback all inside mlbin, overlaying the resulting detections on the video and rendering it fullscreen.

Expected Output

The video plays back with YOLOv8 bounding boxes and class labels decoded by the custom postprocessing callback, overlaid and rendered fullscreen on the display.

To stop the application, press CTRL + C.

Customize application

Python applications run directly from source, so no build or compilation step is required. Edit the script directly and rerun it to apply your changes.

Custom Postprocessing with ResNet101 (Classification)

This example demonstrates the classification postprocessing callback (ClassificationPostprocessCallback). Preprocessing and inference are handled by the SDK, and the raw output tensors from a ResNet101 model are decoded into labeled classification results (name and confidence) by an application-defined callback.

Try me

Check application source code on GitHub: qimsdk_ref_external_postprocess_classification_resnet101.pyPre-built application on device: /usr/bin/qimsdk_ref_external_postprocess_classification_resnet101.py

Download Required Files

File	Save as
Model	Resnet101_Quantized.tflite
resnet101.json	resnet101.json
ai_demo_sample.mp4	classification_sample.mp4

No ideal sample video ships with the SDK for this use case. Supply your own content for best results.

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp Resnet101_Quantized.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp resnet101.json             <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp classification_sample.mp4  <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_external_postprocess_classification_resnet101.py

The application runs inference on the ResNet101 model and uses the custom, application-defined postprocessing callback to decode classification results, which are then displayed.

Expected Output

The video plays back with the top classification result and its confidence score decoded by the custom postprocessing callback, displayed as text overlaid on the frame.To stop the application, press CTRL + C.

Custom Postprocessing with MiDaS v2 (Depth Estimation)

This example demonstrates the depth estimation postprocessing callback (DepthEstimationPostprocessCallback). Preprocessing and inference are handled by the SDK, and the raw output tensors from a MiDaS v2 model are decoded by an application-defined callback into a per-pixel depth map (values and pseudo-color) for downstream visualization.

Try me

Check application source code on GitHub: qimsdk_ref_external_postprocess_depth_estimation_midasv2.pyPre-built application on device: /usr/bin/qimsdk_ref_external_postprocess_depth_estimation_midasv2.py

Download Required Files

File	Save as
Model	midas-tflite-w8a8.tflite
midas-v2-labels.json	midas-v2-labels.json
ai_demo_sample.mp4	ai_demo_sample.mp4

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp midas-tflite-w8a8.tflite  <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp midas-v2-labels.json      <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp ai_demo_sample.mp4        <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_external_postprocess_depth_estimation_midasv2.py

The application runs inference on the MiDaS v2 model and uses the custom, application-defined postprocessing callback to decode the depth map, which is then rendered.

Expected Output

The video plays back with a pseudo-colored depth map decoded by the custom postprocessing callback, rendered fullscreen on the display in place of the original frame.To stop the application, press CTRL + C.

Custom Postprocessing with Gesture Recognition

This example demonstrates gesture recognition using a daisy-chained, multi-model pipeline: palm detection locates the hand region, hand landmark detection extracts keypoints from that region, and a gesture embedder followed by a canned gesture classifier decode the landmarks into a recognized gesture. Preprocessing and inference are handled by the SDK, and the raw output tensors are decoded by an application-defined postprocessing callback. The pipeline uses the on-device camera as its source.

Try me

Check application source code on GitHub: qimsdk_ref_external_postprocess_gesture_recognition.pyPre-built application on device: /usr/bin/qimsdk_ref_external_postprocess_gesture_recognition.py

Download Required Files

Download the gesture recognizer task bundle from Google MediaPipe to obtain the palm detection, hand landmark, gesture embedder, and canned gesture classifier models:

# Download the gesture recognizer task bundle
wget https://storage.googleapis.com/mediapipe-models/gesture_recognizer/gesture_recognizer/float16/latest/gesture_recognizer.task

# Extract the top-level task
unzip gesture_recognizer.task

# Extract hand landmarker models
unzip hand_landmarker.task
# → hand_detector.tflite, hand_landmarks_detector.tflite

# Extract gesture recognizer models
unzip hand_gesture_recognizer.task
# → gesture_embedder.tflite, canned_gesture_classifier.tflite

File	Save as
hand_detector.tflite (see steps above)	palm_detection_full.tflite
hand_landmarks_detector.tflite (see steps above)	hand_landmark_full.tflite
gesture_embedder.tflite (see steps above)	gesture_embedder.tflite
canned_gesture_classifier.tflite (see steps above)	canned_gesture_classifier.tflite
palmd_labels.json	palmd_labels.json
hlandmarks.json	hlandmarks.json
gesture_rec.json	gesture_rec.json

If a downloaded model file is a .zip archive, extract it on your host machine before copying: unzip filename.zip

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels}"
scp palm_detection_full.tflite       <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp hand_landmark_full.tflite        <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp gesture_embedder.tflite          <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp canned_gesture_classifier.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp palmd_labels.json                <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp hlandmarks.json                  <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp gesture_rec.json                 <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/

Run the application

python3 /usr/bin/qimsdk_ref_external_postprocess_gesture_recognition.py

The application captures live video from the on-device camera, runs the daisy-chained palm detection → hand landmark → gesture embedder → canned gesture classifier models, and uses the custom, application-defined postprocessing callback to decode the recognized gesture, which is then overlaid and displayed fullscreen.

Expected Output

The live camera feed is rendered fullscreen on the display with the recognized gesture label, decoded by the custom postprocessing callback, overlaid on each frame in real time.

To stop the application, press CTRL + C.

Custom Postprocessing with HRNet (Pose Estimation)

This example demonstrates the pose estimation postprocessing callback (PoseEstimationPostprocessCallback) using a two-stage pipeline: a YOLOv5 model detects persons in the frame, and an HRNet model estimates keypoints for each detected person. Preprocessing and inference are handled by the SDK, and the raw output tensors are decoded into keypoints and skeleton links by an application-defined callback, which are then overlaid on the video and displayed.

Try me

Check application source code on GitHub: qimsdk_ref_external_postprocess_pose_estimation_hrnet.pyPre-built application on device: /usr/bin/qimsdk_ref_external_postprocess_pose_estimation_hrnet.py

Download Required Files

File	Save as
Model	yolov5m-320x320-int8.tflite
Model	hrnet_pose_w8a8.tflite
yolov5m.json	yolov5m.json
hrnet.json	hrnet.json
ai_demo_sample.mp4	pose_sample.mp4

No ideal sample video ships with the SDK for this use case. Supply your own content for best results.

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp yolov5m-320x320-int8.tflite <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp hrnet_pose_w8a8.tflite      <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp yolov5m.json                <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp hrnet.json                  <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp pose_sample.mp4             <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_external_postprocess_pose_estimation_hrnet.py

The application runs inference on the video file, first detecting persons with YOLOv5 and then estimating pose keypoints with HRNet, using the custom, application-defined postprocessing callback to decode the results, which are then overlaid and displayed fullscreen.

Expected Output

The video plays back with skeleton keypoints and links decoded by the custom postprocessing callback, overlaid on each detected person and rendered fullscreen on the display.To stop the application, press CTRL + C.

Custom Postprocessing with DeepLabv3 (Segmentation)

This example demonstrates the segmentation postprocessing callback (SegmentationPostprocessCallback) using a DeepLabv3 model. Preprocessing and inference are handled by the SDK, and the raw output tensors are decoded into per-pixel semantic labels and colors by an application-defined callback, which are then overlaid on the video and displayed.

Try me

Check application source code on GitHub: qimsdk_ref_external_postprocess_segmentation_deeplab.pyPre-built application on device: /usr/bin/qimsdk_ref_external_postprocess_segmentation_deeplab.py

Download Required Files

File	Save as
Model	dv3_argmax_int32.tflite
dv3-argmax-labels.json	dv3-argmax-labels.json
ai_demo_sample.mp4	ai_demo_sample.mp4

Copy Files to Device

# Replace <user> and <device-ip> with your device credentials.
# ~ resolves automatically to the correct home directory on the device (/root on QLI, /home/ubuntu on Ubuntu).

ssh <user>@<device-ip> "mkdir -p ~/Downloads/qimsdk_samples/{models,labels,media}"
scp dv3_argmax_int32.tflite    <user>@<device-ip>:~/Downloads/qimsdk_samples/models/
scp dv3-argmax-labels.json     <user>@<device-ip>:~/Downloads/qimsdk_samples/labels/
scp ai_demo_sample.mp4         <user>@<device-ip>:~/Downloads/qimsdk_samples/media/

Run the application

python3 /usr/bin/qimsdk_ref_external_postprocess_segmentation_deeplab.py

The application runs inference on the video file, segmenting the frame with DeepLabv3, and uses the custom, application-defined postprocessing callback to decode the per-pixel segmentation results, which are then overlaid and displayed fullscreen.

Expected Output

The video plays back with a semantic segmentation color mask decoded by the custom postprocessing callback, overlaid on each frame and rendered fullscreen on the display.To stop the application, press CTRL + C.

​Multimedia Pipeline

​USB Camera Preview

​Built-in Camera Preview

​Built-in Camera Preview with Still Capture

​Camera Recording to MP4

​MP4 File Playback

​Transport Stream (.ts) File Playback

​Synthetic Test Pattern (VideoFilter API)

​Synthetic Test Pattern (Raw Caps StreamFilter)

​RTSP Camera Stream Playback

​AI Pipeline

​Video-to-Tensor Dump (Offline Preprocessing)

​Live Camera YOLOv8 Detection (Manual Chain)

​Download Required Files

​Download Required Files

​Live Camera YOLOv8 Detection (ML Bin)

​Download Required Files

​Download Required Files

​Offline Video YOLOv8 Detection (Manual Chain, Metamux Overlay)

​Download Required Files

​Offline Video YOLOv8 Detection (ML Bin, Composer Overlay)

​Download Required Files

​Gesture Recognition

​Download Required Files

​PPE & Person/Foot Detection (ML Bin)

​Download Required Files

​YOLOv8 Detection (ML Bin)

​Download Required Files

​AppSink and AppSrc

​Custom Pre and Post Processing

​Custom Preprocessing for YOLOv8 (Detection)

​Download Required Files

​Custom Preprocessing for YOLOv8 (ML Bin variant)

​Download Required Files

​Custom Preprocessing with Lightweight Face Detection

​Download Required Files

​Custom Preprocessing with Palm Detection

​Download Required Files

​Custom Postprocessing with YOLOv5 (Detection)

​Download Required Files

​Custom Post processing with ML Bin

​Download Required Files

​Custom Postprocessing with ResNet101 (Classification)

​Download Required Files

​Custom Postprocessing with MiDaS v2 (Depth Estimation)

​Download Required Files

​Custom Postprocessing with Gesture Recognition

​Download Required Files

​Custom Postprocessing with HRNet (Pose Estimation)

​Download Required Files

​Custom Postprocessing with DeepLabv3 (Segmentation)

​Download Required Files

Multimedia Pipeline

USB Camera Preview

Built-in Camera Preview

Built-in Camera Preview with Still Capture

Camera Recording to MP4

MP4 File Playback

Transport Stream (.ts) File Playback

Synthetic Test Pattern (VideoFilter API)

Synthetic Test Pattern (Raw Caps StreamFilter)

RTSP Camera Stream Playback

AI Pipeline

Video-to-Tensor Dump (Offline Preprocessing)

Live Camera YOLOv8 Detection (Manual Chain)

Download Required Files

Download Required Files

Live Camera YOLOv8 Detection (ML Bin)

Download Required Files

Download Required Files

Offline Video YOLOv8 Detection (Manual Chain, Metamux Overlay)

Download Required Files

Offline Video YOLOv8 Detection (ML Bin, Composer Overlay)

Download Required Files

Gesture Recognition

Download Required Files

PPE & Person/Foot Detection (ML Bin)

Download Required Files

YOLOv8 Detection (ML Bin)

Download Required Files

AppSink and AppSrc

Custom Pre and Post Processing

Custom Preprocessing for YOLOv8 (Detection)

Download Required Files

Custom Preprocessing for YOLOv8 (ML Bin variant)

Download Required Files

Custom Preprocessing with Lightweight Face Detection

Download Required Files

Custom Preprocessing with Palm Detection

Download Required Files

Custom Postprocessing with YOLOv5 (Detection)

Download Required Files

Custom Post processing with ML Bin

Download Required Files

Custom Postprocessing with ResNet101 (Classification)

Download Required Files

Custom Postprocessing with MiDaS v2 (Depth Estimation)

Download Required Files

Custom Postprocessing with Gesture Recognition

Download Required Files

Custom Postprocessing with HRNet (Pose Estimation)

Download Required Files

Custom Postprocessing with DeepLabv3 (Segmentation)

Download Required Files