Skip to main content

​
Why developers should use the Qualcomm Intelligent Multimedia SDK

Single, Unified Multimedia + AI Framework

Combines multimedia pipelines and ML inference into a single, coherent developer experience.

Hardware-Accelerated by Default

QIM SDK natively exploits QCOM accelerators including CPU, GPU, NPU, ISP, VPU and DSP through optimized plugins

Gstreamer-Based, Industry-Standard Pipelines

Built on Gstreamer, providing a mature, composable pipeline model with 40+ Qualcomm-optimized plugins.

Flexible AI Model Support

SDK supports running models from common frameworks such as TFLite/LiteRT, ONNX, and QNN.

Qualcomm AI Hub Integration

Browse, download, and deploy hundreds of pre-trained, quantized models directly from Qualcomm AI Hub. Models arrive ready to run — no manual conversion, calibration, or profiling required.

​
How QIM SDK Delivers These Benefits

What developers get
  • One pipeline for capture, preprocess, inference, overlay, encode, and streaming.
  • One metadata flow for pixels and AI results.
  • Less custom glue code between multimedia and ML stages.
How QIM SDK delivers it
  • Builds on GStreamer, so media and AI elements live in the same pipeline graph.
  • Carries detections, labels, keypoints, and tensor info as frame metadata.
  • Supports sequential flows like detection → classification and parallel flows using tee + qtimetamux.
  • Supports cross-pipeline designs with qtisocketsink and qtisocketsrc when applications need modular stages.
Developer takeaway You describe the full application as a pipeline instead of stitching together separate camera, inference, and rendering subsystems.
What developers get
  • Lower latency.
  • Lower CPU load.
  • Better throughput for multi-stream and real-time AI.
How QIM SDK delivers it
  • ISP: camera ingest and imaging path use Qualcomm imaging hardware.
  • GPU: plugins such as qtimlvconverter, qtivtransform, qtivoverlay, and qtivcomposer offload resize, color convert, crop, rotate, overlay, and composition work.
  • NPU: inference workloads run through supported runtimes on dedicated AI hardware.
  • VPU: v4l2h264dec, v4l2h265dec, v4l2h264enc, and v4l2h265enc handle video decode/encode efficiently.
  • Zero-copy + buffer pools: frames move through the pipeline as DMA-buf handles with fewer copies and lower allocation overhead.
Developer takeaway Most performance-critical stages are expressed by choosing the right plugins, not by writing accelerator-specific application code.
What developers get
  • Hundreds of pre-trained, profiled, quantized models ready to deploy.
  • No manual conversion, calibration, or accuracy validation.
  • Models arrive as .tflite, .dlc, or .bin files compatible with IM SDK runtimes.
How QIM SDK delivers it
  • Models from AI Hub are exported targeting specific Qualcomm SoC backends (HTP, GPU, CPU).
  • qtimltflite, qtimlsnpe, and qtimlqnn load these models directly — no format adaptation layer.
  • Confidence thresholds, post-processing modules, and label files are supplied alongside the model, matching the IM SDK qtimlpostprocess interface.
Developer takeaway Browse AI Hub, download a model, drop it into the pipeline — no custom integration work required.
What developers get
  • Frames never copied through CPU memory between accelerator stages.
  • Lower peak memory bandwidth utilization.
  • Reduced end-to-end pipeline latency, especially at 1080p and above.
How QIM SDK delivers it
  • Plugins negotiate DMA-buf allocation through GStreamer’s allocation query mechanism.
  • ISP (qticamsrc) → GPU (qtimlvconverter, qtivtransform) → NPU (qtimltflite) → display (waylandsink) all share the same buffer handle.
  • The capture-io-mode=dmabuf and output-io-mode=dmabuf-import properties on decode/encode plugins enable hardware-to-hardware buffer passing.
Developer takeaway Zero-copy is opt-in via plugin properties, not a framework limitation — the pipeline negotiates it automatically when all stages support it.
What developers get
  • Built-in modules for detection, classification, segmentation, pose, audio, and super-resolution.
  • Custom module support without SDK recompilation.
  • Runtime module swap — change post-processing logic by updating a .so file on the device.
How QIM SDK delivers it
  • qtimlpostprocess loads libml-postprocess-<name>.so at runtime via dlopen.
  • Each module implements Caps(), Configure(), and Process() from the IModule interface.
  • Modules are deployed to /usr/lib/gstreamer-1.0/ml/modules/ — no pipeline changes required after deployment.
Developer takeaway Swap or extend post-processing without touching the pipeline definition or recompiling the application.
What developers get
  • Hardware-demosaiced, ISP-processed frames delivered directly to downstream AI and multimedia stages.
  • Multi-camera support (primary + secondary sensor).
  • Zero-copy from ISP to GPU/NPU via GBM/DMA-buf.
How QIM SDK delivers it
  • qticamsrc exposes a GStreamer source interface over the Qualcomm camera service.
  • Outputs video/x-raw,format=NV12 — directly consumable by qtimlvconverter, qtivtransform, and encode plugins.
  • Camera properties (resolution, framerate, camera index) are set via standard GStreamer element properties.
Developer takeaway A single pipeline element replaces the entire camera driver integration layer — no V4L2 or camera HAL code to manage.
What developers get
  • Familiar pipeline construction.
  • Easier integration with upstream plugins.
  • Better portability of application logic.
How QIM SDK delivers it
  • Follows standard GStreamer plugin architecture and negotiation rules.
  • Uses caps negotiation for format compatibility.
  • Uses allocation queries for memory negotiation and zero-copy paths.
  • Reuses standard image formats such as NV12, RGBA, and I420.
  • Reuses standard metadata where possible and adds AI-specific metadata only when needed.
Developer takeaway If your team already knows GStreamer, QIM SDK fits into the same workflow with Qualcomm-optimized building blocks.
What developers get
  • Freedom to choose runtime per use case.
  • Support for common edge AI workloads.
  • A path from prototype to optimized deployment.
How QIM SDK delivers it
  • Supports TFLite, SNPE, and QNN runtime paths.
  • Supports use cases such as detection, classification, segmentation, pose, super resolution, and audio AI.
  • Supports chained and parallel model execution in the same media flow.
  • Preserves model outputs as metadata so downstream stages can reuse them.
Developer takeaway You can keep one pipeline architecture while changing the model, runtime, or accelerator strategy based on product needs.

​
Developer Notes

Zero-copy pipelines keep the same frame buffer moving across decode, preprocess, inference, overlay, and encode stages. This reduces memory traffic, CPU overhead, and end-to-end latency — especially important for high-resolution and multi-stream workloads.
A typical AI pipeline must resize, color-convert, and normalize frames before inference. QIM SDK offloads these steps through GPU-accelerated plugins instead of doing them in generic CPU-only stages, which helps preserve frame rate and lowers CPU utilization.
Each example shows the same operation built with upstream GStreamer CPU elements and then with QIM SDK hardware-accelerated plugins. Timing numbers are measured on IQ9 with a USB camera at 1080p input.