Skip to main content

​
Overview

qtirtspbin is a GStreamer bin element that simplifies the construction and management of RTSP streaming pipelines for applications that need to publish media over a network. It operates as an RTSP server, exposing a local streaming endpoint that RTSP-compatible clients, such as VLC, can connect to in order to receive live or processed media streams. Rather than functioning as a client that consumes an external RTSP source, qtirtspbin provides a server-side streaming interface that allows an application to distribute its output to one or more downstream consumers. This makes it well suited for use cases where media, analytics results, or other generated data must be shared across the network in real time. The element supports a variety of payload types, including:
  • Encoded video streams such as H.264 and H.265
  • Audio streams
  • Machine learning metadata
  • Serialized custom data
qtirtspbin can handle multiple input types and automatically assembles the internal pipeline required for stream packetization, transport, and RTSP session handling. By managing these lower-level details internally, it reduces the amount of application-side logic needed to build a complete streaming solution. In addition to basic RTSP transmission, qtirtspbin includes built-in mechanisms for:
  • Session Management: supports multiple simultaneous client connections and manages each RTSP session
  • Stream synchronization: preserves consistent timing across video, audio, and metadata streams
  • Custom payload support: enables transmission of non-standard data types by encapsulating them in RTP payloads
Overall, qtirtspbin provides a flexible and developer-friendly abstraction for building RTSP-based streaming applications that need to deliver synchronized media and auxiliary data to network clients.

​
Example Pipeline

1

Download Required Files

FileDownloadSave as
Sample videoInput videoai_demo_sample.mp4
YOLOX W8A8 modelQualcomm AI Hub — YOLOXyolo_x_w8a8.tflite
Detection labelsyolov8.jsonyolov8.json
2

Copy files to device

# Replace $HOME to the appropriate device path before running the commands.
# For QLI:    /root
# For Ubuntu: /home/ubuntu
# Modify this based on your platform and ensure files are copied to the correct location on the device.
# Run from your host machine — replace <user> and <device-ip>

ssh <user>@<device-ip> "mkdir -p $HOME/{models,labels,media,media/output}"
scp ai_demo_sample.mp4   <user>@<device-ip>:$HOME/media/
scp yolo_x_w8a8.tflite          <user>@<device-ip>:$HOME/models/
scp yolov8.json                  <user>@<device-ip>:$HOME/labels/
3

Connect to device

# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip>
4

Set environment variables

Run below command on your device
export SRC_VIDEO_NAME=ai_demo_sample.mp4
export MODEL_NAME=yolo_x_w8a8.tflite
export LABELS_NAME=yolov8.json
5

Run the pipeline

gst-launch-1.0 -e --gst-debug=2 \
qticamsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! \
v4l2h264enc capture-io-mode=4 output-io-mode=4 ! queue ! \
h264parse config-interval=1 ! queue ! qtirtspbin address=<ip addr of device>

Connect to the stream with VLC: rtsp://<ip addr of device>:8900/live
Ensure that the device has to be connected to network to try out these example pipelines.

​
Hierarchy

GObject
   GstObject
      GstElement
         GstBin
            qtirtspbin

​
Pad Templates

​
sink

Capabilities
video/x-h264format: NA
video/x-h265format: NA
audio/mpegformat: NA
text/x-rawformat: NA
Availability: On request
Direction: sink
Pad Name: sink_%u

​
Element Properties

PropertyDescription
addressIP address of the server.

Type: String
Default: "127.0.0.1"
Flags: readable/writable
modeOperational mode.

Type: Enum
Default: 0, "async"
Range:
    (0): async - Return buffers immediately if no client is connected
    (1): sync - Restricts the number of input buffers
Flags: readable/writable (changeable in NULL, READY)
mpointMounting point.

Type: String
Default: "/live"
Flags: readable/writable
portPort to listen on.

Type: String
Default: "8900"
Flags: readable/writable

​
Internal Architecture and Streaming model:

qtirtspbin is a GStreamer bin element that internally manages the components required to transmit media streams over RTSP. Unlike an RTSP client, it operates as an RTSP server and exposes a mount point that external clients can connect to for receiving the stream

​
Core components

Internally, qtirtspbin dynamically constructs a pipeline that includes the following elements:

​
Input pads

The plugin accepts multiple types of input through its pads, including:
  • Encoded video, such as H.264 and H.265
  • Audio Streams
  • ML metadata, such as JSON
  • Custom serialized data

​
RTP Packetizer

Incoming media and data are converted into RTP packets using the appropriate payloaders, such as:
  • rtph264pay and rtph265pay for video
  • rtpmp4apay for audio
  • rtpgstpay for metadata or other structured custom data

​
RTSP Server Interface

qtirtspbin launches an internal RTSP server, typically on 127.0.0.1:8900, and registers one or more mount points such as /live or /stream. Through this interface, it manages:
  • Client sessions
  • Stream negotiation
  • Reconnection behavior

​
Processing flow

  1. Media Input
    • External elements push encoded media or data into qtirtspbin through request pads
  2. Stream Identification and Routing
    • Each incoming stream is identified
    • The stream is routed to the appropriate RTP payloader.
  3. Packetization
    • Media is packetized into RTP format.
    • Synchronization across streams is handled internally.
  4. Session Management
    • The plugin registers the stream with the RTSP server.
    • Clients connect using RTSP and receive the live stream.
  5. Client Handling
    • Multiple clients can connect at the same time.
    • The plugin manages buffering, latency, and session state.

​
Custom payload support

In addition to standard media types such as video and audio, qtirtspbin can also transport custom data streams over RTSP by encapsulating them in RTP payloads. This is useful for applications that need to deliver non-standard or structured information, such as:
  • Machine learning inference results like bounding boxes, labels, and scores
  • Sensor data such as telemetry or depth information
  • Application-specific metadata such as JSON or other serialized formats
For custom payloads, the flow is typically: Input Custom data is pushed into qtirtspbin through a request pad. Payloading The plugin uses rtpgstpay to encapsulate the incoming buffer into RTP packets. Transmission The RTP stream is published through the internal RTSP server on the configured mount point. Client Consumption Clients such as VLC, FFmpeg, or custom RTSP receivers can consume the stream and deserialize the payload based on the agreed format.

​
Usage

Ensure you have followed the prerequisites before continuing

​
Single stream from camera to the RTSP

Demonstrates real-time RTSP stream reception from live camera source. 
gst-launch-1.0 -e --gst-debug=2 \
qticamsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! \
v4l2h264enc capture-io-mode=4 output-io-mode=4 ! queue ! \
h264parse config-interval=1 ! queue ! qtirtspbin address=<ip addr of device>

Connect to the stream with VLC: rtsp://<ip addr of device>:8900/live

​
Two streams from camera to RTSP

Demonstrates usage of rtspbin to stream two different live feeds. Each qtirtspbin has internal RTSP server and the client should connect to it. When creating an unique port should be set to each instance of the plugin. 
gst-launch-1.0 -e qticamsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! tee name=camsrc ! v4l2h264enc capture-io-mode=4 output-io-mode=4 ! queue ! h264parse config-interval=1 ! queue ! qtirtspbin address=<ip addr of device> port=8900 \
camsrc. ! v4l2h264enc capture-io-mode=4 output-io-mode=4 ! queue ! h264parse config-interval=1 ! queue ! qtirtspbin address=<ip addr of device> port=8901

Connect to the stream with VLC: rtsp://<ip addr of device>:8900/live
Connect to the stream with VLC: rtsp://<ip addr of device>:8901/live

​
Single stream from camera to the RTSP with ML detection

Demonstrates streaming of live camera stream and ML detection metadata on two request pads of the qtirtspbin. The video and ML metadata are received on the other pipeline and muxed together to overlay the objects detected and display the bounding boxes. 
#Console 1:
gst-launch-1.0 -e qticamsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! \
queue ! tee name=split ! \
  v4l2h264enc capture-io-mode=4 output-io-mode=4 ! queue ! \
  h264parse config-interval=-1 ! queue ! qtirtspbin name=rtsp address=<ip addr of device> \
split. ! queue ! qtimlvconverter ! queue ! \
  qtimltflite delegate=external external-delegate-path=libQnnTFLiteDelegate.so external-delegate-options="QNNExternalDelegate,backend_type=htp;" model=$HOME/models/$MODEL_NAME ! queue ! \
  qtimlpostprocess results=5 module=yolov8 labels=$HOME/labels/$LABELS_NAME settings="{\"confidence\": 75.0}" ! text/x-raw ! queue ! rtsp.

#Console 2:
gst-launch-1.0 -e rtspsrc location=rtsp://<ip addr of device>:8900/live ! rtph264depay ! video/x-h264,colorimetry=bt709 ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12 ! queue ! qtimetamux name=metamux ! queue ! qtivoverlay ! waylandsink fullscreen=true async=true sync=false rtspsrc location=rtsp://<ip addr of device>:8900/live ! rtpgstdepay ! text/x-raw ! queue ! metamux.