> ## Documentation Index
> Fetch the complete documentation index at: https://imsdkdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Building Production Quality Pipelines

## Overview

This guide walks through the complete implementation of a GStreamer-based object detection application using Qualcomm hardware-accelerated plugins. The application:

* Reads an **H.264-encoded MP4 file** from disk
* Performs **YOLOv8 object detection** using a TFLite inference pipeline
* **Overlays** detection bounding boxes and labels onto video frames
* **Renders** the annotated output to a Wayland display

<Info>
  The complete application source code is available in the [repository](https://github.com/qualcomm/gst-plugins-imsdk/tree/main/gst-sample-apps/gst-detection-display-example). For foundational concepts — including pipeline construction, state management, caps negotiation, bus handling, and dynamic pads — refer to [Building a Native GStreamer Application](../qimsdk-overview/building-native-app).
</Info>

***

## Pipeline Diagram

<img src="https://mintcdn.com/qimsdk/xdnKhBBjxpS5mUYP/qimsdk-overview/images/build-obj-pipeline.png?fit=max&auto=format&n=xdnKhBBjxpS5mUYP&q=85&s=d7f6d3fa69efdf8d57dc8a31851f35cd" alt="Pipeline Diagram" width="1886" height="306" data-path="qimsdk-overview/images/build-obj-pipeline.png" />

***

## Implementation Steps

<Steps>
  <Step title="Step 1 — Define the Application Context">
    The application consolidates all runtime state into a single context structure:

    ```c theme={null}
    struct _GstAppContext
    {
      GstElement *pipeline;
      GMainLoop  *mloop;
      gchar      *file;
      gchar      *model;
      gchar      *labels;
      GstElement *parse;
    };
    ```

    | Field      | Description                                                                  |
    | ---------- | ---------------------------------------------------------------------------- |
    | `pipeline` | Top-level GStreamer pipeline object for all pipeline operations              |
    | `mloop`    | GLib main loop — keeps the application alive and dispatches events           |
    | `file`     | Path to the input media file                                                 |
    | `model`    | Path to the TFLite inference model                                           |
    | `labels`   | Path to the class label file                                                 |
    | `parse`    | Reference to `h264parse`, stored explicitly for deferred dynamic pad linking |

    <Info>
      This pattern avoids global variables, keeps application state self-contained, and simplifies callback implementations that require access to pipeline objects.
    </Info>
  </Step>

  <Step title="Step 2 — Create the Pipeline Object">
    ```c theme={null}
    pipeline = gst_pipeline_new ("gst-detection-display-example");
    appctx.pipeline = pipeline;
    ```

    A `GstPipeline` object is created using `gst_pipeline_new()`, establishing the top-level container that owns and manages all elements added to it.

    As a `GstBin` subclass, it is responsible for:

    * **Element lifecycle management**
    * **State propagation** across all contained elements
    * **Synchronization** of clocks and timing

    All subsequent pipeline operations — element management, state transitions, and bus access — are performed through this object.
  </Step>

  <Step title="Step 3 — Instantiate Pipeline Elements">
    ```c theme={null}
    source  = gst_element_factory_make ("filesrc",               "file_src");
    demux   = gst_element_factory_make ("qtdemux",               "file_qtdemux");
    parse   = gst_element_factory_make ("h264parse",             "file_h264_parse");
    decoder = gst_element_factory_make ("v4l2h264dec",           "v4l2h264dec");
    mlbin   = gst_element_factory_make ("qtimlvideotflitebin",   "qtimlvideotflitebin");
    overlay = gst_element_factory_make ("qtivoverlay",           "overlay");
    sink    = gst_element_factory_make ("waylandsink",           "waylandsink");
    ```

    Each element is instantiated using `gst_element_factory_make()`. The first argument identifies the element by its factory name; the second assigns a unique instance name (retrievable later via `gst_bin_get_by_name()`).

    | Element               | Role                                                          |
    | --------------------- | ------------------------------------------------------------- |
    | `filesrc`             | Reads the input MP4 file from disk                            |
    | `qtdemux`             | Demuxes the MP4 container into elementary streams             |
    | `h264parse`           | Parses the H.264 bitstream for downstream decoding            |
    | `v4l2h264dec`         | Hardware-accelerated H.264 video decoder                      |
    | `qtimlvideotflitebin` | Runs TFLite-based YOLOv8 object detection inference           |
    | `qtivoverlay`         | Renders detection bounding boxes and labels onto video frames |
    | `waylandsink`         | Displays annotated video output on a Wayland surface          |
  </Step>

  <Step title="Step 4 — Configure Elements">
    Before the pipeline starts, element properties are set to define runtime behavior.

    **Input source** — configure `filesrc` with the path to the input file:

    ```c theme={null}
    g_object_set (source, "location", appctx->file, NULL);
    ```

    **AI inference** — configure `qtimlvideotflitebin` with delegate, post-processing module, model, and labels:

    ```c theme={null}
    gst_element_set_enum_property (mlbin, "inference-delegate",  "external");
    gst_element_set_enum_property (mlbin, "postprocess-module",  "yolov8");

    g_object_set (mlbin,
        "inference-model",    appctx->model,
        "postprocess-labels", appctx->labels,
        NULL);
    ```

    | Property             | Description                                                 |
    | -------------------- | ----------------------------------------------------------- |
    | `inference-delegate` | Selects the hardware backend for model execution            |
    | `postprocess-module` | Specifies the post-processing algorithm (`yolov8` decoding) |
    | `inference-model`    | Path to the compiled TFLite model                           |
    | `postprocess-labels` | Path to the class label file                                |

    **Display sink** — configure `waylandsink` for synchronized, full-screen rendering:

    ```c theme={null}
    g_object_set (sink, "sync",       TRUE, NULL);
    g_object_set (sink, "fullscreen", TRUE, NULL);
    ```

    | Property     | Description                                              |
    | ------------ | -------------------------------------------------------- |
    | `sync`       | Enables clock-synchronized rendering for smooth playback |
    | `fullscreen` | Instructs the sink to occupy the full display surface    |
  </Step>

  <Step title="Step 5 — Add Elements to the Pipeline">
    ```c theme={null}
    gst_bin_add_many (GST_BIN (appctx->pipeline),
        source, demux, parse, decoder, mlbin, overlay, sink, NULL);
    ```

    `gst_bin_add_many()` transfers ownership of all elements to the pipeline, making it responsible for their lifecycle — including state transitions and resource cleanup. At this point, elements share the same execution context but are **not yet connected**.
  </Step>

  <Step title="Step 6 — Link Static Pipeline Elements">
    ```c theme={null}
    gst_element_link (source, demux);
    gst_element_link_many (parse, decoder, mlbin, overlay, sink, NULL);
    ```

    This establishes the following static connections:

    ```
    filesrc → qtdemux
    h264parse → v4l2h264dec → qtimlvideotflitebin → qtivoverlay → waylandsink
    ```

    <Callout type="warning">
      The link between `qtdemux` and `h264parse` is intentionally omitted here. `qtdemux` exposes its output pads **dynamically at runtime** — this connection is established in Step 7 via dynamic pad handling.
    </Callout>
  </Step>

  <Step title="Step 7 — Handle Dynamic Pads (Demuxer)">
    `qtdemux` determines the number and type of elementary streams only after parsing input data, dynamically creating output pads for each discovered stream. Since these pads do not exist at construction time, they cannot be linked statically.

    A callback is registered for the `pad-added` signal on the demuxer:

    ```c theme={null}
    g_signal_connect (demux, "pad-added",
        G_CALLBACK (gst_app_qtdemux_pad_added_cb), appctx);
    ```

    When a new pad is exposed at runtime, the callback inspects its type and links it to `h264parse` if it carries an H.264 video stream — all other stream types (e.g., audio) are ignored:

    ```c theme={null}
    static void
    gst_app_qtdemux_pad_added_cb (GstElement *element, GstPad *srcpad, gpointer userdata)
    {
      gchar *name = gst_pad_get_name (srcpad);

      if (!g_str_has_prefix (name, "video/x-h264")) {
        g_free (name);
        return;
      }

      gst_app_link_dynamic_src_pad (srcpad, appctx->parse);
      g_free (name);
    }
    ```

    This completes the missing link:

    ```
    qtdemux → h264parse
    ```

    The **full data path is now established end-to-end**:

    ```
    filesrc → qtdemux → h264parse → v4l2h264dec → qtimlvideotflitebin → qtivoverlay → waylandsink
    ```
  </Step>

  <Step title="Step 8 — Set Up Bus Message Handling">
    The application retrieves the pipeline bus and registers callbacks for error, EOS, and warning messages:

    ```c theme={null}
    bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
    gst_bus_add_signal_watch (bus);

    g_signal_connect (bus, "message::error",   G_CALLBACK (gst_app_error_cb),   mloop);
    g_signal_connect (bus, "message::eos",     G_CALLBACK (gst_app_eos_cb),     mloop);
    g_signal_connect (bus, "message::warning", G_CALLBACK (gst_app_warning_cb), NULL);
    ```

    | Signal             | Trigger                                | Action                                 |
    | ------------------ | -------------------------------------- | -------------------------------------- |
    | `message::error`   | Fatal error in a pipeline element      | Log error and initiate shutdown        |
    | `message::eos`     | All data processed; stream end reached | Exit main loop and proceed to teardown |
    | `message::warning` | Non-fatal condition                    | Log warning; pipeline continues        |
  </Step>

  <Step title="Step 9 — Handle Ctrl+C (Interrupt Signal)">
    A `SIGINT` handler is registered using `g_unix_signal_add()` to route the OS signal safely into the GLib main loop:

    ```c theme={null}
    g_unix_signal_add (SIGINT, gst_app_handle_interrupt_signal, &appctx);
    ```

    The signal handler checks the current pipeline state and responds accordingly:

    ```c theme={null}
    if (state == GST_STATE_PLAYING) {
      gst_element_send_event (appctx->pipeline, gst_event_new_eos ());
    } else {
      g_main_loop_quit (appctx->mloop);
    }
    ```

    | Pipeline State      | Action                                                             |
    | ------------------- | ------------------------------------------------------------------ |
    | `GST_STATE_PLAYING` | Send EOS event — allows in-flight buffers to drain before stopping |
    | Any other state     | Quit main loop immediately                                         |

    <Info>
      Sending EOS before stopping is particularly important for file-based outputs where finalization steps (e.g., writing file headers/footers) must complete.
    </Info>
  </Step>

  <Step title="Step 10 — Start the Pipeline">
    ```c theme={null}
    gst_element_set_state (pipeline, GST_STATE_PLAYING);
    ```

    This single call propagates the state transition to all contained elements. Each element progresses through intermediate `READY` and `PAUSED` states before reaching `PLAYING`, at which point data begins to flow from `filesrc` through to `waylandsink`.
  </Step>

  <Step title="Step 11 — Run the Main Loop">
    ```c theme={null}
    g_main_loop_run (mloop);
    ```

    The application enters the GLib main loop, which:

    * **Blocks** the main thread for the duration of pipeline execution
    * **Dispatches** bus messages and invokes registered callbacks as events occur
    * **Exits** only when explicitly quit by the EOS callback, error handler, or interrupt signal handler
  </Step>

  <Step title="Step 12 — Handle End-of-Stream">
    When the pipeline processes the last buffer and EOS propagates to the sink, an EOS bus message is delivered to the registered callback:

    ```c theme={null}
    static void
    gst_app_eos_cb (GstBus *bus, GstMessage *msg, gpointer userdata)
    {
      g_main_loop_quit ((GMainLoop *) userdata);
    }
    ```

    The callback quits the main loop, allowing the main thread to resume and proceed with pipeline teardown and resource cleanup.
  </Step>

  <Step title="Step 13 — Handle Errors">
    If a fatal error occurs at any stage of pipeline execution, the error callback is invoked:

    ```c theme={null}
    static void
    gst_app_error_cb (GstBus *bus, GstMessage *msg, gpointer userdata)
    {
      GError *err       = NULL;
      gchar  *debug_info = NULL;

      gst_message_parse_error (msg, &err, &debug_info);
      g_printerr ("ERROR: %s\n", err->message);

      if (debug_info)
        g_printerr ("Debug details: %s\n", debug_info);

      g_clear_error (&err);
      g_free (debug_info);

      g_main_loop_quit ((GMainLoop *) userdata);
    }
    ```

    Errors are treated as **fatal**. The callback:

    1. Logs the error message and debug information for diagnostics
    2. Quits the main loop to initiate controlled shutdown
    3. The application subsequently transitions the pipeline to `NULL` and releases all resources
  </Step>

  <Step title="Step 14 — Teardown and Resource Cleanup">
    Once the main loop exits — whether due to EOS, error, or interrupt — the application performs an orderly teardown:

    ```c theme={null}
    gst_element_set_state (pipeline, GST_STATE_NULL);
    gst_object_unref (pipeline);
    g_main_loop_unref (mloop);
    ```

    | Call                                              | Effect                                                                                                                           |
    | ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
    | `gst_element_set_state(pipeline, GST_STATE_NULL)` | Transitions pipeline and all elements to `NULL` — stops processing, releases device handles, closes files, frees internal memory |
    | `gst_object_unref(pipeline)`                      | Releases the application's reference to the pipeline; destroys it and all owned elements if no other references are held         |
    | `g_main_loop_unref(mloop)`                        | Releases the main loop object and frees associated resources                                                                     |

    <Info>
      This sequence ensures all resources are released in the correct order and that the application exits in a **well-defined, clean state**.
    </Info>
  </Step>
</Steps>

***

## Summary

This guide covered the full implementation of a GStreamer YOLOv8 object detection application across 14 steps:

| Phase              | Steps | Key Operations                                    |
| ------------------ | ----- | ------------------------------------------------- |
| **Initialization** | 1—2   | Define app context, create pipeline               |
| **Element Setup**  | 3—5   | Instantiate, configure, and add elements          |
| **Linking**        | 6—7   | Static links + dynamic pad handling for `qtdemux` |
| **Event Handling** | 8—9   | Bus message callbacks, SIGINT handler             |
| **Execution**      | 10—11 | Set `PLAYING` state, run main loop                |
| **Shutdown**       | 12—14 | EOS/error handling, teardown and resource cleanup |