> ## Documentation Index > Fetch the complete documentation index at: https://imsdkdocs.qualcomm.com/llms.txt > Use this file to discover all available pages before exploring further. # Building a Wrist-Controlled Car Game Using QIM SDK > A two-stage palm detection and hand landmark pipeline turns wrist movement into a virtual steering wheel for a real-time, touchless split-screen driving game, built with QIM SDK
Gesture Recognition

A split-screen touchless driving game controlled entirely by wrist movement, built on a two-stage palm detection and hand landmark pipeline running on Qualcomm QIM SDK with QNN HTP acceleration.

QIM SDK Team · Aug 3, 2026 · ← All posts
## Introduction "TouchlessDrive" is a touchless car game that turns a player's raised hands into a virtual steering wheel. Using a single USB camera, the system tracks both wrists and interprets the line between them as the steering input. When the player tilts their hands left or right, the on-screen car responds in real time. The experience is displayed in a split-screen layout, with the live camera feed shown alongside the game interface. Under the hood, TouchlessDrive runs on a two-stage hand-tracking pipeline built with the Qualcomm QIM SDK. First, a palm detection model identifies each hand in the frame. Then, a hand-landmark model extracts precise keypoints, including the wrist locations used for steering. Both models run through the QNN TFLite delegate, offloading inference to the on-device HTP instead of the CPU. This helps keep the game loop responsive and capable of real-time performance. The project highlights how the QIM SDK's metadata pipeline can power interactive application logic beyond traditional video analytics. In TouchlessDrive, AI-generated hand keypoints are transformed directly into game-controller input, enabling a natural, controller-free gaming experience. ## Use Case Overview Two-stage inference, palm detection followed by hand landmark estimation, locates both of the player's hands and extracts their wrist keypoints every frame. The line connecting the two wrist points is treated as a steering wheel. Its tilt angle, measured relative to horizontal, becomes the steering input. The tilt angle is converted into a clamped steering value that moves the on-screen car left or right, with smoothing applied to avoid jittery motion. The display is divided into two halves: the left shows the raw camera feed with hand-tracking overlays, and the right renders the game itself, including the road, car, obstacles, and score. Obstacles spawn and scroll down the game lane; a collision ends the run and briefly shows a crash state before the game automatically resets. ## Pipeline diagram Touchless Wrist Steering Car Game Pipeline ## Elements used in pipeline | Element | Description | | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | `v4l2src` | Captures raw frames from the USB camera. | | [`qtivtransform`](../plugin-reference/qtivtransform) | Applies a horizontal flip so the camera feed behaves like a mirror, matching the player's actual hand motion. | | [`qtimlvconverter`](../plugin-reference/qtimlvconverter) | Hardware-accelerated resize, color-space conversion, and normalization ahead of each inference stage. | | [`qtimltflite`](../plugin-reference/qtimltflite) | Runs the palm detection and hand landmark TFLite models on the QNN HTP delegate. | | [`qtimlpostprocess`](../plugin-reference/qtimlpostprocess) | Decodes raw model output into structured detections, using the `palmd` module for palm detection and `hlandmark` for hand landmarks. | | [`qtimetamux`](../plugin-reference/qtimetamux) | Synchronizes inference metadata from each stage with the corresponding video frame. | | `qtimetatransform` | Remaps palm-detection metadata (via the `roi-palmd` module) into per-hand regions of interest for the landmark stage. | | [`qtivoverlay`](../plugin-reference/qtivoverlay) | Draws bounding boxes and keypoints onto the camera-feed half of the split screen. | | [`qtimlmetaparser`](../plugin-reference/qtimetaparser) | Exposes per-frame landmark metadata to the application through an `appsink` callback. | | `cairooverlay` | Renders the game itself, including the road, car, obstacles, score, and the steering-wheel visualization, as a Cairo drawing surface. | | [`waylandsink`](../plugin-reference/waylandsink) | Displays the final composited split-screen output. | ## How it works The USB camera feed is flipped horizontally so on-screen motion matches the player's real-world hand movement, then formatted to a fixed resolution and frame rate for the rest of the pipeline. Palm detection runs first, locating each hand's bounding region. `qtimetatransform` uses the `roi-palmd` module to convert each detected region into an aligned crop, which the hand landmark model then processes to extract per-hand keypoints, most importantly the wrist. An `appsink` callback receives the landmark metadata for every frame and pulls out the wrist keypoint (keypoint ID 0) for each tracked hand. When two hands are present, the pair with the largest horizontal separation is selected as the active steering pair. The angle of the line between the two wrist points is computed and mapped to a steering value clamped to a fixed range, then smoothed across frames so small tracking jitter doesn't translate into an erratic car. Each frame, the steering value nudges the car's horizontal position, obstacles advance down the lane, and a collision check runs against the car's current position. A collision triggers a brief crash state and increments state that leads to an automatic restart. A Cairo drawing callback renders both halves of the display in a single pass: the left half shows the camera feed with the tracked wrist line and a steering-wheel indicator overlaid, and the right half draws the road, car, obstacles, and score. ## Setup Requirements ### Hardware HW Setup | Component | Description | | ------------------------ | --------------------------------------------------------------------------- | | **Edge Device** | RB3 Gen 2, IQ8, or IQ9. Runs the hand-tracking inference and the game loop. | | **Camera Source** | USB camera positioned to capture the player's hands and upper body. | | **HDMI Display Monitor** | Connected to the edge device to show the split-screen game output. | ### Software Flash your Qualcomm Edge device by following the device setup and flashing instructions [here](../installation), then install the Python and GStreamer prerequisites (`python3`, `gstreamer1.0`, `python3-gi`) needed to run the pipeline. The palm detection and hand landmark TFLite models, along with their label and settings files, are the same assets used by the [hand gesture recognition pipeline](/blogs/smart-hand-gesture-recognition). See that post's model download steps for the palm and hand landmark model files. Check application source code on GitHub: [`demo_car_game/main.cc`](https://github.com/qualcomm/qimsdk/blob/main/cpp/examples/demo-apps/demo_car_game/main.cc) Pre-built application on device: `/usr/bin/qimsdk_demo_car_game` #### Download Required Files Download the gesture recognizer task bundle from Google MediaPipe to obtain the palm detection and hand landmark models: ```bash theme={null} # Download the gesture recognizer task bundle wget https://storage.googleapis.com/mediapipe-models/gesture_recognizer/gesture_recognizer/float16/latest/gesture_recognizer.task # Extract the top-level task unzip gesture_recognizer.task # Extract hand landmarker models unzip hand_landmarker.task # save hand_detector.tflite as palm_detection_full.tflite # save hand_landmarks_detector.tflite as hand_landmark_full.tflite ``` | File | Save as | | ----------------------------------------------------------------------------------------------------------- | ---------------------------- | | Palm detection model (see steps above) | palm\_detection\_full.tflite | | Hand landmark model (see steps above) | hand\_landmark\_full.tflite | | palmd\_labels.json | palmd\_labels.json | | palmd\_settings.json | palmd\_settings.json | | hlandmarks.json | hlandmarks.json | | hlandmark\_settings.json | hlandmark\_settings.json | If a downloaded model file is a `.zip` archive, extract it on your host machine before copying: `unzip filename.zip` ```bash SCP (SSH) theme={null} # Replace and with your device credentials. ssh @ "mkdir -p ~/Downloads/qimsdk_samples/{models,labels}" scp palm_detection_full.tflite @:~/Downloads/qimsdk_samples/models/ scp hand_landmark_full.tflite @:~/Downloads/qimsdk_samples/models/ scp palmd_labels.json @:~/Downloads/qimsdk_samples/labels/ scp palmd_settings.json @:~/Downloads/qimsdk_samples/labels/ scp hlandmarks.json @:~/Downloads/qimsdk_samples/labels/ scp hlandmark_settings.json @:~/Downloads/qimsdk_samples/labels/ ``` Connect a USB (UVC) camera to the target device and verify it is exposed as `/dev/video2` (update the `--input-config` value below if your camera is exposed on a different node). ```bash theme={null} /usr/bin/qimsdk_demo_car_game --input-config /dev/video2 ``` The application tracks both wrists from the live USB camera feed, mapping their relative tilt to a virtual steering wheel that steers a split-screen driving game in real time. To stop the application, press **CTRL + C**. Check application source code on GitHub: [`qimsdk_demo_car_game.py`](https://github.com/qualcomm/qimsdk/blob/main/python/examples/demo-apps/qimsdk_demo_car_game.py) Pre-built application on device: `/usr/bin/qimsdk_demo_car_game.py` #### Download Required Files Download the gesture recognizer task bundle from Google MediaPipe to obtain the palm detection and hand landmark models: ```bash theme={null} # Download the gesture recognizer task bundle wget https://storage.googleapis.com/mediapipe-models/gesture_recognizer/gesture_recognizer/float16/latest/gesture_recognizer.task # Extract the top-level task unzip gesture_recognizer.task # Extract hand landmarker models unzip hand_landmarker.task # save hand_detector.tflite as palm_detection_full.tflite # save hand_landmarks_detector.tflite as hand_landmark_full.tflite ``` | File | Save as | | ----------------------------------------------------------------------------------------------------------- | ---------------------------- | | Palm detection model (see steps above) | palm\_detection\_full.tflite | | Hand landmark model (see steps above) | hand\_landmark\_full.tflite | | palmd\_labels.json | palmd\_labels.json | | palmd\_settings.json | palmd\_settings.json | | hlandmarks.json | hlandmarks.json | | hlandmark\_settings.json | hlandmark\_settings.json | If a downloaded model file is a `.zip` archive, extract it on your host machine before copying: `unzip filename.zip` ```bash SCP (SSH) theme={null} # Replace and with your device credentials. ssh @ "mkdir -p ~/Downloads/qimsdk_samples/{models,labels}" scp palm_detection_full.tflite @:~/Downloads/qimsdk_samples/models/ scp hand_landmark_full.tflite @:~/Downloads/qimsdk_samples/models/ scp palmd_labels.json @:~/Downloads/qimsdk_samples/labels/ scp palmd_settings.json @:~/Downloads/qimsdk_samples/labels/ scp hlandmarks.json @:~/Downloads/qimsdk_samples/labels/ scp hlandmark_settings.json @:~/Downloads/qimsdk_samples/labels/ ``` Connect a USB (UVC) camera to the target device and verify it is exposed as `/dev/video2` (update the `--input-config` value below if your camera is exposed on a different node). ```bash theme={null} python3 /usr/bin/qimsdk_demo_car_game.py --input-config /dev/video2 ``` The application tracks both wrists from the live USB camera feed, mapping their relative tilt to a virtual steering wheel that steers a split-screen driving game in real time. To stop the application, press **CTRL + C**. ## Application Function Breakdown The TouchlessDrive application is organized into a handful of function groups, each responsible for one stage of the hand-to-steering flow: * **Metadata parsing**: extracts the wrist keypoint (ID 0) for each tracked hand from the raw landmark metadata stream, handling frames where zero, one, or two hands are detected. * **Steering pair selection**: when two hands are present, selects the pair with the largest horizontal separation as the active steering wheel, discarding stray or partial detections. * **Steering angle calculation**: computes the tilt angle of the line between the two wrist points, maps it to a clamped steering value, and smooths it across frames to filter out landmark jitter. * **Game state and physics**: advances the car's horizontal position based on the current steering value, scrolls obstacles down the lane, runs collision checks, and manages the crash/restart state machine. * **Rendering**: a Cairo drawing callback composes the split-screen output each frame — the camera feed with the wrist line and steering-wheel indicator on the left, and the road, car, obstacles, and score on the right. * **Pipeline setup**: assembles the two-stage palm detection and hand landmark QIM SDK pipeline and wires the `appsink` callback and Cairo overlay into the runtime before the pipeline starts. ## Expected Output