Pose Estimation

🟢 W8A8 — Model Precision

Download Required Files

File	Save as
Person/foot detection model	`foot_track_net_w8a8.tflite`
foot_track_net.json	`foot_track_net.json`
foot_track_net_settings.json	`foot_track_net_settings.json`
HRNet pose model	`hrnet_pose_w8a8.tflite`
hrnet.json	`hrnet.json`
hrnet_settings.json	`hrnet_settings.json`
Input video — use an encoded video file containing a single person in frame	`input_video.mp4`

If any downloaded file is a .zip archive, extract it on your host machine before copying: unzip filename.zip

Copy Files to Device

# Replace $HOME to the appropriate device path before running the commands.
# For QLI:    /root
# For Ubuntu: /home/ubuntu
# Modify this based on your platform and ensure files are copied to the correct location on the device.

ssh <user>@<device-ip> "mkdir -p $HOME/{models,labels,media}"
scp foot_track_net_w8a8.tflite       <user>@<device-ip>:$HOME/models/
scp foot_track_net.json              <user>@<device-ip>:$HOME/labels/
scp foot_track_net_settings.json     <user>@<device-ip>:$HOME/labels/
scp hrnet_pose_w8a8.tflite           <user>@<device-ip>:$HOME/models/
scp hrnet.json                       <user>@<device-ip>:$HOME/labels/
scp hrnet_settings.json              <user>@<device-ip>:$HOME/labels/
scp input_video.mp4                  <user>@<device-ip>:$HOME/media/

Connect to device

# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip>

Set Environment Variables

Qualcomm Linux

mkdir -p $HOME/{models,labels,media}
export MODEL_NAME_1=foot_track_net_w8a8.tflite
export LABELS_NAME_1=foot_track_net.json
export LABELS_NAME_2=foot_track_net_settings.json
export MODEL_NAME_2=hrnet_pose_w8a8.tflite
export LABELS_NAME_3=hrnet.json
export LABELS_NAME_4=hrnet_settings.json
export SRC_VIDEO_NAME=ai_demo_sample.mp4

Run the Pipeline

gst-launch-1.0 -e --gst-debug=2 \
  qtimlvconverter name=stage_01_preproc mode=image-batch-non-cumulative \
  qtimltflite name=stage_01_inference delegate=external external-delegate-path=libQnnTFLiteDelegate.so \
  external-delegate-options="QNNExternalDelegate,backend_type=htp,log_level=(string)1;" \
  model=$HOME/models/$MODEL_NAME_1 \
  qtimlpostprocess name=stage_01_postproc results=10 module=qpd labels=$HOME/labels/$LABELS_NAME_1 \
  settings=$HOME/labels/$LABELS_NAME_2 \
  qtimlvconverter name=stage_02_preproc mode=roi-batch-cumulative image-disposition=centre \
  qtimltflite name=stage_02_inference delegate=external external-delegate-path=libQnnTFLiteDelegate.so \
  external-delegate-options="QNNExternalDelegate,backend_type=htp,htp_performance_mode=(string)2,log_level=(string)1;" \
  model=$HOME/models/$MODEL_NAME_2 \
  qtimlpostprocess name=stage_02_postproc results=2 module=hrnet labels=$HOME/labels/$LABELS_NAME_3 \
  settings=$HOME/labels/$LABELS_NAME_4 \
  filesrc location=$HOME/media/$SRC_VIDEO_NAME ! qtdemux ! h264parse ! \
  v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12 ! queue ! \
  tee name=t_split_1 \
  t_split_1. ! queue ! stage_01_preproc. stage_01_preproc. ! queue ! stage_01_inference. stage_01_inference. ! queue ! \
  stage_01_postproc. stage_01_postproc. ! text/x-raw ! queue ! qtimetamux name=metamux_1 \
  t_split_1. ! queue ! metamux_1. metamux_1. ! queue ! tee name=t_split_2 \
  t_split_2. ! queue ! stage_02_preproc. stage_02_preproc. ! queue ! stage_02_inference. stage_02_inference. ! queue ! \
  stage_02_postproc. stage_02_postproc. ! text/x-raw ! queue ! qtimetamux name=metamux_2 \
  metamux_2. ! queue ! qtivoverlay ! queue ! waylandsink fullscreen=true sync=true \
  t_split_2. ! queue ! metamux_2.

Blog

Overview

Getting Started

Building AI Pipelines

Examples

Reference

Advanced Topics