We present a personalized, pose-estimation-based physiotherapy system that delivers real-time movement feedback using 3D pose estimation and efficient frame-level pose matching from monocular video. The pipeline builds on the MotionAGFormer framework, combining a lightweight YOLO-based person detector, HRNet for 2D keypoint detection, and a hybrid transformer-graph convolution network for 3D pose lifting. To align patient movements with reference sequences, we employ a local dynamic time warping (DTW) algorithm constrained by a Sakoe-Chiba band, reducing computational complexity by narrowing the search space, thereby enabling speed-accuracy trade-offs. Evaluation on the 3DFit dataset under varying noise levels shows that small bands maintain accuracy under low noise but degrade with increasing noise, while larger bands recover accuracy close to global DTW. Notably, the smallest tested band achieves a 4.8 \(\times \) speed-up over global DTW. To support individualized rehabilitation, we propose an adaptive audio-visual feedback strategy with a user interface that dynamically adjusts tolerance bounds around target joint angles based on time-dependent performance. This enables continuous, joint-level feedback and fine-grained movement correction. Together, these components form a scalable, interpretable, and adaptive biofeedback system suitable for both clinical settings and home-based digital physiotherapy.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Adaptive Biofeedback for Digital Physiotherapy Using Sakoe-Chiba Constrained Pose Matching

  • Jonas Leo Mueller,
  • Alexander Weiss,
  • Bjoern M. Eskofier

摘要

We present a personalized, pose-estimation-based physiotherapy system that delivers real-time movement feedback using 3D pose estimation and efficient frame-level pose matching from monocular video. The pipeline builds on the MotionAGFormer framework, combining a lightweight YOLO-based person detector, HRNet for 2D keypoint detection, and a hybrid transformer-graph convolution network for 3D pose lifting. To align patient movements with reference sequences, we employ a local dynamic time warping (DTW) algorithm constrained by a Sakoe-Chiba band, reducing computational complexity by narrowing the search space, thereby enabling speed-accuracy trade-offs. Evaluation on the 3DFit dataset under varying noise levels shows that small bands maintain accuracy under low noise but degrade with increasing noise, while larger bands recover accuracy close to global DTW. Notably, the smallest tested band achieves a 4.8 \(\times \) speed-up over global DTW. To support individualized rehabilitation, we propose an adaptive audio-visual feedback strategy with a user interface that dynamically adjusts tolerance bounds around target joint angles based on time-dependent performance. This enables continuous, joint-level feedback and fine-grained movement correction. Together, these components form a scalable, interpretable, and adaptive biofeedback system suitable for both clinical settings and home-based digital physiotherapy.