Scholarlink

<p>AI-enabled Augmented Reality (AR) systems are transforming high-stakes, human-centric domains such as surgery, electronic assembly, aircraft maintenance, and piloting by enhancing users’ situational awareness. Human-Object Interaction Recognition (HOIR) helps in personalization and context-aware assistance in AR systems. User identification is essential for ensuring safety, access control, and operational security in such high-stakes scenarios. Person identification based on fingerprint, iris, face, and gait recognition is ineffective in egocentric videos, where these modalities are not visible. Existing research on egocentric person identification and HOIR predominantly relies on heavy deep learning models that also lack interpretability. In order to address the challenge of HOIR and user identification in low-latency, resource-constrained environments like AR headsets, we present <i>Interact2Sign </i>(I2S), a multi-stage framework that performs unobtrusive user identification based on Human-Object Interaction Recognition, using handcrafted features derived from 3D hand poses in egocentric videos. The feature extraction pipeline organizes 3D hand pose data into five semantically meaningful groups: Spatial, Frequency, Kinematic, Orientation, and a novel descriptor for bimanual interactions, Inter-Hand Spatial Envelope (IHSE). Extensive ablation studies were conducted, and the optimal feature group achieved an average F1-score of 97.52% using 5-fold cross-validation for user identification on a dataset derived from the ARCTIC and H2O datasets. With a model size under 4 MB and an inference time of just 0.1&#xa0;s, I2S demonstrates state-of-the-art performance while remaining suitable for real-time, on-device authentication and HOI-driven assistive applications in security-critical AR environments.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Person identification from egocentric human-object interactions using 3D hand pose

  • 期刊论文

  • 发表日期：

    2026年2月26日

查看全文
  • Muhammad Hamza,
  • Danish Hamid,
  • Muhammad Tahir Akram

摘要

AI-enabled Augmented Reality (AR) systems are transforming high-stakes, human-centric domains such as surgery, electronic assembly, aircraft maintenance, and piloting by enhancing users’ situational awareness. Human-Object Interaction Recognition (HOIR) helps in personalization and context-aware assistance in AR systems. User identification is essential for ensuring safety, access control, and operational security in such high-stakes scenarios. Person identification based on fingerprint, iris, face, and gait recognition is ineffective in egocentric videos, where these modalities are not visible. Existing research on egocentric person identification and HOIR predominantly relies on heavy deep learning models that also lack interpretability. In order to address the challenge of HOIR and user identification in low-latency, resource-constrained environments like AR headsets, we present Interact2Sign (I2S), a multi-stage framework that performs unobtrusive user identification based on Human-Object Interaction Recognition, using handcrafted features derived from 3D hand poses in egocentric videos. The feature extraction pipeline organizes 3D hand pose data into five semantically meaningful groups: Spatial, Frequency, Kinematic, Orientation, and a novel descriptor for bimanual interactions, Inter-Hand Spatial Envelope (IHSE). Extensive ablation studies were conducted, and the optimal feature group achieved an average F1-score of 97.52% using 5-fold cross-validation for user identification on a dataset derived from the ARCTIC and H2O datasets. With a model size under 4 MB and an inference time of just 0.1 s, I2S demonstrates state-of-the-art performance while remaining suitable for real-time, on-device authentication and HOI-driven assistive applications in security-critical AR environments.