Egocentric videos captured by wearable cameras provide a rich, first-person view of human-object interactions, offering unique insights into how people perform everyday activities. However, understanding long-form egocentric video remains challenging due to the complex temporal structure and dynamic nature of this data. In this work, we address the task of Egocentric Action Scene Graph (EASG) generation, which constructs structured graphs that represent hands, objects, and their semantic relationships over time. Unlike previous methods that rely on ground-truth object annotations, we evaluate models in two settings: in the presence of ground-truth object annotations, and in the absence of such annotations, where algorithms rely on predicted objects. We evaluate multiple methods, including transformer-based and fully connected models, for predicting temporally coherent and semantically rich scene graphs. The benchmarked approaches capture the hierarchical and compositional aspects of first-person behavior, potentially enabling improved interpretation and generalization for downstream tasks such as action anticipation and video summarization.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Benchmark of Egocentric Scene Graph Prediction Methods for Understanding Human-Object Interactions

  • Asfand Yaar,
  • Ivan Rodin,
  • Giovanni Maria Farinella,
  • Antonino Furnari

摘要

Egocentric videos captured by wearable cameras provide a rich, first-person view of human-object interactions, offering unique insights into how people perform everyday activities. However, understanding long-form egocentric video remains challenging due to the complex temporal structure and dynamic nature of this data. In this work, we address the task of Egocentric Action Scene Graph (EASG) generation, which constructs structured graphs that represent hands, objects, and their semantic relationships over time. Unlike previous methods that rely on ground-truth object annotations, we evaluate models in two settings: in the presence of ground-truth object annotations, and in the absence of such annotations, where algorithms rely on predicted objects. We evaluate multiple methods, including transformer-based and fully connected models, for predicting temporally coherent and semantically rich scene graphs. The benchmarked approaches capture the hierarchical and compositional aspects of first-person behavior, potentially enabling improved interpretation and generalization for downstream tasks such as action anticipation and video summarization.