Scholarlink

As Artificial Neural Networks (ANNs) reach human and super-human performance, the convergence between the visual strategies learned by ANNs and human perception is a fundamental step to allow their deployment as models of biological vision and their integration into assistive and rehabilitation technologies. In this work, we investigate the alignment between visual strategies learned by an ANN, specifically the Temporal Shift Module (TSM), and human gaze-based visual attention, during classification of social interactions from videos. We present preliminary results comparing the similarity between saliency maps generated by the ANN model after fine-tuning on the social interaction classification task and ground-truth fixation maps of human gaze data.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Pilot Study Exploring the Alignment of Humans and CNN During Perception of Social Interactions

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Guido Vallarino,
  • Lucia Schiatti,
  • Matteo Moro,
  • Yen-Ling Kuo,
  • Mengmi Zhang,
  • Monica Gori,
  • Boris Katz,
  • Andrei Barbu,
  • Alessio Del Bue

摘要

As Artificial Neural Networks (ANNs) reach human and super-human performance, the convergence between the visual strategies learned by ANNs and human perception is a fundamental step to allow their deployment as models of biological vision and their integration into assistive and rehabilitation technologies. In this work, we investigate the alignment between visual strategies learned by an ANN, specifically the Temporal Shift Module (TSM), and human gaze-based visual attention, during classification of social interactions from videos. We present preliminary results comparing the similarity between saliency maps generated by the ANN model after fine-tuning on the social interaction classification task and ground-truth fixation maps of human gaze data.