Scholarlink

<p>Appearance-based gaze estimation, which analyses the visual characteristics of the face, provides crucial hints for comprehending human intentions, especially in the domains of virtual reality and human-computer interaction. It also reveals where an individual focuses its visual attention. We present, in this article, a new gaze estimation method, called "Hybrid ViTNet" which combines two powerful models, ResNet50 and the Transformer, to improve feature representation in eye images. We have used ResNet50 for feature extraction, taking advantage of its ability to capture complex local details. By integrating the Transformer with the Multi-Head-Self-Attention layer, our model takes long-distance relationships into account, thereby improving overall context understanding. These two components are then reinforced by a feedforward network. Our proposed approach focuses on accurate classification of eye images for gaze direction estimation. Evaluated on the MPIIGaze, Columbia Gaze and EyeDiap datasets, it shows promising performance with average angular errors of <InlineEquation ID="IEq1"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="11760_2025_4380_Article_IEq1.gif" Format="GIF" Height="14" Rendition="HTML" Resolution="72" Type="Linedraw" Width="30" /> </InlineMediaObject> <EquationSource Format="TEX">\(4.6^{\circ }\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>4</mn> <mo>.</mo> <msup> <mn>6</mn> <mo>∘</mo> </msup> </mrow> </math></EquationSource> </InlineEquation>, <InlineEquation ID="IEq2"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="11760_2025_4380_Article_IEq2.gif" Format="GIF" Height="14" Rendition="HTML" Resolution="72" Type="Linedraw" Width="30" /> </InlineMediaObject> <EquationSource Format="TEX">\(5.2^{\circ }\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>5</mn> <mo>.</mo> <msup> <mn>2</mn> <mo>∘</mo> </msup> </mrow> </math></EquationSource> </InlineEquation> and <InlineEquation ID="IEq3"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="11760_2025_4380_Article_IEq3.gif" Format="GIF" Height="14" Rendition="HTML" Resolution="72" Type="Linedraw" Width="37" /> </InlineMediaObject> <EquationSource Format="TEX">\(4.80^{\circ }\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>4</mn> <mo>.</mo> <msup> <mn>80</mn> <mo>∘</mo> </msup> </mrow> </math></EquationSource> </InlineEquation>, respectively. These results indicate a clear improvement over existing methods, validating the effectiveness of our model.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An Appearance-based VisionTransformer Network for Enhanced Gaze Estimation

  • 期刊论文

  • 发表日期：

    2025年6月18日

查看全文
  • Rawdha Karmi,
  • Rekka Mastouri,
  • Ines Rahmany,
  • Nawres Khlifa

摘要

Appearance-based gaze estimation, which analyses the visual characteristics of the face, provides crucial hints for comprehending human intentions, especially in the domains of virtual reality and human-computer interaction. It also reveals where an individual focuses its visual attention. We present, in this article, a new gaze estimation method, called "Hybrid ViTNet" which combines two powerful models, ResNet50 and the Transformer, to improve feature representation in eye images. We have used ResNet50 for feature extraction, taking advantage of its ability to capture complex local details. By integrating the Transformer with the Multi-Head-Self-Attention layer, our model takes long-distance relationships into account, thereby improving overall context understanding. These two components are then reinforced by a feedforward network. Our proposed approach focuses on accurate classification of eye images for gaze direction estimation. Evaluated on the MPIIGaze, Columbia Gaze and EyeDiap datasets, it shows promising performance with average angular errors of \(4.6^{\circ }\) 4 . 6 ∘ , \(5.2^{\circ }\) 5 . 2 ∘ and \(4.80^{\circ }\) 4 . 80 ∘ , respectively. These results indicate a clear improvement over existing methods, validating the effectiveness of our model.