Scholarlink

<p>Pretrained vision-language models like CLIP have recently demonstrated outstanding performance across various downstream tasks, including image classification and video understanding. However, in high-level visual tasks like automated deception detection (ADD), the semantic association between labels and visual content is weak and lacks explicit textual descriptions. Consequently, the applicability of such models to automated deception detection remains uncertain. To address this question, this paper applies the pre-trained CLIP model to automated deception detection and proposes a new ADD-CLIP model. Inspired by findings in psychology, we inject domain knowledge of deception behavior into the CLIP model. Specifically, ADD-CLIP consists of a visual part and a textual part. In the visual part, we introduce two prompt learning strategies in the image encoder of CLIP: temporal and micro-difference prompt learning. These strategies are implemented through learnable network layers to capture the temporal dynamics and subtle motion characteristics of faces. In the text part, instead of using traditional category labels as text modality input, we build facial behavior difference textual descriptions from both face and AU perspectives as text supervision with the help of large language model such as ChatGPT. In addition, we add learnable dual prompts corresponding to the behavior descriptions at the facial-level and AU-level, respectively. This design allows the model to automatically learn complementary semantic spaces, resulting in a more accurate decision boundary. Extensive experimental results show that ADD-CLIP achieves strong performance on the DOLOS, Bag-of-Lies, and MU3D datasets, verifying its effectiveness. The code is publicly available at <a href="https://github.com/zdl12345678/Deception-Detection-Meets-Vision-Language-Models">https://github.com/zdl12345678/Deception-Detection-Meets-Vision-Language-Models</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Deception Detection Meets Vision-Language Models

  • 期刊论文

  • 发表日期：

    2026年7月14日

查看全文
  • Dongliang Zhu,
  • Ruimin Hu,
  • Mei Wang,
  • Xiang Guo,
  • Liang Liao,
  • Mang Ye

摘要

Pretrained vision-language models like CLIP have recently demonstrated outstanding performance across various downstream tasks, including image classification and video understanding. However, in high-level visual tasks like automated deception detection (ADD), the semantic association between labels and visual content is weak and lacks explicit textual descriptions. Consequently, the applicability of such models to automated deception detection remains uncertain. To address this question, this paper applies the pre-trained CLIP model to automated deception detection and proposes a new ADD-CLIP model. Inspired by findings in psychology, we inject domain knowledge of deception behavior into the CLIP model. Specifically, ADD-CLIP consists of a visual part and a textual part. In the visual part, we introduce two prompt learning strategies in the image encoder of CLIP: temporal and micro-difference prompt learning. These strategies are implemented through learnable network layers to capture the temporal dynamics and subtle motion characteristics of faces. In the text part, instead of using traditional category labels as text modality input, we build facial behavior difference textual descriptions from both face and AU perspectives as text supervision with the help of large language model such as ChatGPT. In addition, we add learnable dual prompts corresponding to the behavior descriptions at the facial-level and AU-level, respectively. This design allows the model to automatically learn complementary semantic spaces, resulting in a more accurate decision boundary. Extensive experimental results show that ADD-CLIP achieves strong performance on the DOLOS, Bag-of-Lies, and MU3D datasets, verifying its effectiveness. The code is publicly available at https://github.com/zdl12345678/Deception-Detection-Meets-Vision-Language-Models.