Scholarlink

<p>Effective text anomaly detection is crucial given the explosion of textual data across domains like cybersecurity and social media. Current methods struggle with long-range semantics, contextual nuances, high dimensionality, computational bottlenecks for long sequences, and real-time scalability. We propose TAD-VAE, a novel architecture combining Transformer-based contextual encoding with Variational Autoencoders (VAEs), enhanced by hybrid sparse attention for efficient long-range context modeling. TAD-VAE introduces: (1) dynamic contextualized embeddings that adapt to semantic drift and structural variability, (2) a loss-conjunction strategy for distribution reconstruction with uncertainty-sensitiveness, which enhances the preservation of local patterns and probabilistic consistency, and (3) a dual-criterion anomaly scoring framework that unifies reconstruction fidelity and KL-divergence-driven deviation to detect subtle departures from learned contextual distributions. Evaluations on six benchmark datasets demonstrate that TAD-VAE significantly outperforms seven state-of-the-art algorithms, including Isolation Forest, BERT-based detectors, Deep SVDD, and VAEs. Specifically, TAD-VAE achieves substantial improvements in the F1-Score, Precision, Recall, and AUC-ROC compared to the average performance of the benchmarking models across all datasets. These results highlight the robustness and uncertainty-aware adaptability of TAD-VAE in detecting subtle, context-dependent anomalies across diverse textual domains.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Transformer-vae for contextual anomaly detection in text

  • 期刊论文

  • 发表日期：

    2026年3月7日

查看全文
  • Lazhar Farek,
  • Amira Benaidja

摘要

Effective text anomaly detection is crucial given the explosion of textual data across domains like cybersecurity and social media. Current methods struggle with long-range semantics, contextual nuances, high dimensionality, computational bottlenecks for long sequences, and real-time scalability. We propose TAD-VAE, a novel architecture combining Transformer-based contextual encoding with Variational Autoencoders (VAEs), enhanced by hybrid sparse attention for efficient long-range context modeling. TAD-VAE introduces: (1) dynamic contextualized embeddings that adapt to semantic drift and structural variability, (2) a loss-conjunction strategy for distribution reconstruction with uncertainty-sensitiveness, which enhances the preservation of local patterns and probabilistic consistency, and (3) a dual-criterion anomaly scoring framework that unifies reconstruction fidelity and KL-divergence-driven deviation to detect subtle departures from learned contextual distributions. Evaluations on six benchmark datasets demonstrate that TAD-VAE significantly outperforms seven state-of-the-art algorithms, including Isolation Forest, BERT-based detectors, Deep SVDD, and VAEs. Specifically, TAD-VAE achieves substantial improvements in the F1-Score, Precision, Recall, and AUC-ROC compared to the average performance of the benchmarking models across all datasets. These results highlight the robustness and uncertainty-aware adaptability of TAD-VAE in detecting subtle, context-dependent anomalies across diverse textual domains.