Scholarlink

<p>This paper introduces a Transformer-based framework for Persian intent classification and slot filling, tailored for low-resource settings. The model integrates Multi-Head Attention, RMSNorm, and SwiGLU to balance accuracy with efficiency. Using an HPC-accelerated pipeline, a large teacher model is first trained, followed by hard sample mining to tackle class imbalance. Knowledge distillation then transfers learning to a compact student model, reducing parameters by ~ 75% with minimal performance loss. Experiments confirm the approach enables robust, real-time NLU in imbalanced data scenarios, demonstrating that intensive parallelized training can yield efficient, deployable models.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An effective knowledge-distilled transformer for intent detection and slot filling in Persian language

  • 期刊论文

  • 发表日期：

    2026年4月22日

查看全文
  • Saeed Biabani Mahalli,
  • Jamshid Pirgazi,
  • Ali Ghanbari Sorkhi

摘要

This paper introduces a Transformer-based framework for Persian intent classification and slot filling, tailored for low-resource settings. The model integrates Multi-Head Attention, RMSNorm, and SwiGLU to balance accuracy with efficiency. Using an HPC-accelerated pipeline, a large teacher model is first trained, followed by hard sample mining to tackle class imbalance. Knowledge distillation then transfers learning to a compact student model, reducing parameters by ~ 75% with minimal performance loss. Experiments confirm the approach enables robust, real-time NLU in imbalanced data scenarios, demonstrating that intensive parallelized training can yield efficient, deployable models.