Transformer-based end-to-end neural machine translation (NMT) models have demonstrated promising results by leveraging attention mechanisms to acquire translation knowledge from bilingual data. However, such models often overlook direct modeling of syntactic information, there-by failing to exploit the underlying linguistic knowledge effectively. This limitation becomes particularly evident in NMT tasks with low-resource, where the paucity of bilingual data leads to a significant decline in translation performance. To alleviate this issue, a new self-attention method based on dependency distance is proposed. By explicitly integrating dependency distances derived from the source sentence into the model, our approach guides the attention learning process, enabling more accurate representation of the source sentence and subsequently improving the translation quality. Specifically, our methodology involves three steps: (1) performing depend-ency analysis to compute the dependency distances between words in the source sentence, (2) constructing a dependency distance matrix based on these distances, and (3) integrating this matrix into the NMT model’s encoder. Experimental results on various low-resource translation tasks demonstrate that our proposed method not only outperforms the baseline model but also surpasses other syntactic-focused approaches, thereby validating its effectiveness.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Improving Low-Resource Neural Machine Translation with Dependency Distance-Based Self-Attention

  • Hong Yan,
  • Fuxue Li,
  • Yongfu Chen,
  • Chuncheng Chi,
  • Peiju Xie

摘要

Transformer-based end-to-end neural machine translation (NMT) models have demonstrated promising results by leveraging attention mechanisms to acquire translation knowledge from bilingual data. However, such models often overlook direct modeling of syntactic information, there-by failing to exploit the underlying linguistic knowledge effectively. This limitation becomes particularly evident in NMT tasks with low-resource, where the paucity of bilingual data leads to a significant decline in translation performance. To alleviate this issue, a new self-attention method based on dependency distance is proposed. By explicitly integrating dependency distances derived from the source sentence into the model, our approach guides the attention learning process, enabling more accurate representation of the source sentence and subsequently improving the translation quality. Specifically, our methodology involves three steps: (1) performing depend-ency analysis to compute the dependency distances between words in the source sentence, (2) constructing a dependency distance matrix based on these distances, and (3) integrating this matrix into the NMT model’s encoder. Experimental results on various low-resource translation tasks demonstrate that our proposed method not only outperforms the baseline model but also surpasses other syntactic-focused approaches, thereby validating its effectiveness.