Scholarlink

<p>Pre-ordering is a data pre-processing technique that reorganizes the word order in a source language sentence to better align with the syntax of the target language, and it has been shown to significantly impact various tasks. While previous studies have primarily leveraged token position embeddings in pre-ordered sentences to enhance machine translation, they have not directly explored learning contextualized representations that encapsulate richer semantic information and also reflect the word position information in the pre-ordered sentences. In this work, we introduce a novel pre-ordering-aware neural network that explicitly integrates the representations of pre-ordered sentences into the training process. Our network utilizes a pre-ordering encoder, which works alongside the original Transformer encoder to process the pre-ordered source sentence. We propose a Cross-Encoder Consistency (CEC) block, which encourages the original encoder to produce representations that better reflect the word order of the target language by closing the gap between representations of the original and pre-ordered sentence. Additionally, we incorporate a Sentence Encoding Consistency (SEC) block to help the model preserve the semantic integrity of the source sentence. We conduct extensive experiments on multiple low-resource neural machine translation benchmarks, including an endangered language, the Māori language. Our method delivers substantial enhancements in translation quality, achieving improvements of up to +1.46 BLEU in Tr <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\rightarrow\)</EquationSource> </InlineEquation> En translation.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Pre-ordering representations improve low-resource neural machine translation and application in the Māori language

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年1月22日

查看全文
  • Yuan Gao,
  • Feng Hou,
  • Huia Jahnke

摘要

Pre-ordering is a data pre-processing technique that reorganizes the word order in a source language sentence to better align with the syntax of the target language, and it has been shown to significantly impact various tasks. While previous studies have primarily leveraged token position embeddings in pre-ordered sentences to enhance machine translation, they have not directly explored learning contextualized representations that encapsulate richer semantic information and also reflect the word position information in the pre-ordered sentences. In this work, we introduce a novel pre-ordering-aware neural network that explicitly integrates the representations of pre-ordered sentences into the training process. Our network utilizes a pre-ordering encoder, which works alongside the original Transformer encoder to process the pre-ordered source sentence. We propose a Cross-Encoder Consistency (CEC) block, which encourages the original encoder to produce representations that better reflect the word order of the target language by closing the gap between representations of the original and pre-ordered sentence. Additionally, we incorporate a Sentence Encoding Consistency (SEC) block to help the model preserve the semantic integrity of the source sentence. We conduct extensive experiments on multiple low-resource neural machine translation benchmarks, including an endangered language, the Māori language. Our method delivers substantial enhancements in translation quality, achieving improvements of up to +1.46 BLEU in Tr \(\rightarrow\) En translation.