Scholarlink

<p>With the growing demand for cross-lingual real-time communication in smart tourism scenarios, the trade-off between response speed and translation quality in voice translation systems has become increasingly sensitive. Centering on this issue, this study proposes a Multimodal Fusion Transformer Transducer (MF-TT) model for real-time applications and introduces a Transformer-based fusion network into the overall architecture. Meanwhile, it incorporates a segment masking mechanism and a latency constraint loss to adapt to the non-monotonic alignment characteristics between speech and text in translation tasks. Experiments are conducted on the English–Spanish and English-German tasks of the Multilingual Speech Translation Corpus (MuST-C) dataset, as well as the Baidu Speech Translation Corpus (BSTC) Chinese-English dataset. Results show that MF-TT achieves a Bilingual Evaluation Understudy (BLEU) score of 24.32, an Average Proportion (AP) of 0.71, and an Average Lagging (AL) of 1230.8 in the English–Spanish direction. In the English-German direction, this model reaches a BLEU score of 17.64 and an AL of 1466.3, with the overall lag level lower than that of multiple comparative models. On the BSTC Chinese-English dataset, the MF-TT model’s BLEU, AP, and AL are 23.02, 0.721, and 1298.4, respectively. AL significantly outperforms other comparative models, p &lt; 0.05. The ablation experiment further verified the independent contributions of fusion network structure, latency loss function, and segment masking mechanism to real-time performance and translation stability. The results show that the proposed method has good performance and application potential in low-latency cross-lingual real-time interaction scenarios. At the same time, it has a certain reference value in scenarios such as smart tourism that require real-time voice translation.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

MF-TT model for real-time applications in smart tourism English translation under transformer-based fusion network

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月3日

查看全文
  • Shihui Zhu,
  • Yan Zhang

摘要

With the growing demand for cross-lingual real-time communication in smart tourism scenarios, the trade-off between response speed and translation quality in voice translation systems has become increasingly sensitive. Centering on this issue, this study proposes a Multimodal Fusion Transformer Transducer (MF-TT) model for real-time applications and introduces a Transformer-based fusion network into the overall architecture. Meanwhile, it incorporates a segment masking mechanism and a latency constraint loss to adapt to the non-monotonic alignment characteristics between speech and text in translation tasks. Experiments are conducted on the English–Spanish and English-German tasks of the Multilingual Speech Translation Corpus (MuST-C) dataset, as well as the Baidu Speech Translation Corpus (BSTC) Chinese-English dataset. Results show that MF-TT achieves a Bilingual Evaluation Understudy (BLEU) score of 24.32, an Average Proportion (AP) of 0.71, and an Average Lagging (AL) of 1230.8 in the English–Spanish direction. In the English-German direction, this model reaches a BLEU score of 17.64 and an AL of 1466.3, with the overall lag level lower than that of multiple comparative models. On the BSTC Chinese-English dataset, the MF-TT model’s BLEU, AP, and AL are 23.02, 0.721, and 1298.4, respectively. AL significantly outperforms other comparative models, p < 0.05. The ablation experiment further verified the independent contributions of fusion network structure, latency loss function, and segment masking mechanism to real-time performance and translation stability. The results show that the proposed method has good performance and application potential in low-latency cross-lingual real-time interaction scenarios. At the same time, it has a certain reference value in scenarios such as smart tourism that require real-time voice translation.