Recent advances in neural machine translation(NMT) have substantially increased the power of machine translation(MT); however it is still problematic to evaluate the translation quality of low-resource language pairs, like English to Kannada. The four most common MT systems, Google Translate, Microsoft Translator, IndicTrans2, and NLLB, will be compared in this study that will look at translation problems and language performance and domain-specific delicacies. To measure the quality of translation, the following metrics are selected, BiLingual Evaluation Understudy (BLEU), Metric for Evaluation of Translation with Explicit Ordering(METEOR), Character n-gram F-score (ChrF), Translation Edit Rate (TER), and Cross-lingual Optimized Metric for Evaluation of Translation (COMET) are selected, and they jointly evaluate lexical-level accuracy, semantic correctness, fluidity, and percentage of errors. The results point out that IndicTrans2 recorded the best performance in the translation of Kannada-English, with BLEU(21.2), METEOR(0.4919), and COMET(0.8399). NLLB shows worse than the actual results, with less BLEU(18). When comparing English-Kannada translation, Microsoft Translator has the best results in BLEU(9.1) and COMET(0.8589), but IndicTrans2 shows similar statistics in a number of measures. The NLLB ranks poorly in the majority of measures with the same points on low-resource English-Kannada MT. Taken together, the findings indicate the sensitivity of linguistic diversity, domain variation, and data scarcity to quality of translation as well as the need to specify forms of strengthening to enhance their reliability and inclusiveness, respectively. The insights offer a basis upon which future English-Kannada and Kannada-English MT systems could be developed, hence enhancing communication and broader access to languages in most sectors.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Machine Translation in Kannada-English and English-Kannada: A Performance and Quality Assessment

  • Yathish Poojary,
  • Dhanusha,
  • B. Ashwath Rao,
  • Musica Supriya,
  • V. G. Narendra

摘要

Recent advances in neural machine translation(NMT) have substantially increased the power of machine translation(MT); however it is still problematic to evaluate the translation quality of low-resource language pairs, like English to Kannada. The four most common MT systems, Google Translate, Microsoft Translator, IndicTrans2, and NLLB, will be compared in this study that will look at translation problems and language performance and domain-specific delicacies. To measure the quality of translation, the following metrics are selected, BiLingual Evaluation Understudy (BLEU), Metric for Evaluation of Translation with Explicit Ordering(METEOR), Character n-gram F-score (ChrF), Translation Edit Rate (TER), and Cross-lingual Optimized Metric for Evaluation of Translation (COMET) are selected, and they jointly evaluate lexical-level accuracy, semantic correctness, fluidity, and percentage of errors. The results point out that IndicTrans2 recorded the best performance in the translation of Kannada-English, with BLEU(21.2), METEOR(0.4919), and COMET(0.8399). NLLB shows worse than the actual results, with less BLEU(18). When comparing English-Kannada translation, Microsoft Translator has the best results in BLEU(9.1) and COMET(0.8589), but IndicTrans2 shows similar statistics in a number of measures. The NLLB ranks poorly in the majority of measures with the same points on low-resource English-Kannada MT. Taken together, the findings indicate the sensitivity of linguistic diversity, domain variation, and data scarcity to quality of translation as well as the need to specify forms of strengthening to enhance their reliability and inclusiveness, respectively. The insights offer a basis upon which future English-Kannada and Kannada-English MT systems could be developed, hence enhancing communication and broader access to languages in most sectors.