The loss of endangered languages is a severe threat to cultural heritage. Machine translation holds significant promise for documenting and preserving these languages. However, low-resource languages, such as Tujia, which lacks both parallel corpora and a standardized written form, pose substantial challenges for machine translation. In order to explore machine translation methods for low-resource language without written text, this paper takes Tujia as an example. First, we analyzed the grammatical and lexical correlations between Tujia and Chinese, constructing a Tujia dependency parsing treebank and structuring parsing knowledge. Second, in view of the low-resource and non-literate characteristics of Tujia, we applied knowledge transferred from Large Language Models (LLMs), designed prompt engineering, and fine-tuned the Llama and ChatGLM models. Dynamic dataset partitioning is employed, and translation quality is assessed using Translation Edit Rate (TER). Results indicated that the structured fine-tuning approach based on ChatGLM performed best, with LoRA fine-tuning yielding the most effective results. Incorporating dependency parsing rule into the structured instruction set significantly outperformed unstructured instruction fine-tuning, effectively mitigating the loss of grammatical information in unwritten languages and enhancing translation quality. Finally, we utilized the Llama. cpp framework to accelerate inference and improve model performance and practicality.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Fine-Tuning for Low-Resource Language Machine Translation Using Large Language Models Integrated with Dependency Parsing Rule

  • Chongchong Yu,
  • Yan Wu,
  • Zheng Ruan,
  • Zhaopeng Qian

摘要

The loss of endangered languages is a severe threat to cultural heritage. Machine translation holds significant promise for documenting and preserving these languages. However, low-resource languages, such as Tujia, which lacks both parallel corpora and a standardized written form, pose substantial challenges for machine translation. In order to explore machine translation methods for low-resource language without written text, this paper takes Tujia as an example. First, we analyzed the grammatical and lexical correlations between Tujia and Chinese, constructing a Tujia dependency parsing treebank and structuring parsing knowledge. Second, in view of the low-resource and non-literate characteristics of Tujia, we applied knowledge transferred from Large Language Models (LLMs), designed prompt engineering, and fine-tuned the Llama and ChatGLM models. Dynamic dataset partitioning is employed, and translation quality is assessed using Translation Edit Rate (TER). Results indicated that the structured fine-tuning approach based on ChatGLM performed best, with LoRA fine-tuning yielding the most effective results. Incorporating dependency parsing rule into the structured instruction set significantly outperformed unstructured instruction fine-tuning, effectively mitigating the loss of grammatical information in unwritten languages and enhancing translation quality. Finally, we utilized the Llama. cpp framework to accelerate inference and improve model performance and practicality.