The rapid advancement of recent transformer-based Neural Machine Translation models has increased the demand for large corpora. Although Vietnamese parallel corpora with widely used languages such as English and Chinese are well-developed, historical texts in traditional Chinese remain underexplored. In this study, we investigate approaches for constructing a parallel corpus between the Vietnamese National script and ancient Chinese without punctuation, using the Nguyễn dynasty history record “Đại Nam Thực Lục Tiền Biên”. We apply OCR to images of the document and process the obtained texts into meaningful sentences. We propose a novel phrase-matching algorithm, leveraging Sino-Vietnamese transcription, to tackle the alignment task and compare the results with a BertAlign approach. We evaluate sentence segmentation using a pre-punctuated Chinese dataset and alignment using LaBSE-based similarity scores. BertAlign method achieves an average similarity score of 0.4023, while phrase-matching algorithm yields scores of 0.4246, offering a promising alternative without multilingual models.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Parallel Corpus Construction for Chinese and Vietnamese in Historical Texts

  • Minh-Nhut Dang,
  • Thanh-Hau Cao,
  • Thanh-Duy Lam,
  • Minh-Hoang Le,
  • Duc-Nhuan Le,
  • Si-Dien Dinh

摘要

The rapid advancement of recent transformer-based Neural Machine Translation models has increased the demand for large corpora. Although Vietnamese parallel corpora with widely used languages such as English and Chinese are well-developed, historical texts in traditional Chinese remain underexplored. In this study, we investigate approaches for constructing a parallel corpus between the Vietnamese National script and ancient Chinese without punctuation, using the Nguyễn dynasty history record “Đại Nam Thực Lục Tiền Biên”. We apply OCR to images of the document and process the obtained texts into meaningful sentences. We propose a novel phrase-matching algorithm, leveraging Sino-Vietnamese transcription, to tackle the alignment task and compare the results with a BertAlign approach. We evaluate sentence segmentation using a pre-punctuated Chinese dataset and alignment using LaBSE-based similarity scores. BertAlign method achieves an average similarity score of 0.4023, while phrase-matching algorithm yields scores of 0.4246, offering a promising alternative without multilingual models.