Context-Aware Arabic Diacritization Using Transformers
摘要
The presence of diacritics plays a crucial role in representing the meaning and pronunciation of Arabic words and sentences accurately. Over the years, researchers have dedicated considerable efforts to enhance automated diacritization systems. However, to our knowledge, Bidirectional Encoder Representations from Transformers (BERT) models were not yet investigated for the full diacritization approach. This study introduces a novel approach that utilizes transfer learning and BERT for diacritizing Arabic sentences. To show the effectiveness of the proposed approach, this research paper utilizes the Arabic Diacritization (AD) benchmark dataset to compare the constructed BERT model with the most recent research using error metrics. The research findings demonstrate the significance of the proposed approach in improving the accuracy of diacritization. The trained model outperformed all previous techniques and achieved state-of-the-art (SOTA) results in all error metrics. The resulting syntactic DER and WER are 1.14% and 3.35%, respectively. In morphological diacritization, the best results showed a DER of 0.92% and a WER of 1.91%. These results represent a significant relative error reduction of 35% on average compared to previous research.