An External Knowledge Enhanced Multimodal Method for Improving Chinese Spelling Correction
摘要
Chinese Spelling Correction (CSC) focuses on detecting and correcting spelling errors in Chinese text. It is particularly valuable for social media, where informal and rapid communication often leads to spelling mistakes that can cause misunderstandings and hinder effective interaction. While state-of-the-art models utilize BERT-based and GPT models to directly correct each character in an input sentence, they heavily depend on training data and struggle to keep pace with the rapid changes in social media. Moreover, current methods lack an effective supervision mechanism, leading to output sentences that are wrong or deviate from the original meaning. To solve these problems, we propose a post-processing multimodal approach, including a correction augmentation module and an assessor module. Initially, we enrich the predicted corrections from existing CSC models with external knowledge sources, such as obfuscated datasets and topic-related information from search engines like Google or Baidu, effectively supplementing gaps in the training data. Subsequently, an assessor module evaluates these augmented corrections across sentence perplexity, pinyin edit distance, and image similarity to select the optimal correction combination. To the best of our knowledge, we are the first to propose a multimodal assessor for the CSC task. Experimental results demonstrate that our proposed method significantly enhances existing CSC models across three distinct datasets.