CHOLD: Offensive Language Detection Method for Chinese Texts
摘要
Offensive language detection is vital for purifying cyberspace and ensuring the efficient and orderly operation of pretrained language models. Traditional methods of offensive language detection rely on a single matching mechanism, often neglecting the influence of semantic bias on the detection outcomes. This limitation leads to a low accuracy rate and constitutes a significant threat to cyberspace security. To address this issue, this paper proposes a high-accuracy Chinese offensive language detection method named CHOLD. CHOLD integrates various language features, including Chinese character shape, pronunciation, and meaning. Initially, the method constructs diverse lists of offensive language keywords for semantically matching target sentences. The matching words are subsequently added to the keyword list, forming the offensive language candidate set. The high-dimensional features related to the character shape, pronunciation, and meaning are then fused after the dimensionality of the sentences within the candidate set is reduced. Additionally, a semantic constraint method is employed to minimize the impact of semantic deviation on the detection results. Experimental results conducted on multiple public datasets demonstrate that CHOLD outperforms previous detection models in terms of recognition accuracy.