An effective convolutional cross-lingual language model for Vietnamese online social media mining
摘要
With the fast expansion rate of online social media platforms, tons and tons of user-generated content, such as comments, blogs, or messages, are uploaded or sent every hour. This makes monitoring the virtual space impossible to do manually. Machine learning models can be used to integrate sentiment analysis, emotion recognition, and spam detection into business applications. While recent studies are mostly centered on how pre-trained models or particular methods are used to improve the accuracy and the classification of comments on Vietnamese social media, there are not many studies directed explicitly at how their effectiveness can be improved. Additionally, there is a limited availability of models for the Vietnamese language to solve the problem of categorizing social media comments. Moreover, the series of studies focused on low-resource languages has yet to address many challenges in such languages as Vietnamese. In addition to this, we face additional challenges as code-mixed commentaries begin to appear on Vietnamese social media platforms. Extracting relevant features from short social media comments may give the models a better understanding and classification capabilities over time. To address these problems, we present a strategy that leverages the cross-lingual language model with 1D-CNN layers for social media text processing. In addition, we propose a sentiment classifier to effectively capture and represent relevant features for classification and a custom loss function to deal with data imbalance. In addition, we also created the first dataset (ViCM) related to code-mixed Vietnamese, which contains almost 5415 code-mixed hate speech comments. Our method has demonstrated significant improvements over previous studies in these NLP tasks, achieving state-of-the-art performance on benchmark social media datasets such as VSMEC, VSFC, and ViSpam. Additionally, we have made our ViCM dataset(ViCM dataset repository: https://github.com/tiennho2608/ViCM) publicly available to support further research and advancements in this field.