Applying Contrastive Learning to an Attention Neural Model in a Multilingual Context
摘要
This research contributes to the field of Aspect-Based Sentiment Classification (ABSC) of Web data by proposing new cross-, multi- and unilingual ASBC models. We do this by improving the state-of-the-art mLCR-Rot-hop++ attention neural model and its variations. We introduce different multilingual XLM-R embedders to replace the multilingual BERT (mBERT) embedder found within the mLCR-Rot-hop++ model. Furthermore, we add two distinct contrastive learning methods to the existing mLCR-Rot-hop++ model. The first approach integrates sentiment-level contrastive learning, adapted to instances rather than individual token embeddings, into the mLCR-Rot-hop++ model. Our second approach considers the high-level opinion representations of the mLCR-Rot-hop++ model within the contrastive loss function. Our findings indicate that replacing the mBERT embedder with an XLM-R \(_\text {base}\) embedder generally improves performance. Furthermore, sentiment-level contrastive learning usually improves the performance of various models, especially compared to representation-level contrastive learning.