Hybrid Transformer-ANFIS Architecture for Sentiment Analysis
摘要
Sentiment analysis is needed for businesses to understand and analyze customer feedback. Transformer models like RoBERTa and ALBERT represent the state-of-the-art in sentiment analysis. Their key limitation is that their standard usage typically relies only on the maximum predicted probability, discarding key information embedded within the full output probability distribution. This paper investigates enhancing transformer-based sentiment analysis by integrating Adaptive Neuro-Fuzzy Inference Systems (ANFIS) to model predictions based on six scores derived from probability distributions. We fine-tune RoBERTa and ALBERT on the yelp review dataset. Then, we extract six features characterizing prediction confidence, entropy, and potential error indicators from their probability outputs. These features serve as inputs to ANFIS models, which are trained to predict the likelihood of the baseline prediction being correct. We evaluate two strategies for combining the baseline predictions using the ANFIS trust scores: a threshold-based selection method and a trust-weighted averaging method. Evaluations compare the hybrid system against the individual baseline models, showing that ANFIS improves performance. This work contributes a novel methodology for integrating transformer probability outputs with ANFIS for trust assessment, a hybrid architecture enhancing sentiment analysis robustness, and an empirical validation with performance improvements.