Evaluating Lexicon-Based and Transformer-Based Approaches for Sentiment Analysis in Amazon Fine Food Reviews
摘要
Sentiment analysis, a key task in natural language processing (NLP), identifies emotional tone in text. This study compares two sentiment classification approaches: a lexicon-based method using VADER (Valence Aware Dictionary and sEntiment Reasoner) and a transformer-based deep learning method with RoBERTa (Robustly Optimized BERT Pretraining Approach). Using the Amazon Fine Food Reviews dataset of 568,454 customer reviews, the analysis categorizes sentiments as positive or negative. Preprocessing steps, including text normalization and handling missing values, ensure data reliability. VADER efficiently processes short, informal texts using a predefined lexicon but struggles with complex linguistic structures and contextual subtleties. RoBERTa leverages transformer-based architectures to capture intricate word relationships, enabling superior accuracy and nuanced sentiment detection in contextually rich texts. A comparative evaluation demonstrates that RoBERTa outperforms VADER by a significant margin, underscoring the strengths of deep learning for detailed sentiment analysis. These findings emphasize the trade-offs between speed and contextual depth in sentiment analysis models and provide valuable insights for customer feedback interpretation, opinion mining, and broader NLP research.