Enhanced Twitter Sentiment Analysis with NLTK and Transformer Models
摘要
The rapid growth of social media platforms has led to a substantial increase in user-generated content, offering valuable insights into social discourse and public opinion. This work, “Enhanced Twitter Sentiment Analysis with NLTK and Transformer Models,” aims to improve sentiment analysis on Twitter by enhancing precision and contextual understanding. Our hybrid approach leverages the text preprocessing and tokenization capabilities of the Natural Language Toolkit (NLTK) alongside the contextual strengths of transformer-based models, such as RoBERTa, compared against traditional methods like Naïve Bayes models. Using NLTK, we preprocess Twitter data through steps like tokenization, stop-word removal, and lemmatization to standardize the text. We then employ VADER as a baseline for sentiment analysis, followed by RoBERTa for a more nuanced, context-aware sentiment classification. Comparative evaluation shows that RoBERTa surpasses traditional methods, achieving higher accuracy, precision, and recall. These results underscore RoBERTa’s ability to capture intricate language patterns, making it a powerful tool for sentiment analysis applications in fields such as market research and public opinion tracking. Future work will focus on deploying this framework in real-time environments, addressing challenges such as computational efficiency and latency to enable live sentiment monitoring.