Sentiment analysis has numerous applications across various domains such as Social Media Monitoring, Customer Feedback Analysis, Market Research, Financial Markets, Product Reviews, Political Analysis, Employee Feedback, Healthcare, Tourism, and Education. Industries use sentiment analysis to develop informing policy and product design for wider acceptance and improved revenue. Classification methods, from keyword distribution to advanced models like Recurrent Neural Networks and Transformers, have been employed to categorize sentiments into Positive, Negative, and Neutral. Transformer-based models like OpenAI’s chatGPT (a chatbot using Generative Pretrained Transformer (GPT)-based Large Language Model) and Google’s BARD (a chatbot based on Bidirectional Encoder Representations from Transformers (BERT) framework) struggle with local context and bias issues. In this paper, we combine Latent Semantic Analysis (LSA) with Continuous Bag of Words (CBOW), GPT and BERT features to predict the sentiments, addressing local context to enhance the model. We name our proposed method as SGPT. The proposed SGPT method demonstrates significant improvement over state-of-the-art sentiment classification approaches on benchmark finance sentiment datasets.