Since sentiment analysis has been a crucial field for various types of studies, this field needs to develop and improvise its efficiency and accuracy. The problem has been analyzed over time in many different ways, and multiple approaches have been used to solve it. So far, we haven’t utilized the combined power of modern models and contextual information obtained from texts, nor has it analyzed how contextual information could be effectively controlled. By combining deep learning models with rule-based techniques, the proposed approach effectively classifies sentiments for a range of topics. Remarkably, it uses an embedded representation and attention mechanism to process valence shifting cases efficiently. The method, that employs sentiment dictionaries related to specific domains and custom-designed rules, shows better results on three datasets. Probability Proportion Difference (PPD) and Categorical Probability Proportion Difference (CPPD) are used in feature selection, these methods outperformed Information Gain and the standard method. Comparable sentiment detection abilities are revealed through a comparative analysis with Categorical Probability Difference (CPD). Tests performed on datasets shows the superior sentiment classification performance of the CPPD method over other methods.