An integrated framework for emotion and sentiment analysis in Tamil and Malayalam visual content
摘要
Sentiment analysis in low-resource languages such as Tamil and Malayalam presents significant challenges due to the scarcity of linguistic resources and the intricacy of cultural contexts. To address these challenges, we propose the Integrated Multimodal Sentiment Dynamics (IMSD) Framework, an innovative solution specifically tailored for analyzing Tamil and Malayalam visual content. The framework is novel in its integration of visual, audio, and textual modalities, employing advanced feature extraction technologies such as I3D and mBERT for innovative feature fusion and a cultural adaptation layer to ensure sensitivity and appropriateness to regional nuances. Utilizing the DravidianMultiModality dataset, consisting of 1340 multimedia samples, IMSD significantly outperforms traditional models like Naïve Bayes, SVM, and LSTM, achieving impressive metrics with an accuracy of 86.3%, precision of 87.5%, and recall of 86.0%. Notably, the framework exhibits exceptional performance in cross-linguistic adaptability, demonstrated through its successful application to the Bengali MemoSen dataset, enhancing its potential applicability across diverse linguistic contexts. These findings underline IMSD’s capability to set new benchmarks in multimodal sentiment analysis for under-resourced languages, highlighting its adaptability and the critical role of cultural understanding in sentiment analysis.