Detecting sarcasm in social media content has been one of the challenging and interesting tasks in social network analytics due to users’ growing creativity, humor, playfulness, and the complexity of the input data. Today, social media users can employ various methods to convey their thoughts, information, or content uniquely, combining captions (text) and images in complex and nuanced ways. Most existing sarcasm detection models are limited because they focus solely on single modalities (text or image). However, most social media content includes captions and pictures created by users. This paper proposes three approaches to detecting sarcasm from various perspectives. We leverage the relationship between textual and imaging data and utilize state-of-the-art, pre-trained large language models for extracting relevant features and improving the performance of the main problem with labels categorized into four types: multi-sarcasm, non-sarcasm, text-sarcasm, and image-sarcasm. We also provide a deeper level of analysis and discuss potential challenges for more accurate sarcasm detection. This work is one of the attempts to address the multiclass classification of sarcasm detection on Vietnamese social platforms.