During the past century, picture captioning has made tremendous progress, which involves writing textual descriptions for images. In addition to being designed initially to analyze natural language inputs, these transformers have also been modified to analyze visual inputs, providing a path toward multimodal understanding. A new approach combining the benefits of vision transformers and swing transformers is proposed in this study for image captioning jobs. The Swin Transformer is best at hierarchical processing and capturing multi-scale features, while the Vision Transformer is excellent at capturing global dependencies. With Swin Transformers incorporated into our model, we can effectively evaluate visual content at a variety of abstraction levels more efficiently. We integrate visual characteristics into the transformer architecture in order to allow the learning of visual and textual representations at the same time. To communicate effectively between the visual and linguistic modalities, we also encode textual descriptions associated with images using a GPT2 tokenizer. Using benchmark datasets, we demonstrate the effectiveness of our methodology and obtain competitive results.