<p>Skin cancer is one of the most common cancers worldwide where early detection is crucial for effectively diagnosing and treatment. Traditional diagnostic methods largely depend on dermoscopic or skin lesion image analysis. However, they are limited by the clinician’s expertise and primarily rely on image data. While most of the studies focus on improving the diagnostic accuracy using the skin lesion images, this study addresses the limitations of using just the image data and proposes a novel approach that enhances skin cancer detection by combining skin lesion images with patients’ clinical notes. The methodology proposes a multi-modal approach that combines the images data with doctor’s clinical notes to enhance early diagnosis and accurate detection of skin cancer. Clinical notes are synthetically generated using two distinct multimodal large language models (LLMs), corresponding to each patient’s skin cancer images, encapsulating both visual and textual data representative of real-world scenarios. The generated clinical notes are validated through a dual-method approach involving cross-evaluation and consensus scoring, utilizing metrics such as the BLEU score, ROUGE score, Overlap Coefficient, and Jaccard index. Furthermore, this paper compares the performance of a model built using skin lesion images with four other models built using skin lesion images and synthetically generated clinical notes. The experimental results demonstrates a significant improvement in all classification metrics compared to single-modality models, achieving an accuracy of 99.51%, precision of 96.19%, recall of 97.95%and f1-score of 97.03% with the multimodal ALBEF model that uses GPT-4-turbo for clinical notes generation. The results show a significant improvement in performance with the multi modal approach for medical diagnosis. Also, this research sets a foundational framework that can be leveraged for many potential healthcare diagnoses applications.</p>