Skin cancer diagnosis plays a pivotal role in early intervention and treatment. This study presents an approach for skin cancer classification, leveraging a fusion of two distinct models and optimizing the accuracy and robustness of the classification process. The first model employs an optimized convolutional network designed for skin lesion classification. A customized loss function is introduced, combining weighted cross entropy and focal loss. This unique loss function enhances the model’s ability to focus on challenging samples, improving overall performance. The second model integrates a recurrent neural network (RNN) utilizing metadata associated with skin lesions. Additional information enhances the model’s capacity to capture nuanced patterns and relationships within the data. The fusion of these two models’ outputs is further processed through a fully connected neural network (FCNN) for final classification. The HAM10000 (Human Against Machine 10000) dataset serves as the experimental foundation for this work, incorporating multimodal imaging data. The proposed method achieves outstanding results with an accuracy of 96.96%, precision of 97%, recall of 97%, and an F1-score of 97%. The combined strength of the convolutional network’s optimized architecture, the customized loss function, and the RNN’s ability to utilize metadata collectively contribute to the high-performance metrics in skin cancer classification.