Scholarlink

<p>Skin lesion identification and treatment is a critical and challenging task in the healthcare sector. Usually, several CNN classifiers have been applied to identify skin diseases using only simple visual information. However, in this context, recent research has focused on multimodalities. Since different modalities contain unique information their combination represents the underlying concepts more clearly, increasing the accuracy of the model. This study proposes a multimodal architecture integrating two methods for fusing image and metadata features. One is, Vision Transformers (ViT) for extracting visual characteristics from images, and the other is an MLP (Multilayer perceptron) for the metadata information. Following pre-training on massive volumes of data, the ViT surpasses cutting-edge convolutional networks in various benchmarks while using fewer computational resources to train and achieve better performance. Furthermore, to avoid the problems of discovering a high volume of labeled data, the fuzziness-based semi-supervised deep learning (FSSDL) method is applied. Using low-quality fuzzy samples for retraining classifiers has been found to enhance the performance of the classifier. The integrated model was applied to the PAD-UFES_20 dataset, containing both image and metadata. In order to strengthen this research, the dataset was classified for binary and multiclass classification. The model’s performance was assessed using various performance metrics (e.g. <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(F_1\)</EquationSource> </InlineEquation>-score and G-mean), as well as by estimating the model’s time complexity, optimality, and completeness. The findings indicate that the model has shown significant advancement in enhancing the classifier’s performance in comparison to other cutting-edge approaches, for both binary and multiclass classification tasks. Our proposed model achieved significant improvements in both binary and multiclass skin lesion classification tasks. For binary classification, the F1-score reached 0.92, while the GMean achieved 0.93. In multiclass classification, the model’s <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(F_1\)</EquationSource> </InlineEquation>-score and Gmean were 0.89 and 0.90, respectively.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Vision transformer-based fuzzy semi-supervised multimodal learning for skin lesion image classification

  • 期刊论文

  • 发表日期：

    2026年1月21日

查看全文
  • Israt Yasmin,
  • Suriya Sultana,
  • Subrina Akter,
  • Weipeng Cao,
  • M. Jamshed Alam Patwary

摘要

Skin lesion identification and treatment is a critical and challenging task in the healthcare sector. Usually, several CNN classifiers have been applied to identify skin diseases using only simple visual information. However, in this context, recent research has focused on multimodalities. Since different modalities contain unique information their combination represents the underlying concepts more clearly, increasing the accuracy of the model. This study proposes a multimodal architecture integrating two methods for fusing image and metadata features. One is, Vision Transformers (ViT) for extracting visual characteristics from images, and the other is an MLP (Multilayer perceptron) for the metadata information. Following pre-training on massive volumes of data, the ViT surpasses cutting-edge convolutional networks in various benchmarks while using fewer computational resources to train and achieve better performance. Furthermore, to avoid the problems of discovering a high volume of labeled data, the fuzziness-based semi-supervised deep learning (FSSDL) method is applied. Using low-quality fuzzy samples for retraining classifiers has been found to enhance the performance of the classifier. The integrated model was applied to the PAD-UFES_20 dataset, containing both image and metadata. In order to strengthen this research, the dataset was classified for binary and multiclass classification. The model’s performance was assessed using various performance metrics (e.g. \(F_1\) -score and G-mean), as well as by estimating the model’s time complexity, optimality, and completeness. The findings indicate that the model has shown significant advancement in enhancing the classifier’s performance in comparison to other cutting-edge approaches, for both binary and multiclass classification tasks. Our proposed model achieved significant improvements in both binary and multiclass skin lesion classification tasks. For binary classification, the F1-score reached 0.92, while the GMean achieved 0.93. In multiclass classification, the model’s \(F_1\) -score and Gmean were 0.89 and 0.90, respectively.