Scholarlink

In this study, we focused on changes in feature representations across layers in convolutional neural networks (CNNs) and verified whether shallow layers extract generalizable features that are independent of the training dataset. To quantitatively evaluate the similarity of internal representations between models trained on different datasets, we adopted Centered Kernel Alignment (CKA). The results suggest that features common across different datasets are preserved in shallow layers. Additionally, we explored whether fine-tuning layers with low CKA similarity could improve classification performance in transfer learning. CKA-based layer selection showed some effectiveness, but we were unable to identify a clear threshold. One reason for the inability to identify a clear threshold was the use of CIFAR-10 as the transfer source dataset. In particular, when the weights were fixed, it may have been difficult to obtain sufficient features for effective transfer learning. This study deepens the understanding of CNN internal representations and provides concrete insights for optimizing transfer learning strategies.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Feature Generalizability in Shallow Layers of CNN and its Applicability to Transfer Learning

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Mizuki Dai,
  • Kenya Jin’no

摘要

In this study, we focused on changes in feature representations across layers in convolutional neural networks (CNNs) and verified whether shallow layers extract generalizable features that are independent of the training dataset. To quantitatively evaluate the similarity of internal representations between models trained on different datasets, we adopted Centered Kernel Alignment (CKA). The results suggest that features common across different datasets are preserved in shallow layers. Additionally, we explored whether fine-tuning layers with low CKA similarity could improve classification performance in transfer learning. CKA-based layer selection showed some effectiveness, but we were unable to identify a clear threshold. One reason for the inability to identify a clear threshold was the use of CIFAR-10 as the transfer source dataset. In particular, when the weights were fixed, it may have been difficult to obtain sufficient features for effective transfer learning. This study deepens the understanding of CNN internal representations and provides concrete insights for optimizing transfer learning strategies.