An Efficient Metadata Processing Method Based on Attention Mechanism
摘要
Clinical information is crucial in the diagnosis of skin cancer, which improves diagnostic accuracy by providing key backgrounds such as medical history, symptoms and signs. However, the multimodal cross-fusion analysis using image data and patient clinical data is limited, and there is no efficient fusion method. In order to make up for this deficiency, a metadata processing module AtnMetaBlock based on attention mechanism is proposed. By combining clinical information with skin lesion images, the module uses the attention mechanism to dynamically adjust the weights of different data types, thereby enhancing the most relevant features and improving the classification effect. AtnMetaBlock is applied to seven convolutional neural networks (CNN), and trained and tested on PAD-UFES-20 skin lesion dataset. The experimental results show that compared with the model without metadata, concatenation method, MetaNet and MetaBlock, AtnMetaBlock has a maximum increase of about 10.4% in the balance accuracy (BACC) index. Five of the seven CNN models achieve the best performance. In addition, AtnMetaBlock shows higher stability than the other two methods, which further verifies its potential as an efficient metadata processing method.