Enhanced radiology report generation via comprehensive sequence rearrangement and multi-scale cross-region attention
摘要
In the medical domain, accurate and detailed radiology reports are pivotal for disease diagnosis and treatment. Despite existing methods showing promise, challenges persist in extracting effective features and focusing on critical regions. To address these issues, we introduce a radiology report generation model, CSR-LMCA, which integrates comprehensive sequence rearrangement with multi-scale cross-region attention. Our model enhances focus on disease-related areas through Saliency-guided Discriminative Attention Mapping (SDAM), significantly improving lesion region identification and background noise suppression. Additionally, the Sequence Rearrangement Mamba (SR-Mamba) module efficiently extracts discriminative features from rearranged long sequences. The Local Multi-scale Cross-region Attention (LMCA) mechanism models local attention relationships and performs cross-region information fusion, strengthening the model’s ability to capture global features and focus on key areas. Experiments on the IU X-ray and MIMIC-CXR datasets demonstrate that CSR-LMCA outperforms state-of-the-art methods, achieving BLEU-4 scores of 0.175 and 0.118, respectively, on these datasets. Here we show that our model not only generates informative and coherent radiology reports but also offers significant improvements in text completeness, coherence, and readability. The code and datasets are available at: https://github.com/CQNU-ZhangLab/CSR-LMCA.