The interpretability of deep learning is crucial for evaluating the reliability of medical imaging models and reducing the risks of inaccurate patient diagnoses. This study addresses the “human-out-of-the-loop” and “trustworthiness” issues in medical image analysis by integrating medical professionals into the interpretability process and model guidance. We propose a disease-weighted attention map refinement network (DWARF) that leverages expert feedback to enhance model relevance and accuracy. Our method employs cyclic training [13] to iteratively improve diagnostic performance, generating precise and interpretable feature maps. Experimental results demonstrate significant improvements in interpretability and diagnostic accuracy on three publically-available multi-label chest X-ray datasets. The proposed DWARF approach fosters effective collaboration between AI systems and healthcare professionals, ultimately aiming to improve patient outcomes. The code is available on https://github.com/Roypic/DWARF .