To address the problems of insufficient feature representation and large changes in target scale in medical image segmentation, this paper proposes two modules, namely, Adaptive Feature Attention Module (AFAM) and Multi-Scale Dilated Convolution (MDC), both of which are plug-and-play modules. The AFAM module consists of three parts, namely, weight generation, feature generation, and weighted aggregation, which can dynamically capture the complex associations between multi-scale features, effectively suppress redundant information and enhance the key features. The MDC module is based on the multi-branch dilated convolution design, combined with the residual connection mechanism, which expands the sensory field without reducing the resolution and realizes the fusion of local details and global context. To verify the effectiveness of these two modules, this paper conducts ablation experiments on the ISIC 2017 medical image segmentation dataset as well as comparison experiments with other models, and the experimental results show that the combination of the two can effectively improve the image segmentation performance. In addition, this paper conducts experiments on Pascal VOC 2007 and PCB-NET target detection datasets, and the results show that AFAM enables the model to achieve faster inference efficiency while ensuring detection accuracy.