Accurate segmentation of breast ultrasound images is critical for early cancer detection yet poses significant challenges due to severe noise and blurred boundaries of weak echo signals. Traditional convolution-based segmentation networks struggle to effectively balance noise suppression and detail description constrained by fixed receptive fields, while Transformer-based segmentation networks have greatly enhanced long-range dependency modeling; however, hierarchical downsampling often results in the loss of high-frequency details and is deficient in dynamically adapting to ultrasound noise. To address these limitations, we propose the Echo-Enhanced Dual-Dilated Residual Attention Network ( \(\text {E}^{2}\text {D}^{2}\) R-Net). First, EchoFocus Window Multi-head Self-Attention (EFW-MSA) integrates global channel recalibration with parallel spatial-channel dual attention, enabling noise-adaptive suppression and enhanced delineation of fine boundaries. Second, Dual-Dilated Merging (DDM) substitutes linear downsampling, capturing micro-texture and macro-context via lightweight double-dilated convolution branches. Finally, Lightweight Residual Separable Attention Gate (LRSAG) is incorporated into the jump connection, incorporating a separable attention gate and residual edge preservation branch, thereby effectively filtering encoder noise, preserving weak echo features, and significantly reducing false positives in small lesion segmentation. \(\text {E}^{2}\text {D}^{2}\) R-Net enhances the integration of local and global context, minimizes computational overhead, and improves clinical applicability. Experiments on the BUSI and UDIAT datasets confirm the feasibility and efficacy of the proposed method.