Breast MRI provides superior soft-tissue contrast and lesion conspicuity compared to mammography but its large-scale deployment is hampered by the need for fine-grained annotations. We propose BE-WISE, a transformer-based framework for interpretable breast MRI classification that jointly learns breast-level diagnosis and slice-level lesion localization from minimal radiologist input. The approach integrates a Swin transformer backbone into an attention-based multiple-instance learning scheme and optimizes a unified Gaussian-based objective that couples global and local supervision. On the multicenter ODELIA Breast MRI dataset, BE-WISE with focal loss attains a test AUC of 0.8683 and an Odelia score of 0.7098, improving over the medical slice transformer baseline by more than 7% in AUC and 14% in odelia score. Slice-wise prediction profiles align with expert-indicated lesion slices, supporting the interpretability of the model. These findings indicate that weak, slice-level expert guidance can substantially enhance diagnostic performance and enable human-in-the-loop AI for breast MRI.