Vision Mamba typically uses multidirectional scanning to weaken the unidirectional causal properties of Mamba while retaining its long sequence modeling capability. Although these methods are effective, they do not adequately conduct local feature extraction, and the architecture consisting entirely of Mamba-based models are singularly focused on the global perspective, which limits Mamba’s performance in visual tasks and is detrimental to the generalization performance of the models. In this paper, we propose a hierarchical weakly causal Mamba (MFMamba) with multi-scale feature fusion. High-resolution shallow layers use deep convolution to capture local spatial features of an image in spatial dimensions in parallel, and low-resolution deep layers use bidirectional Mamba for long-sequence modeling to improve the model’s upper bound and capacity of the model. This hierarchical structure enhances the model’s ability to extract local features, and the bidirectional scanning ensures weak causality. Meanwhile, a multi-scale feature fusion method based on learnable scale factors is proposed to enhance the generalization ability of the model. Experiments show that MFMamba achieves competitive results on vision tasks such as image classification, semantic segmentation and object detection, and achieves advantages in terms of accuracy and computational efficiency over existing models in the Transformer series and the Mamba series.