Lower limb activity analysis using wearable sensors holds significant potential for healthcare and human-computer interaction, yet accurate recognition remains challenging due to complex kinematic patterns and multi-sensor data heterogeneity. In this paper, we propose a Multi-scale Attention Fusion Module (MAFM), a lower limb motion pattern recognition model that integrates multi-scale convolution and dual attention mechanisms. The architecture employs parallel 3 × 3, 5 × 5, and 7 × 7 convolutional branches to extract multi-scale spatial features, followed by additive fusion of the tri-branch outputs. A channel attention module dynamically recalibrates channel-wise importance through global average pooling and adaptive weighting via a two-layer fully connected network. Building on these channel-refined features, a spatial attention module generates spatial weight maps by concatenating max-pooled and average-pooled representations, processing them with 1 × 1 convolution and Sigmoid activation to emphasize critical regions. The refined channel and spatial attention features are fused, compressed through 1 × 1 convolution, and fed into fully connected layers with Softmax for classification. Evaluated on the self-collected Self-Leg dataset (6 lower limb activities with 4 IMU sensors), the model achieves 96.36% overall accuracy and 96.27% average F1-score, surpassing baseline methods including CNN, LSTM, and COND-CNN. Experiments confirm its robustness and generalization capability with multi-sensor inputs. The architecture adaptively optimizes both feature significance (“what to focus”) and spatial localization (“where to emphasize”), demonstrating enhanced recognition accuracy for real-time wearable applications in practical scenarios.