With the enhancement of computing power in edge computing hardware and the improvement in sensor accuracy and portability, a solid foundation has been laid for the real-time collection and processing of human activities. In recent years, an increasing number of scholars have begun to focus on the recognition of human activities. Traditional convolutional neural networks often face limitations when extracting features, particularly in terms of global information and multi-scale features. To address these issues, we propose a network structure called ME-Transformer, which integrates median-enhanced channel attention modules with Transformer layers. This structure effectively combines spatial and temporal information, making it novel and capable of capturing and integrating features at various scales. The effectiveness and superiority of the ME-Transformer model have been demonstrated through datasets, achieving an accuracy of 97.62%.