Skeleton-based action recognition is essential in video analysis. While Graph Convolutional Networks (GCN)-based methods effectively represent spatio-temporal features, they often struggle with fine-grained actions, leading to a decline in classification performance. To address this issue, we propose a novel method inspired by spatio-temporal semantic learning, which explicitly enhances feature discrimination by emphasizing differences between actions through spatio-temporal semantics. Our method introduces a plug-and-play spatio-temporal semantic extraction module (STSM), which encompasses the separate extraction of spatio-temporal features and the clustering of similar semantics for different actions. By adjusting the distance between semantic categories, the feature discrimination of the model is enhanced. Furthermore, the STSM is integrated into various stages of GCNs to extract multi-level spatio-temporal semantic features, and a spatio-temporal semantic loss is constructed for more effective supervision. Extensive experiments on three datasets demonstrate that our model leads to superior classification accuracy and outperforms advanced methods.