EEG-based emotion recognition can effectively monitor the users’ real-time emotional states, provide more objective physiological data support for mental health assessment, and thus detect the users’ potential psychological problems in a timely manner. Although existing research has made notable advancements, The biological and topological information among brain areas has not been sufficiently utilized. To the end, the paper proposes an emotion recognition network, dubbed TS-Net, based on the temporal-spatial features of EEG signals. TS-Net contains a special-purpose temporal feature extraction component (1DCNN) and a special-purpose spatial feature extraction component (gMLP), which enable it to fully analyze users’ emotional states based on the neural activity intensity in different brain functional areas. Experiment results show that TS-Net reached an overall accuracy of 97.87%, 96.79%, and 97.99% for arousal, valence, and dominance evaluated with the dataset DREAMER, respectively, which demonstrates that TS-Net has outperformed the existing advanced methods for emotion recognition. Finally, we conducted tests on two self-collected emotion classification datasets, and our model also achieved satisfactory results.