Recent advancements in face expression recognition (FER) have introduced transformer-based algorithms to extract long-range semantic information. However, these approaches often lack interpretability and can be computationally intensive. To address the challenge of capturing long-range dependencies in images, we propose an innovative Adaptive Graph Attention Network (AGAT). This network applies graph attention mechanisms to nodes derived from feature maps, effectively modeling the relationships between different regions of an image. Specifically, we employ graph projection to construct graph structures in 2D feature maps, treating similar regions as nodes in the graph. Subsequently, a graph attention neural network is utilized to evaluate the significance of long-range semantic information. The enhanced features are then reprojected into a 2D format. Our proposed AGAT network has been tested on two well-known FER datasets, RAF-DB and FER-plus. Ablation studies confirm the efficacy of our method, demonstrating significant improvements over existing techniques. The source code for our AGAT network will be accessible at: https://github.com/jxLu2023/AGAT .