Visual location recognition technology involves the fields of computer vision, pattern recognition and robotics. This is the key to achieving precise positioning from autonomous vehicle technology, and it is also one of the research focuses of Simultaneous Localization and Mapping (SLAM). Although most current VPR methods work well under ideal conditions, their performance is generally unsatisfactory in complex environments characterized by illumination changes, seasonal changes, and occlusions caused by moving objects. In view of the robustness of the semantic segmentation network to the above environment, we hope to express the deep spatial geometric relationship of the image through the rich high-level semantic features in the high-precision semantic segmentation network to solve the above problems. In this study, we propose a novel VPR architecture named convG-sem, which not only proposes a relatively efficient feature aggregation method, but also makes full use of semantic features to express the relative geometric relationship between semantic regions. Finally, we experimentally demonstrate that the proposed convG-sem architecture significantly outperforms the current state-of-the-art methods.