<p>A topological map with the spatial relationship is an inescapable object in the research of map fusion, as it is a priori knowledge for planning path. However, there are some difficulties in topological map fusion in a dynamic environment. Therefore, this paper proposes a fusion method for the hybrid topological map based on the memory sphere. A hybrid topological map is composed of occupancy grid maps and the topological structure. The hybrid map fusion can rely on rich features in occupancy grid maps. By analyzing the process of recalling scene, a memory sphere is designed to store the features and the semantic label extracted from occupancy grid maps. Then the core is the matching of the memory sphere, which is divided into two parts, fast retrieval and fine matching. We verify the effectiveness of our method in simulation and real environments, demonstrating that our method has a great performance in the dynamic environment.</p>