The Kirchhoff index Kf(G) of G is defined as the sum of the resistance distances between all pairs of vertices in G. In 2024, Gottwald, Majstorović Ergotić and Došlić studied the problem of identifying graphs for which \(Kf(G)=Kf(G-v)\) for all \(v\in V(G)\) and found only one graph with this property: the cycle with 5 vertices. In this paper, we explore a relaxed version of this problem. Specifically, we aim to identify graphs that preserve the Kirchhoff index after the removal of a specific vertex, which we term a good vertex. Our investigation focuses on unicyclic graphs and cactus graphs, leading to several results. We construct infinitely many unicyclic graphs with girth 3 that contain exactly one good vertex. Moreover, for a given positive integer \(c\ge 4\) we show the existence of an infinite family of unicyclic graphs with girth c with at least two good vertices. Additionally, we prove that the smallest number of vertices in a unicyclic graph, which is not a cycle and contains at least two good vertices, is 8. We expand our research to cactus graphs to show that for a fixed positive integer \(k\ge 2\) there exists an infinite family of cactus graphs with at least k cycles, where each graph in the family has at least two good vertices.