<p>Large Language Models (LLMs) combined with Retrieval-Augmented Generation (RAG) frameworks greatly improve natural language processing performance, but they incur substantial computational overhead because many similar queries are processed repeatedly. To mitigate this, semantic caching has been introduced to store past responses and reuse them for semantically similar inputs, thereby reducing computation costs. Yet, semantic caching mechanisms that depend only on semantic similarity are vulnerable to adversarial exploitation: carefully engineered malicious queries with minor lexical variations can trigger incorrect cache hits, undermining both the reliability and the security of the system. This paper examines security vulnerabilities in semantic proximity caching systems such as GPTCache, a widely used open-source semantic cache that exemplifies these issues, and introduces a new approach called SAFE-CACHE, which is built to withstand adversarial attacks. SAFE-CACHE adopts a cluster-centroid-based caching strategy that is fundamentally distinct from GPTCache’s single-query embedding method. It uses unsupervised clustering of historical query–answer pairs, statistical detection of noisy clusters, bi-encoder–based refinement, and conditional cluster enrichment driven by a fine-tuned lightweight LLM to infer the underlying intent of cached queries. During runtime, incoming queries are compared to cluster centroids instead of individual cached entries, enabling stronger semantic validation and improved resilience against adversarial behavior. Our experimental evaluations demonstrate that SAFE-CACHE dramatically reduces adversarial attack success rates from 52.77% to 14.27% compared to GPTCache, representing up to 72% improvement in adversarial resistance.</p>