Large Language Models (LLMs) face significant challenges in maintaining robust contextual understanding across extended sequences, primarily due to bidirectional interference phenomena in their finite working memory. This includes proactive interference (PI), where earlier information impedes retrieval of more recent content, and retroactive interference (RI), where newly arriving information corrupts previously stored context. While existing research has primarily focused on proactive interference, this paper provides the first systematic investigation of bidirectional interference as a fundamental constraint in LLMs’ long-context processing capabilities. We propose a novel, training-free framework that operates across three synergistic dimensions: (1) temporal-semantic context management that segments inputs into semantically coherent chunks augmented with timestamps and structural cues; (2) query-aware temporal-semantic weighting that dynamically scores contextual relevance based on temporal proximity and semantic similarity; and (3) meta-semantic labels that protect critical information from unintended overwriting. Integrated together, these mechanisms effectively mitigate both PI and RI. To evaluate our approach, we introduce IRBench, a comprehensive benchmark covering proactive and retroactive interference tasks across multiple LLMs. Experiments show significant improvements over existing methods, including recent techniques like active context management and prompt compression, confirming our framework’s efficacy and generalizability. This work advances the understanding of LLM memory limits and offers a principled solution for enhancing long-context robustness without retraining.