Modern database management systems face increasingly complex and dynamic application scenarios, which place higher demands on buffer management. Simple, single-purpose strategies often fail to adapt. This limitation is especially evident in highly variable workloads. On the other hand, more sophisticated strategies, while capable of improving hit rates, typically increase system complexity, significantly reducing throughput and ultimately degrading the overall performance of the database system. In this paper, we propose a Reinforcement Learning-Enhanced Adaptive Buffer Replacement (REABR) strategy which integrate two simple but effective strategies tailored for real-world systems. REABR leverages online learning techniques to provide an efficient page replacement strategy without introducing additional runtime overhead, thereby enhancing the overall system performance. Furthermore, we deployed REABR in the open source openGauss database system and conducted extensive experiments to validate its effectiveness. The results on TPC-C and synthesized datasets have shown that REABR achieves a favorable balance between simplicity, adaptability, and performance, proving its practical value in real-world scenarios.