Online Continual Learning (CL) is a challenging scenario that focuses on enabling models to incrementally learn new knowledge from sequential, non-i.i.d. data with only a single pass through the data. Replay-based methods have shown great potential in this scenario, but they still suffer from catastrophic forgetting. In contrast, the human brain exhibits great advantages in online CL scenario, with memories swiftly encoded as short-term memories in the hippocampus and subsequently consolidated into distributed long-term memories within the neocortex. Inspired by this neural mechanism, we introduce the Distributed Long-Term Memory Guided CL(DLTMG-CL) method, which guides the network to retain old knowledge while learning new information by consolidating and preserving distributed long-term memories. This approach facilitates efficient learning in online CL scenario. Our algorithm has achieved state-of-the-art (SOTA) performance among various CL methods.