CARL: Efficient Deep Reinforcement Learning via Contrastive Replay
摘要
Deep reinforcement learning (DRL) methods have proven to be very effective in solving complex decision-making issues; however, their reliance on high memory and computational resources has a tendency to limit their application in restricted environments. This paper presents a memory-saving DRL model specifically developed for the Snake game, not just with competitive performance but with significant memory reduction. Our new approach employs a hybrid Convolutional Neural Network–Multi-Layer Perceptron (CNN-MLP) structure to perform efficient feature extraction and decision-making, along with a compact First-In-First-Out (FIFO) replay buffer. The method also offers an original contrastive experience replay mechanism that selectively enhances useful transitions, thus avoiding the problem of uniform sampling as well as the computation cost involved with methods like Prioritized Experience Replay. Experimental results explicitly show that our approach has good learning performance comparable to state-of-the-art DRL algorithms but significantly less memory usage. This paper is an important milestone towards building scalable DRL solutions for use cases where memory and computational efficiency are important.