Self-supervised contrastive learning aims to extract discriminative information from data by conducting instance-level contrastive analysis. However, this approach is limited by stochastic data augmentation techniques, which may result in either trivial augmentations that fail to enhance the contrastive model or excessive augmentations that impede model convergence. To address these, recent studies have explored the use of Reinforcement Learning (RL) to develop dataset-specific data augmentation strategies by leveraging pre-trained models in a supervised manner. Nevertheless, these methods are heavily dependent on the quality of the pre-trained models and are not well-suited for unsupervised learning scenarios. Data augmentations generated by these methods are dataset-specific and remain static during training, which leads the model to converge to a local optimum. Thus, we propose a novel approach that frames the task of identifying effective data augmentations for contrastive learning as a game, thereby transforming the parameter tuning problem into a policy search task that aligns with RL. Building on this, we introduce an online self-supervised framework designed to generate challenging yet convergent self-supervised contrastive tasks. We evaluate the proposed method on multiple datasets for visual self-supervised representation learning and empirically demonstrate that our approach achieves significant performance improvements at most 1.2%.