We propose a Rough set-theoretic approach for solving the stochastic Multi-Armed Bandit (MAB) problems. The proposed approach is a modification to the Epsilon-greedy ( \(\epsilon \) -greedy) algorithm used to solve the stochastic multi-armed bandit problems. In our proposed approach, initially, we randomly explore all the arms for some time steps to gather basic reward data for each arm. Using this collected basic reward data, rough estimates of the expected rewards of the arms are calculated. Based on the rough estimates of the expected rewards of all the arms, we partition the arms into three parts following the principles of rough set theory. In the subsequent time steps, different exploration rates are used for different partitions to guide arm selection, to balance between exploring new options and exploiting known performers. We periodically update each arm’s estimated mean reward and re-partition them into three parts following a defined process. We continuously monitor for stability in the reward structure of the problem and adaptively adjust the exploration-exploitation balance in response. As the algorithm progresses, the arms with the potential to become the best arm are identified and the exploration is narrowed, which leads to a concentration of effort on arms that consistently yield higher rewards, leaving out the other arms. This strategic selection of arms directs the exploration toward the most promising arms, enhancing the efficiency of the learning process, which is proved with the support of the experimental results.