<p>High-utility itemsets mining aims to discover valuable itemsets from transactional data and is a key method in the field of data mining. Traditional exact algorithms face bottlenecks such as high computational costs and search space explosion when handling large-scale or complex data, while existing heuristic algorithms, although they have improved scalability to some extent, generally suffer from slow convergence, insufficient population diversity, and a tendency to get stuck in local optima. To address these issues, this study proposes a dung beetle optimization algorithm combined with off-policy reinforcement learning algorithm for the high-utility itemsets mining. To compress the search space and improve temporal and spatial efficiency, a reinforcement learning mechanism based on dynamic state clustering was designed. Breaking through the limitations of traditional methods that rely on single-state features, this study proposes a three-dimensional state fusion modeling and dynamic clustering strategy to achieve a refined characterization of search states. A multi-role collaborative evolution strategy based on beetle individuals was constructed, deeply coupling reinforcement learning with the evolutionary process to enhance global search capabilities and computational efficiency; simultaneously, a layered pruning and population repair strategy is proposed to enhance population diversity while suppressing premature convergence of the algorithm and reducing the loss of itemsets. Experimental results on multiple datasets demonstrate that, compared to current state-of-the-art algorithms, the proposed algorithm exhibits superior performance in terms of the completeness of high-utility itemsets, runtime, memory consumption, recall, and precision, while also demonstrating a faster convergence rate.</p>