Numerous researchers have adopted deep learning algorithms to supplement traditional investors in making trading decisions and have achieved notable success. Deep reinforcement learning (DRL) is a methodology used to train agents to navigate their environment. Recently, several techniques have been developed that effectively leverage DRL as an intelligent system for investment decision-making in the stock market. However, the existing research on stock decision support systems neglects the critical role of trading strategies in the decision-making process. Furthermore, they do not adequately address the significant variations present in stock time-series data. Our study addresses these limitations by introducing a stock trading decision-making system based on reinforcement learning and the box theory. Drawing on the traditional box theory, we developed a novel stock time-series segmentation method called StockTrendr. This algorithm partitions the oscillatory box of price fluctuations to identify suitable trading opportunities. These boxes are used to create experience replay weights in the Deep Q-Network (DQN), thereby augmenting the decision-making capacity of the model when there are significant shifts in stock trends. Furthermore, for DQN training, we developed an innovative stock-trading strategy that generates more precise rewards for system training based on the actions taken by the agent. Experimental results demonstrate that, compared with other state-of-the-art approaches, our proposed method achieves higher returns in both the Chinese and US stock markets.