Making rapid and accurate decisions under uncertain market conditions is challenging in high-frequency trading environments. This study focuses on enhancing trading performance in low timeframes, where increased noise and volatility pose significant challenges. We propose a reinforcement learning framework that addresses this issue using a hierarchical multi-agent structure. The core of our system is a 5-minute trading agent supported by expert agents operating on 1-hour and 1-day timeframes. These expert agents are trained independently and provide high-level strategic signals to guide the low-timeframe agent’s decisions. The financial market is modeled as a Markov Decision Process (MDP), and we utilize the Proximal Policy Optimization (PPO) algorithm with an LSTM-based feature extractor to effectively capture temporal patterns. Empirical results on Ethereum price data (June 2023–January 2024) show that our 1-day expert agent outperforms traditional technical indicators (e.g., MACD, RSI) and achieves better results than the representative model from the existing literature [22], particularly on the ETH dataset. Furthermore, the 5-minute agent significantly improves its performance when guided by the expert agents, highlighting the potential of multi-agent cooperation for robust high-frequency trading.