The on-ramp merging area plays an important role in improving the overall safety and efficiency of the traffic. At present, the widely used ramp merging control methods are mostly based on reinforcement learning (RL), but these methods may make erroneous decisions due to bad generalization ability, potentially jeopardizing traffic efficiency and safety under unseen scenarios. Noticing the great success of large language models (LLMs) on other domain, this paper proposes a novel method based on LLM-enhanced RL agent for traffic control at on-ramp merging areas. Specifically, the proposed method utilizes RL agents to make initial actions based on surrounding environment, after that, an LLM model refines these decisions by the designed prompt and chain of thoughts. Experiment results show that the proposed method outperforms the other methods for on-ramp merging, with a promotion of about 14% on average speed of involved vehicles, and the promotion becomes 22.1% under severe dense traffic conditions.