The open-pit mining process is prone to geological hazards that are detrimental to production. To ensure the normal and orderly progress of mining, it is necessary to conduct overall stability research on open-pit mines. Using Rhino software to establish a three-dimensional geological geometric model of Inner Mongolia Shengli No.1 open-pit mine, numerical calculations were carried out by importing it into FLAC 3D software to study the displacement field and stress field of the mining area. It shows that the displacement of various strata generally have a trend of rapid growth followed by gradual stabilization, and as the depth increases, the amplitude of displacement changes in each stratum decreases, and the rock and soil mass gradually tends to stabilize. The stress distribution is relatively uniform, and as the depth increases, the stress increases. The maximum and minimum principal stress concentration phenomena appear along the bottom of the soil dumping site. Establishing the concept of principal stress difference strength ratio to study the overall stability of Shengli No.1 open-pit mine. The ratio of principal stress difference intensity from small to large is as follows: around the mining site and slope, north and south soil dumping sites, along the slope and inner soil dumping sites, south side of F12 fault, and the north side of F12 is fault. The mining site and slope are the most stable, and most of the north side of F12 fault is around 0.9. Due to the distance between the north side of F12 fault and the mining site, it has little impact on the stability of the mining site and slope. Overall, the entire mining area is in a stable state. This article studies the overall stability of open-pit mines through numerical simulation, which can provide certain reference significance for open-pit mining.