The pedestrian trajectory prediction has been extensively studied in some fields such as autonomous driving and intelligent surveillance, wherein the existing methods can not effectively capture the high-order dependencies and the diverse interaction patterns among pedestrians. To this end, this paper proposes a lightweight pedestrian trajectory prediction model called Spatio-Temporal Graph Trajectory (STGTraj), which not only enhances the ability on capturing both the global and local spatiotemporal dynamic features of trajectory, but also achieves the uniform coverage and stability of the trajectory sample distribution. Based on the proposed STGTraj model, this paper futher presents a pedestrian trajectory prediction model called the Spatio-Temporal Multi-scale Graph Trajectory (STMGTraj). Through the multi-scale interaction model, STMGTraj can captures the spatiotemporal features more comprehensively compared with STGTraj. In addition, this paper adopts the Box-Muller transformation to enhance the stability of normally distributed random numbers with the Quasi-Monte Carlo (QMC) methods. The experiments and analyses are conducted on the ETH and UCY datasets, and the results confirm that the proposed methods outperform the existing state-of-the-art models across multiple metrics.