<p>Most particle swarm optimization (PSO) variants do not make full use of the gradient information of optimized problems and ignore the velocity information in the previous generations except for that in the last generation when updating the velocities, resulting in lower efficiency and effectiveness. To cope with these issues, this paper proposes a Gradient Fractional-order PSO with Social learning and terminal Replacement mechanisms (GFPSOSR) for numerical optimization. Specifically, GFPSOSR first uses fractional calculus to capture the velocity information in the previous generations to update the current velocity. Then, GFPSOSR takes gradient information to update the velocity towards better solutions. In addition, a novel social learning mechanism and a terminal replacement mechanism are designed to enhance the searching ability. We conduct extensive experiments on 26 popular test functions (including 6 unimodal functions, 15 multimodal functions, and 5 hybrid functions) with different dimensions by comparing the proposed GFPSOSR with 14 algorithms (including 7 PSO advanced variants and 7 other types of numerical optimization algorithms). The experimental results indicate that the GFPSOSR outperforms the compared algorithms in terms of the evaluation metrics. Particularly, the overall accuracy of GFPSOSR ranks first with 3 out of 4 types of dimensions. The average running time taken by GFPSOSR to solve a 100-dimensional problem is about 2&#xa0;s, significantly lower than that of the comparative methods. Furthermore, we discuss the impacts of the GFPSOSR’s parameters and analyze each mechanism’s contribution. All these confirm that the proposed GFPSOSR is promising for numerical optimization. </p>