In this paper, a novel particle tracking method is proposed to investigate the kinematics of Leighton Buzzard sand (LBS) particles exhibiting slight- and medium-level fragmentation by combining PointConv and PointNetLK networks. Firstly, a series of image processing algorithms were employed on the raw CT slices to facilitate reproduction of particle morphology. Subsequently, all particles were represented as point sets, down sampled and grouped, yielding substantial datasets for neural network training and testing. Additionally, Gaussian noise was generated and introduced into the particle point sets to enhance the network robustness. Then, the PointConv network was implemented to efficiently match the particles under different strains. This success was attributed to the good preservation of morphological features of these particles and the excellent capacity of PointConv to capture morphological features. Next, the PointNetLK network was trained and incorporated with the particle correspondence obtained by PointConv, aiming to determine the optimal transformation matrix for the corresponding particles. Finally, the predictive results are evaluated based on the visualization of the transformed point cloud, and the particle kinematics is analyzed.