This work presents an innovative approach to non-rigid point set registration, addressing challenges commonly encountered in sparse and noisy point clouds, such as those encountered in medical image analysis and 3D object reconstruction. Traditional methods often struggle with outliers, inaccurate normal estimations, and the loss of detail during the deformation process. To overcome these issues, we introduce a method that incorporates a globally consistent normal estimation and a locally adaptive rigidity constraint. Our method uses the best parts of the Symmetric Point-to-Plane (SP2P) distance metric and improves them with an Adaptive Rigid Point Set (ARAP) regularization term that changes based on local geometric features and global normal consistency ensures accurate estimation of normal vectors at each iteration. This method ensures that the deformation process is both accurate and robust, preserving fine details while aligning with the target surface effectively. We test our approach across a range of datasets, encompassing human motion tracking and medical imaging, and benchmark it against cutting-edge methodologies. Our approach is more accurate and reliable than other methods, especially when there are complex geometric structures and a lot of noise, as shown by the results. Our method can handle non-isometric deformations and keep local details, which makes it a useful tool for tasks that need to be very precise in non-rigid registration.