In recent years, human bone tracking technology has attracted much attention in the field of virtual reality. The importance of human skeleton tracking technology in the field of virtual reality cannot be underestimated, and human skeleton tracking technology is the basis of character models in virtual reality systems. However, most of the current research on skeletal point tracking is limited to body tracking and does not combine the body skeleton with the finer hand skeletal points. Aiming at the problem that the skeletal points of the human body and the skeletal points of the hand have been studied separately in previous studies, this paper proposes a novel depth camera-based skeletal point tracking method. The method uses a lower-cost monocular depth camera as the basic equipment, and by processing the 3D coordinates of the skeletal points and reconstructing the coordinate system, it successfully achieves the effect of tracking the torso skeletal points and hand skeletal points at the same time with only one depth camera. The method can be applied to virtual reality systems to realise the mapping from real users to virtual characters by building a character model in the system and corresponding the skeletal points of the model to those of the real users one by one.