The state of the art in hand pose estimation can benefit from larger, annotated datasets, covering ideally the full spectrum of hand poses. However, annotating keypoint locations in large numbers of hand images can be prohibitively time-consuming and expensive. We propose a new type of model for hand pose estimation that can be used to speed up manual annotations of hand images. The key idea is that, when the human annotator provides correct locations for some keypoints, that provides additional information (for which we use the term “partial annotation”) that can be exploited. An appropriately designed model can benefit from this additional information to produce more accurate estimates compared to the estimate that it produces just based on the image. In this paper, we propose such a model and demonstrate that using partial annotations indeed improves accuracy compared to estimates that are based solely on the hand image itself.