Robust handwritten text recognition (HTR) systems require diverse and abundant annotated data, which can be expensive to acquire. Unlike commonly used methods, such as transfer learning and data augmentation, that reduce the need for labeled data, this article proposes a novel self-supervised handwritten text recognition (HTR) framework to eliminate that need: self-HTR. Self-HTR leverages a high-level loss that informs about the textual dissimilarity between an input image and an image generated with a pretrained generative adversarial network containing the network’s predicted text. Based on preliminary experiments with handwritten character recognition, experiments were conducted on the IAM dataset for word recognition. Self-HTR was implemented by integrating the generative architecture GANwriting with a CNN-BLSTM architecture. Experiments were conducted with two image-based and two style-invariant losses. Trained on IAM images recreated with GANwriting and using transfer learning, self-HTR achieved a character- and word error rate of 0.44% and 1.56% on recreated test data. While on simulated data, these results indicate the potential of self-HTR on real data. Future research is necessary to validate the framework further on real data and use high-quality handwritten text generation models.