Fully Transformer-based architectures have gained significant attention in Handwritten Text Recognition (HTR) due to their ability to model long-range dependencies. However, these models are highly susceptible to overfitting, limiting their generalization capabilities, particularly in data-scarce scenarios. In this work, we systematically analyze overfitting in a fully Transformer-based HTR model and explore various mitigation techniques. By evaluating different strategies, we gain insights into how architectural components contribute to overfitting and how specific techniques enhance model robustness. Our findings align with recent trends in HTR and provide a deeper understanding of overfitting in fully Transformer-based models, thereby offering valuable insights for future advancements in the field.