As the transformer family of models for deep learning became the leading type of neural architecture, researchers have noted the significant advantage that pretraining provides. Learning (pretraining) performance of transformer models scales as a power law with the amount of training data and model parameters (Kaplan et al. 2020). Hence, there began a race for training larger and larger language models. GPT-2 (Radford et al. 2019) was trained on about 8 million text documents and the model variants had between 117 million and 1.5 billion parameters. Google’s T5 model (Raggel et al. 2019) scaled up to 11 billion parameters. The GPT-3 family of models (Brown et al. 2020) has up to 175 billion parameters. GPT-4 is estimated to have up to about 1.8 trillion parameters (Schreiner 2023). Other contemporary large language model families include LLaMA (Touvron et al. 2023), Gemma/Gemini (Gemma Team 2024), Mistral (Jiang et al. 2023) and many others. Zhao et al. (2023) present a comprehensive multi-faceted review of large language models. Note a terminological distinction between LLM—large language models, and PLM—pretrained language models. The former emphasizes the model size, while the latter emphasizes the pretrained aspect.