Tokenization inherently affects the way information is processed within Large Language Models (LLMs), with dramatic implications for performance. Common strategies such as Byte Pair Encoding (BPE) used in the likes of GPT-2 tend to output linguistically inconsequential segments simply because they operate statistically, posing a risk of impeding predictability of models (quantified by perplexity). This work explores tokenization techniques to get lower perplexity. We present AMEST, a new hybrid tokenization that leverages both linguistic insight (data-driven morphological segmentation) and statistical soundness (probabilistic sampling of Unigram), with primary emphasis on favouring morphemes for familiar terms and sampling to handle unknown ones. We measured by training models of GPT-2 Small from scratch across eight varied sets of data to demonstrate that AMEST reliably registers significant perplexity decreases, improvements in convergence and competitive throughput vs. GPT-2 BPE and Baseline Unigram tokenizer. Our analysis, including ablation studies and cross-domain evaluations, confirms that the synergy between morphological awareness and probabilistic fallback is key to these improvements.