Text embedding is a crucial element in computational linguistics, allowing computers to represent human language semantics numerically. A key approach is the Vector Space Model (VSM), which maps text into high-dimensional vectors and supports tasks like classification, clustering, and semantic search. Building on this, techniques like word embeddings (e.g., word2vec) and document embeddings have advanced Natural Language Processing (NLP), enabling sentiment analysis, machine translation, and summarization. However, challenges persist in morphologically rich languages (MRLs) such as Arabic, Hebrew, Russian, and Turkish. These languages feature complex structures with prefixes, suffixes, and roots, complicating text representation and increasing out-of-vocabulary (OOV) words. Addressing these challenges is essential for improving NLP’s applicability across diverse linguistic contexts. This study proposes a novel methodology to overcome existing limitations by utilizing probabilistic corpus decomposition. The corpus is processed in three forms: words are represented as wholes, decomposed into grammatical components, or broken down into letters. The Word2Vec algorithm is applied to this “triple corpus”, generating vectors for words, their components, and letters, ensuring consistent semantic alignment across all forms. Experiments with English and Hebrew corpora demonstrate the effectiveness of this approach. The fused vectors of grammatical components achieve high-quality representations, often ranking first in cosine similarity, thereby reducing out-of-vocabulary (OOV) issues and expanding the lexicon. By improving the representation of morphological structures, this methodology enhances embeddings for both English and Semitic languages, advancing NLP capabilities in a variety of linguistic contexts.