Text classification is a fundamental task in natural language processing, essential for organizing and retrieving information across various domains. This study compares the effectiveness of two document representation schemes for text classification, namely, Latent Semantic Indexing (LSI) leveraging Singular Value Decomposition (SVD) and trainable word embeddings using two different datasets, namely, the Reuters and BBC News. We used the Reuters dataset with more categories that are somewhat hard to distinguish from each other, while the BBC News dataset had only five semantically distinct categories. The LSI-based approach focuses on dimensionality reduction while tracking term co-occurrences through SVD to capture latent semantic structures, while the word embedding-based approach creates trainable embeddings that track word co-occurrences while taking their sequencing into account in their rich semantic representations. Both methodologies were evaluated using multi-layer perceptrons (MLPs). Performance was assessed based on precision, recall, F1-score, and accuracy. The results indicate that LSI offers a balance between computational efficiency and classification accuracy, making it suitable for large datasets with limited resources. Word embeddings, while requiring more computational power, provide higher accuracy due to their ability to capture nuanced semantic relationships. The distinctiveness and balance of the categories in the datasets significantly influenced performance, with the Reuters dataset showing slightly better results with LSI and the BBC News dataset exhibiting near-perfect classification results with higher-dimensional trainable embeddings. Additionally, a novel combined representation approach, integrating whole document and average document embeddings, demonstrated improved performance, particularly for the BBC dataset. This comparative analysis highlights the strengths and weaknesses of each approach, providing valuable insights into selecting the optimal strategy for text classification tasks.