Token modalities underpin the model’s ability to effectively fuse and process diverse data types, essential for comprehensive understanding and robust performance over various multi-modal learning tasks. Batch size holds paramount importance in deep learning research, as it profoundly affects GPU resource utilization efficiency and training stability, significantly shaping model generalization and convergence dynamics. In this paper, we investigate the role of batch size and token modalities on document-understanding by fine-tuning LayoutLMv3 on a small subset of the DocVQA. To highlight the importance of spatial layout on an extremely challenging task like Document-based VQA we conducted an ablation study where we only fed the text tokens and compared the model’s performance with its multi-modal baseline. Additionally, we evaluated the significance of batch size on learning and generalizing in computationally constrained environments by training our model with varying sample size and batch size, and the results are presented. Our experimental results demonstrate an average improvement of approximately 38% when utilizing multi-modal learning approaches over relying solely on text-based approaches, especially in scenarios where training data is limited. When training the model with different sample sizes and batch sizes, we observed that employing a larger batch size effectively alleviates over-fitting, resulting in improvements of up to 16%. As a recommendation, we advise selecting an optimal batch size that maximizes GPU memory utilization while preserving training stability.