Neural Machine Translation (NMT) has been a prevailing paradigm for multilingual translation, particularly with advancements in Sequence-to-Sequence (Seq2Seq) models and Transformer models. Bengali and Hindi are low-resource languages that are difficult to translate into English due to limited parallel corpora, dense morphology, and diverse syntactic patterns. This paper proposes a Transformer-based Seq2Seq model with language-specific Bidirectional Encoder Representations from Transformers(BERT) tokenizers Bangla BERT and Hindi BERT for context aware and efficient translation. The system is developed and deployed with PyTorch on the Samanantar corpus, using attention mechanisms, position encoding, and greedy decoding for real-time inference. Experimental results demonstrate that the Bengali to English model outperforms the Hindi to English alternative on the most critical metrics such as Bilingual Evaluation Understudy (BLEU) score, precision, recall, F1-score, and perplexity. The results confirm the importance of linguistically tuned tokenization and memory augmentation in enhancing translation quality for low-resource languages.