This study focuses on improving the MT by enhancing the attention-driven Seq2Seq neural networks, which is aimed at solving some of the significant hurdles in the French-to-English MT, including via contextual ambiguity, idiomatic expression, syntax complexity, gendered nouns, complex verb conjugation as well as syntax. Here, we present a new model, the sequence-aligned Transformer with CNNs, or Transformer-SA-RC, that integrates contextual features from cross-modal data streams while facilitating text-stream translation with visual, semantic, and textual data flows. Recent experiments with a high volume of phrases have been conducted based on the WMT 2014 English–French dataset that suggests significant improvements in translation quality with improvements in BLEU, ROUGE-L, and METEOR scores. The model proposed in this study was approximately 38.2% and higher than the baseline models, such as the Seq2Seq and traditional Transformer model, considering linguistic aspects, context-related uncertainty, and idiomatic expressions. Finally, the influences of the model proposed in this paper are tested to reduce the computational time by 12.5% compared to the traditional Transformers when applied in practice. These findings establish new state-of-the-art self-attentive methods and sequence-aligned CNNs for reliable and fast machine translation with no significant setbacks.