Diagnostic Trouble Codes (DTCs) produced by On-Board Diagnostic Systems (OBDs), and the research focused on their use for predictive maintenance have been around for a while now. In the last few years, we have witnessed advancement in terms of how these DTCs are utilised to perform self-supervised end-to-end prediction with the introduction of sequential prediction models, where the goal is to utilize past occurred fault events to predict the next DTC fault event. These models mainly use neural embeddings to encode the DTCs, along with their features, before applying neural networks capable, in turn, of processing sequential data. For instance, DTC-TranGru, which uses a GRU layer on top of a Transformer, has reported better results than LSTM and Attention-based models. In this paper, we first put forward an enhanced version of the DTC-TranGru model called DTC-GOAT (GRU’s Optimized Alignment with Transformer), proposing optimizations including a better alignment of the Transformer with GRU’s output, end-of-sequence EOS tokens, and strategically placed 1D spatial-dropout layers, to boost the accuracy of DTC prediction. Secondly, we also introduce an so-called Ensemble approach that uses multiple models for next-DTC prediction and show that it gives slightly higher top-5 accuracy results than the individual models.