Artificial Intelligence has revolutionised industries, enhancing productivity and efficiency across different sectors. However, its deployment in life-critical domains such as road and air traffic safety, demands trustworthiness in the sustainable development of AI technology. The challenge is to develop AI solutions with long-term sustainability in mind, ensuring they remain relevant and reliable to the users. This paper highlights the role of eXplainable AI (XAI) and transparency in ensuring trustworthiness and fostering sustainability in AI solutions. XAI techniques, which can often be interpretable models and post-hoc explanations, are particularly vital in domains where AI decisions have significant implications, such as predicting machinery failures in manufacturing. Despite its potential, achieving explainability remains challenging due to trade-offs between model complexity and interpretability, the absence of universal standards, and diverse societal expectations. The paper outlines key steps to ensure explainability and transparency in reliable AI development supported by relevant use cases.