The use of XAI methods has become essential with more and more high-capacity ML models especially deep neural networks more being deployed in areas such as healthcare, finance and law. However, these systems often function as “black boxes,” limiting trust and usability in critical, regulated environments. At the same time, there is a significant amount of complementary information – ontologies, knowledge graphs, and logical constraints – that defines the structured approach to improve the model transparency and its correspondence to the domain. This work proposes a synthesis of this approach in order to develop RI, KBE enhanced, explainable ML systems. The paper offers a plan of action that uses public knowledge repositories (DBpedia, Wikidata etc.), graph-based architectures (Neo4j, GraphDB etc.), and open-source ML toolkits (PyTorch Geometric, RDFLib etc.) to facilitate fast proof-of-concept development of interpretable, knowledge-based solutions. We explain our approach on a toy example in the domain of health care diagnostics and show that the use of domain constraints and a limited knowledge graph leads to better performance and model interpretability. The influence of scaling to real-world datasets is also considered in the work, and the research concludes with a consideration of further prospects for reliable user-oriented AI.