Due to the emergence of global challenges, the European policy called Industry 5.0 demands technologies to become resilient, sustainable and humancentric; the integration of Artificial Intelligence (AI) and Knowledge Management (KM) (i.e., AI-based KM) represents an opportunity to accomplish this ambitious goal thanks to the possibility of combining humans and computers’ unique skills. While the compliance of AI-based KM with human-centric requirements is apparent, its long-term sustainability is still an object of discussion. In response to I5.0’s request for metrics of sustainability, this paper presents a lifecycle analysis of AI-based KM’s applications retrieved from literature to establish whether they can be deemed socially, environmentally, and economically sustainable. A list of sources was collected by means of a literature search, to be later evaluated through sustainability criteria specific for the design and use phases of life. It was found that AI-based KM applications supporting fractions of the knowledge lifecycle may become fragile in the long-term, because potentially leading to repositories’ indefinite growth and information overload in humans and computers; furthermore, it emerged that not enough attention was paid to AI’s accountability and data governance. The mentioned pain points, complemented in this article with practical examples of the counter-measures adopted by virtuous sources, provide scholars with practical indications to lead future research efforts, and contribute to the validation of a framework to assess the longterm sustainability of AI solutions.