Reception and social integration of Third Country Nationals (TCNs) remains a pressing challenge in contemporary societies shaped by migration. The SALLY project supports the integration of migrants in Greece through a multilingual conversational agent designed to assist users in accessing information related to legal, healthcare, employment, and social services. This paper presents preliminary work on a knowledge extraction framework that captures key elements from natural language conversations—such as topics, entities, and relationships—using Large Language Models (LLMs). These are structured as RDF Knowledge Graphs guided by an ontology that reflects the dynamics of user-agent interaction. By translating informal dialogue into semantically meaningful representations, the system offers a foundation for better understanding user needs and behavior, while enhancing the transparency and responsiveness of the SALLY conversational agent.