[Context and motivation] User stories are central in agile software development, yet creating and managing high-quality ones remains challenging. Existing tools for user story quality enhancement often lack semantic analysis or project-specific context alignment. [Questions] This paper investigates how multi-agent systems (MAS) based on large language models (LLMs) can refine user story quality and strengthen traceability by linking them to project artifacts, while preserving human oversight. [Results] We present a context-aware system that integrates LLMs with retrieval-augmented generation (RAG) to refine, group, and link user stories to Jira tickets. Implemented using LangGraph, the system distributes reasoning across designed agents and allows feedback loops for oversight. The implementation was evaluated on three projects comprising ten user stories each, with project members participating as experts in the assessment. Results show notable improvements in user story quality in completeness (mean +2.32) and testability (mean +2.01), and overall quality (mean +0.84), while inter-rater agreement indicated high reliability for user story grouping ( \(\alpha \)  = 0.771) and modest agreement in linking stories with Jira tickets ( \(\alpha \)  = 0.491). [Contribution] The study contributes an LLM-based architecture for AI-assisted requirements refinement and management. It demonstrates how modular agent roles, RAG-based contextual information, and human-in-the-loop evaluation enhance the performance. It illustrates evidence of both the opportunities and current limitations of AI-driven requirements management.