Recent advances in generative artificial intelligence are reshaping our daily lives. Large language models (LLMs) – the technology underlying chatbots like ChatGPT – are able to produce coherent text responses upon user prompts. For this reason, LLMs are being used to automate tasks in many disciplines, like law, human resources, marketing, or media content creation. Software development is no exception to this trend, and conversational assistants based on LLMs have started to appear. However, there is still the need to understand the integration and interaction possibilities of these assistants within integrated development environments (IDEs), enabling the addition of new assistive tasks in a simple manner, coordinating multiple assistants, and tracing the assistants’ contributions to the project under development. We tackle this gap by exploring alternatives for integrating assistants within IDEs, and proposing a general architecture for conversational assistance in IDEs. The architecture features extensibility mechanisms to add new assistive tasks externally without resorting to programming, a rich traceability model of the user-assistant interaction, and a multi-assistant coordination model. We have realised our proposal within Eclipse, building an assistant for Java development called Caret. The assistant supports tasks like code completion, documentation, code comprehension, maintenance and testing, but can be easily extended with additional ones. Finally, we present an evaluation for one of these tasks: method renaming. The evaluation results are promising since the recommendations of our assistant were generally perceived as more appropriate than the original method names and a baseline.