We discuss a number of engineering challenges arising when integrating conversational AI with conventional GUI-based interaction in an application. The issues addressed range from the architectural design of integrated applications and the provisioning of shared knowledge as common ground to specific questions of interaction and conversation design and implementation. We exemplify the challenges using a multi-method recommendation system. The conversational and direct manipulation paradigms have complementary advantages that may be integrated and leveraged to create applications that offer users a high level of flexibility and support. Yet, integrating GUI elements and conversation in an interface entails a range of design and engineering challenges, many of which have not been sufficiently addressed in existing research. By describing some of these challenges in a structured form, we aim at providing an initial framework for future research and development.