Smart environments, such as smart homes, promise to enhance our daily lives. How we interact with these spaces, which encompass a wide range of devices and functionalities, remains a challenge that research seeks to address with increasingly natural forms of interaction, such as speech. However, the latter may not be appropriate in various situations (e.g., noisy environments, speech difficulties) where gestures can be a suitable alternative (or complement). To inform our research on designing gesture-based interaction for smart environments, we conducted a focus group to gather users’ views and preferences regarding the use of gestures to interact with smart home devices. Overall, gestures were found to be better suited for discrete binary commands (e.g., on/off, previous/next) and participants suggested that actions, not the controlled devices, should dictate the gestures. This study provides useful insights on users’ expectations and mental models for gesture interaction in smart environments, informing future research and development.