Non-stationary dueling bandits are promising for modeling evolving preferences in human-robot interaction, but current evaluations rely on synthetic data and ignore user experience. This short contribution proposes a regret formulation that incorporates interaction costs—such as annoyance, cognitive effort, and trust degradation. Furthermore, suggestions for standardized, user-in-the-loop evaluation is presented. Recognizing the challenges of long-term studies with real users, large language models could be used to simulate non-stationary user behavior for early-stage testing. This enables deeper insight into algorithm usability and supports the development of adaptive systems that are not only efficient, but also aligned with human expectations and experience.