Knowing the Interviewer: Friend or Foe? Differences in Ease, Frustration and Quit Rates Between Friends and Strangers in Usability Testing
摘要
Companies may use convenience sampling (e.g., coworkers) for their usability test participants. This like-minded, motivated sample could downplay usability issues that could occur in the rest of the population. We developed a purposely frustrating (counterintuitive navigation pathways) high-fidelity chat app prototype to test whether the relationship between the interviewee and the interviewer (friends or stranger) influences the interviewee in sharing directly and indirectly their frustration. Six (balanced genders) interviewers had participants who were either their friends or strangers complete six tasks using the cognitive walkthrough procedure. Friends rated the prototype as more easy, less frustrating, and had significantly lower variation in ease ratings than strangers. Additionally, friends quit the task significantly less often than strangers for the most difficult tasks.