Recommender systems have gained a noticeable popularity due to their ability to enhance the user experience with personalized and accurate suggestions, especially in the era of data explosion, where finding what aligns with the needs of the user requires going through a long search journey. One of the recommender system approaches that has marked a great success is collaborative filtering. However, as a technique that predicts suggestions based on the historical interactions of similar users, it remains vulnerable to multiple challenges such as cold start, data sparsity, and grey sheep user detection. In this paper, we provide a review of various methods used to detect grey sheep users. Through this review, we identified several challenges related to the evaluation protocol. To address these challenges, we propose a new evaluation protocol aiming to improve the evaluation process of grey sheep user detection. The evaluation results showed that the proposed protocol outperforms current evaluation protocols.