In recent years contrastive learning has become the prevalent approach to training embedding models used in tasks such as information retrieval. Although, due to the nature of contrastive learning, during which negative samples are pushed further apart and positive samples are brought closer together in the embedding space, it imposed several new challenges for effective training. One such challenge lies in the creation of adequately hard negative samples. Most commonly hard-negative samples are created automatically by leveraging an existing retrieval model, which in turn introduces the risk of generating false negative training samples. To combat that, several filtering approaches that rely on ranks or similarity scores have been proposed. The fundamental flaw in those approaches lies in the ambiguity of relevance judgment given by the ranking models. Given only similarity scores, it is hard to accurately determine whether a particular document constitutes a positive or a negative sample. Our proposed method uses LLMs to resolve that issue by determining an optimal cutoff position through generating binary relevance labels. Such an approach also largely mitigates the problems with determining optimal filtration parameters present when dealing with raw relevance scores. The effectiveness of our method has been tested empirically by fine-tuning open-source retrieval models BGE-Reranker-v2-m3 and multilingual-e5-base. Our experiments on publicly available datasets have shown improvements in ranking metrics up to 2,29% in R-Precision and 1,12% in NDCG@10 compared to existing approaches.