Prostate cancer is the most common malignant tumor in men and the second leading cause of cancer-related deaths. Its incidence significantly increases with age, leading to a rising number of diagnosed cases yearly. Multiparametric magnetic resonance imaging (mpMRI) has emerged as the preferred diagnostic method for prostate cancer, though interpreting these images can be challenging and requires considerable expertise. This study aims to develop a machine-learning model to segment suspicious lesions in the prostate and evaluate how multi-annotation techniques and cross-validation affect its performance. We utilize a 378 annotated mpMRI cases dataset, exploring various ensembling techniques incorporating expert annotations. Our findings indicate that the proposed model achieves an average precision of 0.76 and AUROC of 0.93–0.95. This represents a significant improvement over a baseline model, with an average precision of 0.69 and an AUROC of 0.92. The results suggest that effective ensemble strategies can enhance the outcome of machine-learning models in prostate cancer diagnostics.