The current work represents a study of different methods used in developing convolutional neural network models for semantic segmentation or pixel classification tasks when dealing with datasets that have noisy labels. We perform some comparisons between different methods that treat multilabel pixel annotation of medical images. We describe the preprocessing step of the dataset and how to obtain agreement masks for the training of deep neural networks. We test these methods on Gleason2019, a dataset of medical images of Prostate Cancer tissue microarray (TMA). We study the task of pixel-level classification of Gleason grade. The scope is to see how our classification models behave when trained on different variations of the computed masks. We also analyze which is the best method to handle the noise found in the annotations of these images.