Food segmentation is essential for dietary monitoring, yet it faces significant challenges due to the scarcity of pixel-level labels and the need to accurately segment foods from new images. We propose a novel method that addresses these challenges by leveraging selectively pre-trained convolutional neural network (CNN) deep features for pixel-wise clustering, requiring minimal pixel-level labels. We begin by generating pixel-wise feature representations from a convolutional layer and concatenated multiple layers. We then select the most suitable representations, perform pixel-wise clustering by k-means, and choose the clusters with the best clustering quality. Our method performed well on three public food segmentation datasets, achieving an aAcc of 0.8967 and mIoU 0.8094 on UECFOODPIX COMPLETE, 0.8750 and 0.7768 on FoodSeg103, and 0.9304 and 0.8373 on UNIMIB2016. These results show that our method achieves segmentation performance close to supervised models while reducing reliance on extensive pixel-level labels. It also adapts well to diverse food images, making it suitable for various applications.