Tongue segmentation from the human face has been a crucial prerequisite step for medical diagnosis in traditional Chinese medicine. Previous methods require pixel-level labels to do tongue segmentation, where pixel-level labels are expensive and time-consuming to obtain especially for small clinics. In this paper, we propose an automatic tongue image segmentation method with no labels. Our method can obtain clear tongue mask from tongue image without any pixel-level labels. We demonstrate the effectiveness of our proposed method on a large scale real-world tongue image dataset. Our method achieves a nearly matching result compared with state-of-the-art supervised semantic segmentation methods, where pixel-level labels are needed.