Vegetables are very diverse, so distinguishing them is not easy for many people. In addition, the quality of vegetables is also an important information which makes the identification task even more complicated. In this work, we have collected and labeled a dataset of Vietnamese vegetable images ourselves. This dataset contains approximately 7,400 images of 6 plant species and 3 quality grades levels. We recommend the use of a Hierarchical Multi-task Learning that concurrently discerns various types of vegetables and assesses their grades quality, while comparing this method against the traditional convolutional neural networks used for classification problem and quality evaluation. Experimental results show that the proposed model significantly outperforms traditional classification models in both accuracy and robustness, demonstrating its effectiveness for vegetable type identification and quality grading.