Identifying special food is helpful for the preservation and inheritance of special food, and it also helps to improve the level of people's better life. Usually when encountering unfamiliar food, people can only ask people around or ask for help online but this requires a long process and may get wrong answers. Therefore, in order to help with the identification of special food and promote special food, we decided to study the possibility of automatic classification of special food. In this project, we collected a total 1800 pictures of Malaysian specialties from multiple data sources such as Google Images, and used 3 classification models of transfer learning (Inception V3, Xception, and ResNet50) and a proposed classification model called Small Res CNN which trains and tests it. Finally, in independent tests, our proposed lightweight model can achieve 83% accuracy, and Xception can achieve 87.4% accuracy. This shows that the CNN-based transfer learning model can better realize the automatic classification of special food, and it also shows that the lightweight model can also achieve similar results.