Crop insurance is a critical tool for safeguarding farmers from financial losses caused by unforeseen events such as extreme weather, insect and pest infestations, and diseases. However, current claims processes including manual documentation and submission of damage evidence, are inefficient and slow. To address these challenges, this paper proposes an automated system that leverages machine learning techniques to classify and assess crop damage using images taken by farmers. This approach, known as picture-based insurance, enables rapid and accurate loss estimation based on image data submitted through a smartphone app. The reported work evaluated several machine learning models, including state-of-the-art architectures like Vision Transformers, ConvNeXt, and Swin Transformers, to classify different types of crop damage. In order to evaluate the model’s performance, standard metrics to assess accuracy, precision, recall and f1-score were used. The experimental results demonstrate the effectiveness of the proposed approach in automating crop damage detection and classification, offering significant potential to improve the efficiency of crop insurance claims processing.