As economic conditions improve, the demand for vegetables, fruits, meat, seafood, and other products continues to rise, indicating a highly favourable market outlook. However, food is also susceptible to corruption and spoiling, resulting in considerable waste and economic losses. Managing the quality, safety, and marketability of fresh produce requires accurate prediction of its shelf life. Traditional methods of predicting shelf life are based on visual feedback, which is time-consuming and needs manpower. In this paper, the power of transformer-based models, i.e., Vision Transformer, is utilized to predict the shelf life of fruits and vegetables. Here, collected RGB images are analyzed to analyze the texture variation and decay pattern to classify them as fresh and rotten, and to predict their shelf life. The model gets an accuracy of 96.82%, which shows the efficacy of the proposed architecture with conventional CNN-based approaches. The proposed framework shows its efficacy in providing a solution for minimizing food wastage.