A vision transformer ensemble and mobile augmented reality solution for mushroom toxicity classification
摘要
Wild mushrooms found in nature are often collected consciously or unconsciously based on traditional or ancestral knowledge and consumed as food in many cultures. However, this practice can lead to serious health risks, including fatalities, due to the accidental consumption of poisonous species. To address this issue, this study proposes a novel deep learning-based approach, Vision Transformer-Based Stacking Ensemble combined with a Mobile Augmented Reality application, to classify wild mushrooms as edible or poisonous. A custom dataset containing 14.000 images representing 14 mushroom classes, 7 edible and 7 poisonous, was generated using publicly available internet resources. Two Vision Transformer models, ViT-B16 and ViT-B32, were individually fine-tuned and modified by adjusting their Multi-Layer Perceptron heads and classification layers. Their prediction outputs were then integrated using a stacking ensemble method with a meta-learner to build a more robust and generalized model. Experimental results demonstrate that the ViT-B16 model achieved 96.78% accuracy, ViT-B32 achieved 93.21%, while the proposed ViT-BSE model significantly outperformed both with an accuracy of 99.04%. This indicates that the ensemble strategy successfully combines the strengths of individual ViT architectures and enhances classification performance, especially in complex visual tasks like mushroom identification. Furthermore, the developed AR application allows users to receive real-time feedback on mushroom edibility in the field, providing both an educational tool and a safety mechanism. Compared to existing models in the literature, the ViT-BSE approach demonstrates superior performance and highlights the potential of transformer-based ensembles in biological image classification.