Food recognition is a challenging fine-grained classification task with practical applications in health monitoring, dietary assessment, and intelligent food services. Although convolutional neural networks, transformer-based vision models, and multimodal approaches have advanced rapidly, their comparative strengths for food recognition remain insufficiently explored. To bridge this gap, in this paper we present a unified experimental framework that enables robust evaluation and yields new insights into the design and deployment of effective food recognition systems. Results reveal that transformer architectures consistently outperform convolutional baselines, achieving over 94% accuracy, while multimodal frameworks demonstrate competitive zero-shot performance without task-specific fine-tuning. Beyond raw performance, our analysis uncovers systematic error patterns linked to high intra-class variability and visual similarity across food categories. We further introduce a lightweight web application for real time inference and explainable predictions, highlighting the practical implications of our findings.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Food Image Recognition: From CNNs to Transformers and Multimodal Learning

  • Onisim Constantin,
  • Ruxandra Tapu,
  • Bogdan Mocanu,
  • Mirela Grosu

摘要

Food recognition is a challenging fine-grained classification task with practical applications in health monitoring, dietary assessment, and intelligent food services. Although convolutional neural networks, transformer-based vision models, and multimodal approaches have advanced rapidly, their comparative strengths for food recognition remain insufficiently explored. To bridge this gap, in this paper we present a unified experimental framework that enables robust evaluation and yields new insights into the design and deployment of effective food recognition systems. Results reveal that transformer architectures consistently outperform convolutional baselines, achieving over 94% accuracy, while multimodal frameworks demonstrate competitive zero-shot performance without task-specific fine-tuning. Beyond raw performance, our analysis uncovers systematic error patterns linked to high intra-class variability and visual similarity across food categories. We further introduce a lightweight web application for real time inference and explainable predictions, highlighting the practical implications of our findings.