Scholarlink

This work addresses a research gap in the automated detection of Food Waste (FW) on canteen plates by presenting an innovative system that combines Computer Vision (CV) and Visual Large Language Models (VLLMs) in a hybrid Artificial Intelligence approach. The system uses a pretrained CV model to identify and differentiate between food and garbage, quantify waste, and recognize food items within the plate area. In the CV approach the percentage of food waste is calculated by analyzing segmented regions, measuring the food and garbage areas in pixels, and accounting for possible variations in food density. Since most VLLMs rely on vision encoders pretrained on loosely aligned image-text pairs often resulting in limited visual reasoning capabilities we incorporate Dual-Level Visual Knowledge (DLVK) extracted from a pretrained CV model to enhance understanding. The system is evaluated using a real-world dataset and three benchmarking datasets featuring food on plates. To evaluate the effectiveness of VLLMs in estimating FW from plate images, we designed and tested four prompting strategies of increasing complexity and visual guidance. These strategies were applied across three VLLMs, Gemini 2.5 Pro, GPT-4o, GPT-o3, and LLaVA 1.6 (7b Q:4). Gemini 2.5 Pro, when combined with the refined DLVK, delivers the lowest error and the highest share of explained variance reaching an MSE of 84.69 and an R2 of 0.88, outperforming all other configurations. We conclude that pure segmentation signals whether delivered to a model (ex. LLaVA 1.6) or used in the YOLOv11 baseline were insufficient, confirming that structured priors and explicit reasoning are essential once visual noise, occlusion and class imbalance enter the scene.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Food Waste Detection in Canteen Plates with Visual Large Language Models

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Raffaele Calì,
  • João Ferreira,
  • Paulino Cerqueira,
  • Jorge Ribeiro

摘要

This work addresses a research gap in the automated detection of Food Waste (FW) on canteen plates by presenting an innovative system that combines Computer Vision (CV) and Visual Large Language Models (VLLMs) in a hybrid Artificial Intelligence approach. The system uses a pretrained CV model to identify and differentiate between food and garbage, quantify waste, and recognize food items within the plate area. In the CV approach the percentage of food waste is calculated by analyzing segmented regions, measuring the food and garbage areas in pixels, and accounting for possible variations in food density. Since most VLLMs rely on vision encoders pretrained on loosely aligned image-text pairs often resulting in limited visual reasoning capabilities we incorporate Dual-Level Visual Knowledge (DLVK) extracted from a pretrained CV model to enhance understanding. The system is evaluated using a real-world dataset and three benchmarking datasets featuring food on plates. To evaluate the effectiveness of VLLMs in estimating FW from plate images, we designed and tested four prompting strategies of increasing complexity and visual guidance. These strategies were applied across three VLLMs, Gemini 2.5 Pro, GPT-4o, GPT-o3, and LLaVA 1.6 (7b Q:4). Gemini 2.5 Pro, when combined with the refined DLVK, delivers the lowest error and the highest share of explained variance reaching an MSE of 84.69 and an R2 of 0.88, outperforming all other configurations. We conclude that pure segmentation signals whether delivered to a model (ex. LLaVA 1.6) or used in the YOLOv11 baseline were insufficient, confirming that structured priors and explicit reasoning are essential once visual noise, occlusion and class imbalance enter the scene.