Multi-label learning in food image recognition presents a promising avenue for understanding the visual composition of meals through joint ingredient prediction. In this article, we improve an existing GCN-based framework by replacing its standard co-occurrence matrix with a novel semantic variant, constructed using large language models (LLMs). Unlike traditional approaches that derive co-occurrence statistics solely from the training data which often introducing biases and limiting generalization, our method leverages prior knowledge extracted from LLMs to build an adjacency matrix that captures broader and more contextually grounded ingredient relationships. We evaluated our approach on the MAFood-121 and VireoFood-172 datasets, significantly outperforming the benchmark method that relies on dataset-conditioned co-occurrence graphs. On MAFood-121, our model improved the mean average precision (mAP) from 82.77% to 87.46%, while on VireoFood-172, it increased from 60.88% to 65.28%. The results demonstrate the effectiveness of integrating LLM-derived semantic structure into graph-based multi-label models for structured food recognition.