Food image generation, as a crucial branch of multimodal content generation, holds significant application value in intelligent catering systems and dietary culture digitization. However, existing diffusion models face three major challenges when adapted to the food domain: insufficient material detail and realism, high computational costs of full-parameter fine-tuning, and weak adaptability of traditional parameter-efficient methods. To address these issues, this paper proposes PromptDiff, a food image generation framework based on hierarchical parameter-efficient fine-tuning. Specifically, we construct ingredient prompt vectors in the input space as semantic prior constraints for the generator, while utilizing a dynamic conditional weight generation module for domain feature adaptation. Finally, hierarchical parameter selection is employed to fine-tune only 0.4 \(\%\) of network parameters, avoiding computational redundancy from global adjustments. Experiments show that the proposed method achieves 1.5 \(\times \) higher training efficiency than full parameter fine-tuning while surpassing mainstream fine-tuning methods in performance. Notably, this method requires only 15 MB of storage space (a 98 \(\%\) reduction from the baseline model) to achieve optimal generation quality, providing a solution that balances performance and efficiency for food image generation.