mulTIDiff model: bridging language and visual representation
摘要
Multilingual text-to-image generation using diffusion models is an emerging area of research that combines the capabilities of language understanding and image synthesis across multiple languages. Our objective was to make a simple model that could take input in any language and provide us with a relevant image as per the prompt. So, we have proposed a model mulTIDiff (multilingual Text-to-Image using Diffusion) that aims to generate images from textual descriptions in various languages, broadening the accessibility and usability of generative models across diverse linguistic contexts. Previous works only supported limited languages. However, our model supports more than a hundred languages. The model comprises two modules for automated translation and image synthesis by leveraging the diffusion method. The performance of the proposed model is evaluated based on metrics such as inception score (IS), Fréchet Inception Distance (FID), and CLIP score.