Front-end development often involves the repetitive and time-consuming task of transforming a Graphical User interface (GUI) design into Front-end Code. The GUI design could either be an image or a design created on tools like Figma, Sketch, etc. This process can be particularly challenging when the website designs are experimental and undergo multiple iterations before the final version gets deployed. In such cases, developers work with the designers to make continuous changes and improve the look and feel of the website. This can lead to a lot of rework and a poorly managed codebase that requires significant developer resources. To tackle this problem, researchers are exploring ways to automate the process of transforming image designs into functional websites instantly. This paper explores the use of machine learning, specifically Long Short term Memory (LSTM) networks to generate an intermediate code from an image design and then compile it into a React web front-end code. By utilizing this approach, designers can essentially transform an image design into a functional website, granting them creative freedom and the ability to present working prototypes to stakeholders in real-time. To overcome the limitations of existing publicly available datasets, this paper places significant emphasis on generating synthetic datasets. As part of this effort, the paper proposes a novel method to double the size of the pix2code dataset by incorporating additional complex HTML elements such as login forms, carousels, and cards. This approach has the potential to enhance the quality and diversity of training data available for machine learning models. Overall, the proposed approach offers a promising solution to automate the repetitive and time-consuming task of transforming GUI designs into front-end code.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automating Generation of Web GUI from a Design Image

  • Ajitesh Singh,
  • Ajay Bansal

摘要

Front-end development often involves the repetitive and time-consuming task of transforming a Graphical User interface (GUI) design into Front-end Code. The GUI design could either be an image or a design created on tools like Figma, Sketch, etc. This process can be particularly challenging when the website designs are experimental and undergo multiple iterations before the final version gets deployed. In such cases, developers work with the designers to make continuous changes and improve the look and feel of the website. This can lead to a lot of rework and a poorly managed codebase that requires significant developer resources. To tackle this problem, researchers are exploring ways to automate the process of transforming image designs into functional websites instantly. This paper explores the use of machine learning, specifically Long Short term Memory (LSTM) networks to generate an intermediate code from an image design and then compile it into a React web front-end code. By utilizing this approach, designers can essentially transform an image design into a functional website, granting them creative freedom and the ability to present working prototypes to stakeholders in real-time. To overcome the limitations of existing publicly available datasets, this paper places significant emphasis on generating synthetic datasets. As part of this effort, the paper proposes a novel method to double the size of the pix2code dataset by incorporating additional complex HTML elements such as login forms, carousels, and cards. This approach has the potential to enhance the quality and diversity of training data available for machine learning models. Overall, the proposed approach offers a promising solution to automate the repetitive and time-consuming task of transforming GUI designs into front-end code.