<p>This paper introduces a behavior-driven, closed-loop framework for adaptive user interface (UI) generation powered by generative AI models. Unlike traditional open-loop approaches that treat prompts as static and rely solely on random latent sampling, our method incorporates real-time user feedback—such as replacements, likes, and dwell behaviors–into an iterative generation loop. The architecture integrates a preference prediction module, a prompt fusion mechanism, and a diffusion-based synthesis engine. As users interact with the system, a latent preference vector, <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\({\textbf{p}}_i\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi mathvariant="bold">p</mi> <mi>i</mi> </msub> </math></EquationSource> </InlineEquation>, is continuously updated, and used to adapt the control prompt, <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\({\textbf{c}}_i\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi mathvariant="bold">c</mi> <mi>i</mi> </msub> </math></EquationSource> </InlineEquation>, for subsequent iterations. We conducted a within-subject user study (<InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(N=20\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>N</mi> <mo>=</mo> <mn>20</mn> </mrow> </math></EquationSource> </InlineEquation>) comparing our system to a baseline open-loop method across four realistic UI design tasks. The results show that our framework improves satisfaction scores (6.15 vs. 4.35), reduces the number of replace iterations (3.2 vs. 6.8), and shortens task completion time by 27%. Additional metrics–such as preference convergence and style consistency–further support the system’s ability to internalize and reflect user intent. Visual analyses of round-wise behavior trends and radar plots also suggest that users reach their desired outcomes with less effort and faster alignment. Taken together, these findings highlight the potential of real-time behavior modeling and adaptive prompt fusion in enabling scalable, personalized, and user-aware UI generation. Our framework provides a promising direction for developing co-creative and intelligent design systems, with broader implications for adaptive HCI, low-code development, and generative design platforms.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Behavior-Driven Adaptive User Interface Generation Framework with Iterative Preference Modeling and Prompt Fusion

  • Juan Chen,
  • Bochen Chen,
  • Jingyi Lei,
  • Xiaotong He,
  • Ling Chen,
  • Won SukLing Kim

摘要

This paper introduces a behavior-driven, closed-loop framework for adaptive user interface (UI) generation powered by generative AI models. Unlike traditional open-loop approaches that treat prompts as static and rely solely on random latent sampling, our method incorporates real-time user feedback—such as replacements, likes, and dwell behaviors–into an iterative generation loop. The architecture integrates a preference prediction module, a prompt fusion mechanism, and a diffusion-based synthesis engine. As users interact with the system, a latent preference vector, \({\textbf{p}}_i\) p i , is continuously updated, and used to adapt the control prompt, \({\textbf{c}}_i\) c i , for subsequent iterations. We conducted a within-subject user study ( \(N=20\) N = 20 ) comparing our system to a baseline open-loop method across four realistic UI design tasks. The results show that our framework improves satisfaction scores (6.15 vs. 4.35), reduces the number of replace iterations (3.2 vs. 6.8), and shortens task completion time by 27%. Additional metrics–such as preference convergence and style consistency–further support the system’s ability to internalize and reflect user intent. Visual analyses of round-wise behavior trends and radar plots also suggest that users reach their desired outcomes with less effort and faster alignment. Taken together, these findings highlight the potential of real-time behavior modeling and adaptive prompt fusion in enabling scalable, personalized, and user-aware UI generation. Our framework provides a promising direction for developing co-creative and intelligent design systems, with broader implications for adaptive HCI, low-code development, and generative design platforms.