DirectPose-VTON: A direct 3D parameter injection paradigm for geometrically-consistent virtual try-on
摘要
Existing high-fidelity Virtual try-on (VTON) models often exhibit noticeable geometric distortions when dealing with non-frontal views or occluded human poses. We identify this limitation as stemming from the dominant indirect guidance paradigm, where essential 3D structural information is lost during the projection of human poses into 2D representations. To address this problem, we propose DirectPose-VTON, a novel framework built upon the concept of Direct 3D Parameter Injection. Specifically, our method directly incorporates numerical Skinned Multi-Person Linear (SMPL) parameters into a pre-trained diffusion backbone through a Multi-Layer Perceptron (MLP) encoder and a cascaded cross-attention mechanism. Our proposed approach preserves critical 3D geometric cues, thereby enabling the diffusion model to generate photo-realistic and pose-consistent try-on results. Extensive experiments on two public benchmarks, VITON-HD and DressCode, demonstrate that DirectPose-VTON consistently outperforms representative baselines across multiple quantitative and qualitative metrics. Moreover, evaluations on a specifically curated Challenging Pose Subset, corroborated by human preference studies, highlight substantial gains in producing geometrically accurate and perceptually convincing images. Crucially, robustness analyses reveal that our spatially-agnostic parameter injection enables graceful degradation against upstream pose estimation noise, effectively preventing the catastrophic structural failures typical of 2D-conditioned baselines. These findings confirm that direct 3D parameter injection offers a powerful and generalizable strategy for mitigating geometric inconsistencies in modern VTON systems.