Dynamic Spatial-Aware Network for Occlusion Robust 3D Hand Mesh Reconstruction from RGB Images
摘要
Reconstructing 3D hand mesh from a monocular RGB image is pivotal for human-computer interaction. However, this task becomes highly challenging due to severe occlusions during hand-object interactions. Existing methods suffer from fixed convolutional kernels of CNNs and unreliable similarity computations in Transformers. Consequently, they fail to solve dynamic occlusions and background interference, leading to pose inaccuracies and mesh distortions. To overcome these challenges, we propose HandDSA, an occlusion-robust network that dynamically exploits spatial-aware information to enhance reconstruction quality. First, we propose the Spatial Attention-Guided Dynamic Convolution (SAD-Conv) that dynamically generates region-specific kernels by analyzing spatial patterns. Specifically, smaller receptive fields are employed in visible hand regions to obtain detailed local features while larger receptive fields are utilized in occluded interaction regions to capture global contextual semantics. Then, we design the Gated Attention Filtering (GAF) that combines channel-spatial attention with gated mechanism to adaptively suppress irrelevant interference from background regions. Extensive experiments demonstrate the efficacy of our approach across multiple evaluation metrics on the HO3Dv2 and DexYCB datasets.