<p>3D Gaussian splatting has emerged as a promising technique for real-time scene representation, making interactive 3D segmentation increasingly important for scene manipulation. However, inconsistent 2D segmentation results across different viewpoints present significant challenges in learning 3D segmentation feature fields. When cross-view 2D segmentation results conflict, the accuracy of 3D segmentation decreases substantially. To address this, we present Spatio-temporal Feature-guided Learning for 3D Gaussian Segmentation (SFL-GS), an efficient interactive 3D segmentation framework. SFL-GS employs a novel Spatio-temporal Feature-guided Learning (SFL) strategy that captures spatio-temporally consistent features and masks from 2D segmentation results across multiple views, effectively guiding the learning of 3D segmentation feature fields. Given the computational complexity of 3D segmentation, high-performance computing (HPC) capabilities are essential to process complex scenes with high accuracy. Additionally, to refine feature and mask consistency in challenging scenarios, particularly under severe occlusion, our framework incorporates an enhanced optimization strategy that combines statistical filtering, dynamic scale growth, and edge-aware optimization. This approach results in clearer boundaries and significantly improves segmentation accuracy, even in difficult environments. Extensive experiments demonstrate that our method achieves superior accuracy in segmentation tasks, making it well-suited for precise and efficient 3D segmentation.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SFL-GS: spatio-temporal feature-guided learning for 3D Gaussian segmentation

  • Fang Wan,
  • Xianjin Shi,
  • Tianyu Li,
  • Guangbo Lei,
  • Li Xu,
  • Zhiwei Ye

摘要

3D Gaussian splatting has emerged as a promising technique for real-time scene representation, making interactive 3D segmentation increasingly important for scene manipulation. However, inconsistent 2D segmentation results across different viewpoints present significant challenges in learning 3D segmentation feature fields. When cross-view 2D segmentation results conflict, the accuracy of 3D segmentation decreases substantially. To address this, we present Spatio-temporal Feature-guided Learning for 3D Gaussian Segmentation (SFL-GS), an efficient interactive 3D segmentation framework. SFL-GS employs a novel Spatio-temporal Feature-guided Learning (SFL) strategy that captures spatio-temporally consistent features and masks from 2D segmentation results across multiple views, effectively guiding the learning of 3D segmentation feature fields. Given the computational complexity of 3D segmentation, high-performance computing (HPC) capabilities are essential to process complex scenes with high accuracy. Additionally, to refine feature and mask consistency in challenging scenarios, particularly under severe occlusion, our framework incorporates an enhanced optimization strategy that combines statistical filtering, dynamic scale growth, and edge-aware optimization. This approach results in clearer boundaries and significantly improves segmentation accuracy, even in difficult environments. Extensive experiments demonstrate that our method achieves superior accuracy in segmentation tasks, making it well-suited for precise and efficient 3D segmentation.