Creating high-quality, scalable 3D environments for Extended Reality (XR) applications is often labor-intensive, requiring manual modeling, mesh optimization, and texturing. This paper introduces a novel AI-powered pipeline—developed under the SENSO3D project—that automates the transformation of 2D photographs and textual prompts into fully textured 3D objects and immersive scenes. Leveraging advanced models such as F-Cube R-CNN for object detection and F-TripoSR for single-image 3D reconstruction, the system generates 3D assets enriched with AI-driven texture synthesis and semantic categorization. These assets are seamlessly integrated into Unity, enabling real-time deployment in XR contexts such as virtual conferences, product showcases, and interactive training environments. The pipeline achieved a reconstruction accuracy of 92% across varied environments, with object detection precision exceeding 95% across 50+ classes. Additionally, a prompt-based scene generator empowers non-technical users to create complete 3D environments from natural language input. The entire workflow is optimized for speed and scalability, reducing development time from days to minutes. This work demonstrates how AI can bridge the gap between static visual data and dynamic immersive experiences, enabling faster, more accessible XR content creation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SENSO3D: Structured Pipelines for AI-Based 3D Object Detection, Classification, and Texture Generation

  • Mohammad Mohammad Amini,
  • Ali Hajqani,
  • Mohammad Hasan Bahari,
  • Davood Fanaei Sheikholeslami,
  • Ali Mazaheri,
  • Mohammad Mohammadi

摘要

Creating high-quality, scalable 3D environments for Extended Reality (XR) applications is often labor-intensive, requiring manual modeling, mesh optimization, and texturing. This paper introduces a novel AI-powered pipeline—developed under the SENSO3D project—that automates the transformation of 2D photographs and textual prompts into fully textured 3D objects and immersive scenes. Leveraging advanced models such as F-Cube R-CNN for object detection and F-TripoSR for single-image 3D reconstruction, the system generates 3D assets enriched with AI-driven texture synthesis and semantic categorization. These assets are seamlessly integrated into Unity, enabling real-time deployment in XR contexts such as virtual conferences, product showcases, and interactive training environments. The pipeline achieved a reconstruction accuracy of 92% across varied environments, with object detection precision exceeding 95% across 50+ classes. Additionally, a prompt-based scene generator empowers non-technical users to create complete 3D environments from natural language input. The entire workflow is optimized for speed and scalability, reducing development time from days to minutes. This work demonstrates how AI can bridge the gap between static visual data and dynamic immersive experiences, enabling faster, more accessible XR content creation.