Three-dimensional ultrasound (US) offers many clinical advantages over conventional 2D imaging, yet its widespread adoption is limited by the cost and complexity of traditional 3D systems. Sensorless 3D US, using deep learning to estimate a 3D probe trajectory from a sequence of 2D US images, is a promising alternative.Local features such as speckle patterns can help predict frame-to-frame motions, while global features, such as coarse shapes and anatomical structures, can situate the scan relative to anatomy and help predict its general shape. In prior approaches, global features are either ignored or tightly coupled with local feature extraction, restricting the ability to robustly model these two complementary aspects. We propose DualTrack, a novel dual encoder architecture leveraging decoupled local and global encoders specializing in their respective scale of feature extraction. The local encoder uses dense spatiotemporal convolutions to capture fine-grained features, while the global encoder utilizes an image backbone such as a 2D CNN or foundation model and temporal attention layers to embed high-level anatomical features and long-range dependencies. A lightweight fusion module then combines these features to estimate trajectory. Experimental results on a large public benchmark show that DualTrack achieves state-of-the-art accuracy and globally consistent 3D reconstructions, outperforming previous methods and yielding an average reconstruction error below 5 mm. (Code and model weights are provided at https://github.com/ImFusionGmbH/DualTrack .)

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

DualTrack: Sensorless 3D Ultrasound needs Local and Global Context

  • Paul F. R. Wilson,
  • Matteo Ronchetti,
  • Rüdiger Göbl,
  • Viktoria Markova,
  • Sebastian Rosenzweig,
  • Raphael Prevost,
  • Parvin Mousavi,
  • Oliver Zettinig

摘要

Three-dimensional ultrasound (US) offers many clinical advantages over conventional 2D imaging, yet its widespread adoption is limited by the cost and complexity of traditional 3D systems. Sensorless 3D US, using deep learning to estimate a 3D probe trajectory from a sequence of 2D US images, is a promising alternative.Local features such as speckle patterns can help predict frame-to-frame motions, while global features, such as coarse shapes and anatomical structures, can situate the scan relative to anatomy and help predict its general shape. In prior approaches, global features are either ignored or tightly coupled with local feature extraction, restricting the ability to robustly model these two complementary aspects. We propose DualTrack, a novel dual encoder architecture leveraging decoupled local and global encoders specializing in their respective scale of feature extraction. The local encoder uses dense spatiotemporal convolutions to capture fine-grained features, while the global encoder utilizes an image backbone such as a 2D CNN or foundation model and temporal attention layers to embed high-level anatomical features and long-range dependencies. A lightweight fusion module then combines these features to estimate trajectory. Experimental results on a large public benchmark show that DualTrack achieves state-of-the-art accuracy and globally consistent 3D reconstructions, outperforming previous methods and yielding an average reconstruction error below 5 mm. (Code and model weights are provided at https://github.com/ImFusionGmbH/DualTrack .)