<p>Colonoscopy is currently one of the most sensitive screening methods for colorectal cancer. This study investigates the frontiers of intelligent colonoscopy techniques and their prospective implications for multimodal medical applications. With this goal, we begin by assessing the current data-centric and model-centric landscapes through four tasks for colonoscopic scene perception, including classification, detection, segmentation, and vision-language understanding. Our assessment reveals domain-specific challenges and underscores the need for further multimodal research in colonoscopy. To address these gaps, we establish three foundational initiatives: a large-scale multimodal instruction tuning dataset ColonINST, a colonoscopy-designed multimodal language model ColonGPT, and a multimodal benchmark. To facilitate continuous advancements in this rapidly evolving field, we provide a public website for the latest updates: <a href="https://github.com/ai4colonoscopy/IntelliScope">https://github.com/ai4colonoscopy/IntelliScope</a>.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Frontiers in Intelligent Colonoscopy

  • Ge-Peng Ji,
  • Jingyi Liu,
  • Peng Xu,
  • Nick Barnes,
  • Fahad Shahbaz Khan,
  • Salman Khan,
  • Deng-Ping Fan

摘要

Colonoscopy is currently one of the most sensitive screening methods for colorectal cancer. This study investigates the frontiers of intelligent colonoscopy techniques and their prospective implications for multimodal medical applications. With this goal, we begin by assessing the current data-centric and model-centric landscapes through four tasks for colonoscopic scene perception, including classification, detection, segmentation, and vision-language understanding. Our assessment reveals domain-specific challenges and underscores the need for further multimodal research in colonoscopy. To address these gaps, we establish three foundational initiatives: a large-scale multimodal instruction tuning dataset ColonINST, a colonoscopy-designed multimodal language model ColonGPT, and a multimodal benchmark. To facilitate continuous advancements in this rapidly evolving field, we provide a public website for the latest updates: https://github.com/ai4colonoscopy/IntelliScope.