Recent advancements in Multimodal Large Language Models (MLLMs) have created new opportunities for medical AI, particularly through the integration of Medical Vision-Language Models (VLMs) into clinical applications. While these models exhibit robust capabilities in processing multimodal medical data, their practical adoption remains challenging due to the complexity of patient-physician interactions and clinical workflows. Understanding how to effectively adapt these models for real-world use is therefore essential. We propose a practical-oriented method that aligns multimodal medical data with synthetic patient-physician dialogues. Rather than relying on real-world dialogue datasets, we generate structured medical conversations using VLM-based techniques. This study aims to bridge the gap between Medical VLM research and clinical applicability by examining whether MLLMs trained on synthetic dialogues can effectively support AI-assisted medical decision-making. We expect our findings to contribute to the development of safer, more reliable AI-driven clinical assistants and to facilitate their successful implementation in healthcare settings.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Aligning Multimodal Large Language Models with Patient-Physician Dialogues for AI-Assisted Clinical Support

  • Junyong Lee,
  • Jeihee Cho,
  • Jiwon Ryu,
  • Shiho Kim

摘要

Recent advancements in Multimodal Large Language Models (MLLMs) have created new opportunities for medical AI, particularly through the integration of Medical Vision-Language Models (VLMs) into clinical applications. While these models exhibit robust capabilities in processing multimodal medical data, their practical adoption remains challenging due to the complexity of patient-physician interactions and clinical workflows. Understanding how to effectively adapt these models for real-world use is therefore essential. We propose a practical-oriented method that aligns multimodal medical data with synthetic patient-physician dialogues. Rather than relying on real-world dialogue datasets, we generate structured medical conversations using VLM-based techniques. This study aims to bridge the gap between Medical VLM research and clinical applicability by examining whether MLLMs trained on synthetic dialogues can effectively support AI-assisted medical decision-making. We expect our findings to contribute to the development of safer, more reliable AI-driven clinical assistants and to facilitate their successful implementation in healthcare settings.