Aligning Multimodal Large Language Models with Patient-Physician Dialogues for AI-Assisted Clinical Support
摘要
Recent advancements in Multimodal Large Language Models (MLLMs) have created new opportunities for medical AI, particularly through the integration of Medical Vision-Language Models (VLMs) into clinical applications. While these models exhibit robust capabilities in processing multimodal medical data, their practical adoption remains challenging due to the complexity of patient-physician interactions and clinical workflows. Understanding how to effectively adapt these models for real-world use is therefore essential. We propose a practical-oriented method that aligns multimodal medical data with synthetic patient-physician dialogues. Rather than relying on real-world dialogue datasets, we generate structured medical conversations using VLM-based techniques. This study aims to bridge the gap between Medical VLM research and clinical applicability by examining whether MLLMs trained on synthetic dialogues can effectively support AI-assisted medical decision-making. We expect our findings to contribute to the development of safer, more reliable AI-driven clinical assistants and to facilitate their successful implementation in healthcare settings.