Studying patients’ physician selection behavior through multimodal data is a key focus in online healthcare community. However, there is limited research on multimodal data fusion in this field. This paper aims to analyze how different types of data influence patients’ online physician selection from the perspective of multimodal data fusion, providing insights into patients’ decision-making in the era of big data. Based on multimodal data from the Haodf platform, this study includes a sample of 13,166 records. Two predictive models were developed: a feature-engineered neural network model and an end-to-end deep learning model. The former combines selected features extracted from images and text with structured data to predict physician selection behavior. The latter uses raw data, employing ResNet50 and BERT for image and text feature extraction, then integrates structured data for prediction. The analysis confirmed that both models can effectively predict patients’ physician selection behavior and offer a certain degree of interpretability. The feature-engineered model achieved a test set MSE of 0.4124, MAE of 0.4579, and an R2 of 0.5304, while the end-to-end model achieved a test set MSE of 0.3991, MAE of 0.4327, and an R2 of 0.6321. Overall, the end-to-end model outperformed the feature-engineered model. Both the feature-engineered and the end-to-end models effectively explore patients’ physician selection behavior in online healthcare community. The results demonstrate that the multimodal information—including doctor-generated content, patient-generated content, and system-generated content—significantly influences patient decision-making. Additionally, different features from images and text influence patients’ choices in distinct ways.