AI-Powered Brain Tumor Segmentation and Treatment Recommendation System Using Mask-RCNN and Gemini Model Integration
摘要
Brain tumor is one of the most essential and dangerous diseases whose diagnosis and the corresponding planning of how to effectively solve the disease is of paramount importance so that a patient could be treated appropriately. The present study suggests an end-to-end artificial intelligence (AI)-based solution to segment the brain tumor, as well as provide individual therapy suggestions. The segmentation module employs the mask region based convolutional neural network (Mask-RCNN) to provide instance level localization of the sub-regions of the brain tumor enhancing MRI slices that have necrotic core and peritumoral edema. In this direction, a Large Multimodal Model (Gemini) is also included in the pipeline that can understand various types of input, including clinical notes, pathology notes, and structured metadata to make contextual propositions that are allegedly meant to incorporate therapeutic options. Also, GNNs are used to impose relational constraints on the tumor features, past treatment performance and spatial communication to boost the model in comparison to contextual interpretability and generalizability on the whole. System evaluation is performed with the use of benchmark sets and performs better than the methodologies mentioned above as measured by the Dice coefficient, precision, recall and agreement with clinical recommendations. Ablation study is performed to propose the value added by each aspect and qualitative assessment implies interpretability and clinical relevance of the system. The modular architecture will make sure that it can be extended to other types of cancer and other hospital information systems. The structured process will be able to help bridge the gap between evidence-based planning of treatment and diagnostic imaging to encourage the use of AI in precision neuro cancer.