The accurate identification of circulating abnormal cells (CACs) in four-color fluorescence images is highly dependent on the fluorescence expression under each channel. Previous studies have utilized instance segmentation and target detection algorithms to identify cells and signal points in four-color fluorescence in situ hybridization (FISH) microscopy images. However, these algorithms require high accuracy in cell edge segmentation and signal point detection, which hinders the success of CAC detection. In this study, we propose a novel method for discriminating CAC cells using four-color fluorescence channels and the fusion of information from different models based on previous techniques. In particular, we utilize the information distribution of the four-color fluorescence channels to train the MMFM-DL network and MMFM-ML model, respectively. Thereafter, the model fusion strategy is employed to enhance the performance of the deep learning and shallow machine learning methods, thereby achieving a more comprehensive and accurate identification of CACs. This method requires the interpretation of only 0.92% of the cells in the microscopy images of four-color fluorescence in situ hybridization, thereby ensuring that the CAC recall rate is guaranteed to be over 98%. This is a significant improvement over the previous method.