Confocal Corneal Microscopy (CCM) image is an important metric in ophthalmological corneal-related diagnosis. CCM is able to provide high-resolution microscopy that enables accurate monitoring and diagnosis of various corneal diseases, including keratitis, corneal ulcers, and dry eye syndrome. Current corneal fiber nerve image segmentation methods are not suitable for small-size Langerhans’ Cells Segmentation. In this paper, we introduce a dual-stream network for the Langerhans’ Cells detection and segmentation in CCM images, aiming to assist in the diagnosis of dry eye syndrome. The proposed method comprises both segmentation and detection branches, and the fusion of their mutual information is achieved through the Targetized Convolution Module (TCM) and Semantic Mask Guide Refinement Module (SMGR). Compared to the existing methods, our proposed framework can improve the performance for both segmentation and detection tasks simultaneously.