<p>Stomatal trait analysis is essential for optimizing crop photosynthesis and transpiration, yet deep learning studies have focused mainly on monocotyledons, leaving dicotyledonous crops such as melon (<i>Cucumis melo</i> L.) understudied. To bridge this gap, we established a dedicated melon stomatal dataset comprising 5,708 training images, 1,631 validation images, and 815 test images. On this basis, we developed an improved Mask R-CNN framework using Vision Transformer (ViT) as the backbone. Specifically, standard Multi-Head Attention (MHA) was replaced with Dynamically Composable Multi-Head Attention (DCMHA), which enhances information exchange across attention heads and alleviates the low-rank limitation of conventional attention. In addition, a modified effective Squeeze-and-Excitation (eSE) module was incorporated into the Feature Pyramid Network (FPN) to strengthen channel dependency modeling and multi-scale feature representation. On the melon dataset, the proposed model achieved a mean average precision (mAP) of 72.40 ± 0.09%, with AP50 and AP75 of 91.93 ± 0.14% and 84.59 ± 0.19%, respectively. Repeated-run statistical analyses showed that eSE significantly and consistently improved the main detection metrics across backbones, whereas DCMHA provided a more moderate gain within the ViT-based setting, with clearer support for AP50 than for mAP or AP75 under the baseline FPN setting. Overall, the combined configuration remained among the top-performing models for stomatal instance segmentation. Ellipse fitting further enabled automated quantification of stomatal length, width, count, area, and circumference, showing strong agreement with manual measurements (Pearson <i>r</i> = 0.978). The model also showed preliminary transferability to cucumber, watermelon, pumpkin, and loofah, with an average species-specific R² of 0.86, although each species was evaluated on a limited sample set.</p>