Background <p>Sperm morphology detection technology has important research value in diagnosing male infertility. Traditional manual detection methods are time-consuming, labor-intensive, and highly subjective. The sperm morphology analysis method based on deep learning can objectively and efficiently complete the sperm detection task, and has the advantage of end-to-end learning.</p> Results <p>This study proposed a two-stage sperm morphology analysis method, which used the UNet + + network to segment individual sperm, and then used the Multi-Head Mobilevit Net model based on multi-output classification to achieve one-time detection of the head, midpiece, and principal piece of a single sperm, and integrated the Mixing Loss function into the classification model.</p> Conclusions <p>Based on the data enhancement technology, the expanded 10,802 improved Papanicolaou stain sperm morphology classification dataset was tested. The results showed that the accuracy of the sperm head, midpiece, and principal piece reached 83.47%, 96.19%, and 94.99%, respectively. This study can quickly realize the automated detection of sperm morphology, providing strong technical support for the development of reproductive medicine.</p>