The use of generative AI tools in medical education, such as Copilot and Gemini, has shown great promise for improving student evaluation and learning experiences. This study assesses the effectiveness of these tools in producing diagnostic microbiological evaluations, with a focus on the quality and discrimination level of their questions. The results show significant variations between the two instruments, especially in terms of question scores and concentration regions. Copilot-generated evaluations produced better total scores and featured a variety of theoretical and practical questions, whereas Gemini’s assessments were more visually focused and had a broader range of score variances between questions. The analysis of discrimination levels in both evaluations revealed that good, fair, and poor discrimination questions appeared in both instruments. Questions with significant discrimination related to practical skills and critical thinking, whereas basic knowledge questions frequently had weaker discrimination. Despite the promising results, limitations were noted, including the necessity for thorough editing to assure the accuracy of AI-generated questions and the clarity of phrasing to avoid student confusion. This study demonstrates the potential for generative AI to advance medical education by increasing students’ practical and rational skills. However, it emphasizes the significance of human monitoring in validating and refining AI-generated material. With appropriate integration, generative AI techniques can provide a viable path for novel and successful instructional practices in diagnostic microbiology and beyond.