Background <p>Clinical rotations are fundamental for medical students, but Chinese medical schools currently lack locally designed and valid instruments for comprehensive competency evaluation. The purpose of this study was to establish and internally validate a tool for assessing the competency of undergraduate medical students and compare the scores before and after clinical rotations to identify existing problems and improve teaching methods.</p> Methods <p>Combining theoretical analysis and empirical grounding, we developed a five-scale competency assessment questionnaire. Senior medical students filled out the questionnaire for pilot testing. Exploratory factor analysis was employed to analyze construct validity. Cronbach’s alpha and Pearson correlation analysis were used for reliability test. Descriptive statistics were utilized to present current status assessments of competency. Independent samples t-tests were conducted to compare scores between pre- and post-clinical rotation status, genders and program types.</p> Results <p>The final instrument was composed of five dimensions: Humane accomplishment, Knowledge acquisition, Health systems science, Clinical practice skills and Clinical fundamentals. Reliability analysis reported an overall Cronbach’s alpha &gt; 0.92. By current status analysis, we identified humane accomplishment and health systems science as two competency deficiencies during clinical rotations.</p> Conclusions <p>The resulting competency assessment questionnaire is a trustworthy instrument. Humane accomplishment and health systems science competencies were identified as two competency deficiencies in undergraduate medical education. The relevant teaching reforms and competency enhancement are needed.</p>