Background <p>Competency-based education defines competencies as educational outcomes and guiding instruction and assessment. In Japan, standardised competency frameworks for medical laboratory scientists in clinical settings are underdeveloped. This study aimed to develop and validate a competency scale for novice medical laboratory scientists with less than 3 years of clinical experience into an assessment tool.</p> Methods <p>This study involved scale development and validation conducted in four phases: a preliminary study to test content validity for exploratory purposes, a pilot survey to examine the feasibility of the evaluation sheet under conditions similar to those of the main survey, a main survey to verify reliability and validity and a final phase to establish the completed scale. In the main survey, each novice medical and biomedical laboratory scientist (self-evaluator) was paired with two supervising evaluators from the same institution. The supervising evaluators, comprising educational supervisors, senior medical laboratory scientists or managers, were directly involved in the guidance and assessment of the corresponding novice.</p> Results <p>Exploratory factor analysis of the self-evaluation dataset identified four factors with 51 items using a five-point Likert scale. The cumulative variance explained by the four factors was 52%, and Cronbach’s alpha ranged from 0.78 to 0.94 for each factor. The validity of the scale was confirmed through confirmatory factor analysis with the supervisory evaluation dataset.</p> Conclusions <p>The Japan Medical Laboratory Scientists Competency Scale demonstrated robust psychometric properties, establishing it as an effective tool for assessing novice medical laboratory scientists in Japan through self- and supervisory evaluation. Those findings would support enhancements in workforce development and competency-based education in this field.</p>