In a world where changes occur increasingly rapidly, organizational maturity is indispensable for companies to weather critical situations. Within the framework of organizational maturity principles, project management emerges as an evolving subject. Consequently, there arises a necessity to assess and reevaluate Project Management Maturity models and organizational management maturity models to cater to the genuine needs of Small and Medium-sized Enterprises (SMEs) across diverse sectors. In pursuit of identifying potential enhancements and contributing to advancing the discussed topic, this study aimed to evaluate a diagnostic tool (questionnaire) employed by a startup. The study encompasses a comparative analysis of the criteria utilized in the diagnostic tool and the validation of said tool through the application of Item Response Theory (IRT). Upon scrutinizing the contents of the questionnaire items, the findings suggest that the tool effectively gauges Organizational Maturity rather than Project Management Maturity as a latent trait. Furthermore, employing IRT enabled the development of a metric proficient in expressing such a latent trait, thereby facilitating the classification of SMEs into six levels. Hence, the study underscores the prospect of enhancement in the quest for a model conducive to SMEs’ maturation in Project Management.