There is a non-trivial gap in skills students possess upon graduation and what employers expect from graduates when they join the workforce. To address this problem, various efforts are underway to shift from the traditional knowledge-based model of educational programs to one that includes a competency-based approach. To foster the introduction and implementation of competency-based concepts to academic programs in a timely fashion, it is necessary to provide the faculty with as many resources as possible. In this context, this paper describes the development, testing, and use of an evaluation process for tools meant to build or evidence competency in Cybersecurity and reports on the lessons learned.