Assessments play a critical role in education, but developing assessments can be difficult. This dissertation research focuses on data visualization literacy (a person’s ability to interpret visualizations) and K–12 educational assessments. Visualization literacy can influence people’s critical decisions. To understand and help people improve this ability, we must first be able to measure it with assessments. There are many challenges associated with visualization literacy assessments: e.g., (RQ1) how can we measure this ability in a repeatable and timely manner and (RQ2) how can we create diverse visualization items (question-answer pairs about visualizations) at scale? In K–12 education, teachers create assessments to assess student progress and instructional strategies. Despite recent studies on using fully automated ai approaches to generate K–12 level questions, few have investigated (RQ3) how to develop human-centered ai systems for holistic assessment development. This dissertation research aims to answer these questions by developing adaptive, scalable, and human-centered technology for assessment development.