ZSLViz: interactive optimization of class-attribute matrix for zero-shot learning via visual analytics
摘要
Because of its broad application potential in scenarios with limited annotated data, zero-shot learning (ZSL) has attracted considerable attention and has become an important research direction in the field of computer vision. By leveraging semantic descriptions (e.g., class-attribute matrix), ZSL aims to learn discriminative and transferable visual features from seen classes to facilitate the recognition of unseen classes. However, the construction of a high-quality class-attribute matrix remains challenging. Manual attribute definitions frequently entail considerable subjectivity and incur substantial time and resource costs, compounded by insufficient quantitative evaluation mechanisms within human–AI collaborative processes. To address these challenges, we propose ZSLViz, a visual analytics system synthesizing expert knowledge with artificial intelligence to optimize the class-attribute matrix through interactive optimization techniques. The system facilitates dynamic analysis of the classification process and results based on the existing class-attribute matrix. Furthermore, it incorporates an attribute relation inference method grounded in context-aware embeddings, employing large language models to elucidate semantic associations among higher order attributes. Moreover, we present a localized optimization algorithm designed to guide interactive attribute weight adjustments, enhancing the representational capacity of the class-attribute matrix. Case studies and expert evaluations demonstrate that ZSLViz mitigates attribute weight subjectivity through human–AI interaction, reduces reconstruction cost, and enhances the overall quality of the class-attribute matrix, ultimately leading to improved classification accuracy.
Graphical abstract