Unlocking the Black Box of Artificial Media with Quantified and Explainable Co-Creative AI Systems
摘要
Artificial media is co-created by humans and generative artificial intelligence (AI) agents in a process of human-AI collaboration. Artificial media is like a black box of creative efforts of both the user and AI agent because their contributions are not easily discerned in the final product. We aim to unlock this black box by adding explainability to the human-AI co-creative process. By quantifying the co-creative process along a variety of dimensions, there is a record of the interaction dynamics that gave rise to a piece of artificial media. Interaction dynamics include the timing and proximity of contributions, the content of contributions, turn-taking interactions, communication strategies, feedback, and coordination. Quantified co-creative systems track data provenance in artificial media, creating a more complete representation of the combined effort of the user and AI system. This article introduces the Co-Creative Sensemaking Cognitive Framework (CCSM) as a structured approach for quantifying and explaining co-created products using interaction dynamics. CCSM provides a cognitive-based interaction coding scheme that tracks data provenance in artificial media through four categories of data collection: cognitive dynamics, interaction dynamics, collaboration dynamics, and domain dynamics. By considering the entire interaction history, CCSM enables a more complete and transparent representation of the combined effort of the user and the AI system in co-creating artificial media. This paper also proposes the concept of quantified co-creative AI system analysis, which utilizes CCSM to quantify, model, and visualize the interaction dynamics occurring on the platform, ultimately providing insights into the creative process. Finally, to illustrate how CCSM could be utilized to improve the transparency of co-created artificial media, the CCSM framework is applied to the creative domain of drawing, exemplifying how quantified co-creative systems would work in practice and contribute to the analysis and creation of artificial media.