Mapping Artificial Neural Networks’ Processing Data in Audiovisual Artworks
摘要
In a non-generative approach to artificial intelligence in an artistic practice, this work looks at mapping processing data from different artificial neural networks (NNs) onto sound and visuals. One aim of this practice-based piece of research is to offer insights into how these ubiquitous, yet notoriously unintelligible algorithms operate, sometimes by exposing the audience to that very unintelligibility. The other is to use these vast amounts of abstract data as a blank canvas for audiovisual artworks. At the heart of the whole work is a link and cross fertilization between the use of sounds and visuals aesthetically associated with errors and digital malfunction, and the use of actual ‘waste’ data (from NN training), which acts as a trace of their operation. We look at different kinds of NNs, and at various in-training data-streams: from the easily apprehensible output of generative networks, through various performance metrics, to the highly-abstracted activation and weight values in and between hidden layers. We believe these pieces uncover some interesting features about the evolution of GANs’ outputs in training and the consistent patterns they follow in the latent space, illustrate the potential for errors (mode collapse, noise, etc.) in those networks, and outline the inherently abstract nature of weight changes during an MLP’s training.