Enhancing Live Performances with AI-Driven Visuals: A Machine Learning and Generative AI Approach
摘要
Machine Learning (ML) techniques and Generative AI gained an increased importance in the Entertainment Industry, enhancing the audience interest by adding new valences to various entertainment events and performances. The projection of the performance of large LED screens became a common practice for performances made in wide open air or indoor spaces. This article proposes a solution for adding new value to a performance by using, alongside the usual performance projection, an additional different one. This projection considers the actors involved in the performance from different perspectives based on a specific algorithm, including the use of ML techniques and Generative AI. Our approach makes use of existing IMAG (Image Magnification) systems and robotic cameras in a master-slave configuration for real-time tracking of the performance. The images gathered by the robotic cameras are analyzed and refined through separate computer vision processing pipelines. The results of the image processing operations are used to select, composite and display the image of one or many performers against an AI-generated image background. The image generation strategy of the AI model is selected by image sentiment analysis and a threshold-based algorithm. For a better understanding and motivation of our proposal, we include a brief state of the art and comparison of most relevant object detection methods and generative models suitable for our goal.