Developing an Esports Assistant System for First Person Shooting Game Based on Multimodal Learning
摘要
We plan to develop a specialized training system to enhance the competitive skills of players in the first-person shooter game “Valorant”, aiming to improve their abilities and tactical understanding within the game. We will adopt a multimodal image-text fusion technique to enhance the model’s learning of sample similarity. Specifically, we will use the cross-entropy function to maximize the diagonal values of the similarity matrix, thereby capturing the features of similar samples and reducing the similarity between dissimilar samples. Through this process, we ensure that the model can map the same entity across different modalities into similar embedding representations, while distinguishing unrelated samples. Our goal is to find a way of embedding representation during the learning process that enables the model to better understand and represent multimodal data. We will utilize the YOLOv7 model for object detection to identify destructible skills. Based on the results of object detection, we will generate relevant textual descriptions for each object. This will enable novice players to quickly understand the function of diverse and complex skills and take appropriate actions when encountering them.