Multimodal fusion for fish behavior analysis in aquaculture: a survey
摘要
In aquaculture, real-time monitoring and analysis of fish behavior can provide valuable input information for developing scientific management strategies, such as feeding and early warning of fish diseases. Single-modality approaches, such as machine vision or acoustics, often struggle to capture the full features in open aquaculture environments and under uncontrollable fish movements, thus failing to meet the requirements for accuracy and robustness. Multimodal fusion techniques can effectively extract key complementary information across modalities, offering an effective solution to the above problems. This paper reviews the application of multimodal fusion techniques in fish behavior recognition, covering areas such as feeding behavior, abnormal behavior detection, fish disease recognition, and complex fish behavior recognition. Meanwhile, it investigates the technical details of applying multimodal fusion to fish behavior recognition, including input, output, models, evaluation metrics, parameters, computational complexity, data, and hardware. The findings indicate that the main advantages of multimodal fusion to fish behavior recognition are improved accuracy and adaptability. However, challenges remain, such as heavy reliance on annotated data and high computational complexity. In summary, this review aims to help researchers and aquaculture practitioners better understand the current state of multimodal in fish behavior recognition, thereby providing technical support for enhancing the mechanization, automation, and intelligence levels of aquaculture.