Sign Language Recognition and Translation with Large Multimodal Models. A Systematic Review
摘要
Sign language recognition (SLR) and translation (SLT) have seen notable advances in recent years, particularly with the arrival of new proposals related to the implementation of large language models (LLM) and large multimodal models (MLLM). This systematic review examines works published between 2020 and 2025, retrieved from various databases, including Google Scholar, ArXiv, CVPR, and Scopus, to understand the current state of the art in different methodologies and potential areas of opportunity in automatic sign language recognition and translation. Our analysis reveals a growing interest in Gloss-Free Dynamic/Continuous Sign Processing Approaches, highlighting those that utilize the advantages of MLLMs by extracting more information from the different channels that comprise a sign, such as hand and arm positions and facial expressions. Furthermore, the review highlights several persistent obstacles, such as the global imbalance in research representation, the limited availability of diverse and annotated datasets, and the complexity of modeling the multimodal nature of sign languages. We also highlight inconsistencies in evaluation methods and significant barriers to implementing these approaches in real-time systems due to the high computational resource demands of current models. This study provides an updated perspective on the state of the art in SLR and SLT, offering a clearer picture of defining future research based on identified areas of opportunity and developing inclusive, efficient, and context-sensitive systems capable of bridging communication gaps for the global deaf community.