Sign language recognition (SLR) and translation (SLT) have seen notable advances in recent years, particularly with the arrival of new proposals related to the implementation of large language models (LLM) and large multimodal models (MLLM). This systematic review examines works published between 2020 and 2025, retrieved from various databases, including Google Scholar, ArXiv, CVPR, and Scopus, to understand the current state of the art in different methodologies and potential areas of opportunity in automatic sign language recognition and translation. Our analysis reveals a growing interest in Gloss-Free Dynamic/Continuous Sign Processing Approaches, highlighting those that utilize the advantages of MLLMs by extracting more information from the different channels that comprise a sign, such as hand and arm positions and facial expressions. Furthermore, the review highlights several persistent obstacles, such as the global imbalance in research representation, the limited availability of diverse and annotated datasets, and the complexity of modeling the multimodal nature of sign languages. We also highlight inconsistencies in evaluation methods and significant barriers to implementing these approaches in real-time systems due to the high computational resource demands of current models. This study provides an updated perspective on the state of the art in SLR and SLT, offering a clearer picture of defining future research based on identified areas of opportunity and developing inclusive, efficient, and context-sensitive systems capable of bridging communication gaps for the global deaf community.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Sign Language Recognition and Translation with Large Multimodal Models. A Systematic Review

  • Jaime-Rodrigo González-Rodríguez,
  • Diana-Margarita Córdova-Esparza,
  • Juan Terven,
  • Julio-Alejandro Romero-González

摘要

Sign language recognition (SLR) and translation (SLT) have seen notable advances in recent years, particularly with the arrival of new proposals related to the implementation of large language models (LLM) and large multimodal models (MLLM). This systematic review examines works published between 2020 and 2025, retrieved from various databases, including Google Scholar, ArXiv, CVPR, and Scopus, to understand the current state of the art in different methodologies and potential areas of opportunity in automatic sign language recognition and translation. Our analysis reveals a growing interest in Gloss-Free Dynamic/Continuous Sign Processing Approaches, highlighting those that utilize the advantages of MLLMs by extracting more information from the different channels that comprise a sign, such as hand and arm positions and facial expressions. Furthermore, the review highlights several persistent obstacles, such as the global imbalance in research representation, the limited availability of diverse and annotated datasets, and the complexity of modeling the multimodal nature of sign languages. We also highlight inconsistencies in evaluation methods and significant barriers to implementing these approaches in real-time systems due to the high computational resource demands of current models. This study provides an updated perspective on the state of the art in SLR and SLT, offering a clearer picture of defining future research based on identified areas of opportunity and developing inclusive, efficient, and context-sensitive systems capable of bridging communication gaps for the global deaf community.