Surface Electromyography (sEMG) is a non-invasive biosignal widely used for hand gesture recognition (sEMG-HGR). While prior work has focused on architectural modifications to capture the temporal dynamics of sEMG signals, we propose a novel direction by leveraging Large Language Models (LLMs), known for their strength in temporal sequence understanding. Inspired by the temporal nature of sEMG-HGR, we introduce SemgLLM, a framework that is capable of accepting sEMG signals as input for hand gesture analysis, and outputting recognition results in textual form. At its core, SemgLLM employs an sEMG-to-Linguistics Mapping (SLM) module, which converts sEMG signals into discrete, interpretable tokens for LLMs. We further enhance LLM performance with tailored designs for gesture understanding. Extensive experiments show that SemgLLM achieves state-of-the-art results.