Single-cell sequencing has transformed the study of cellular heterogeneity and its role in health and disease. Current models predominantly adopt expression-driven learning frameworks that encode cellular states through the joint modeling of genes and their associated expressions. Current methods treat gene expression in various ways, such as using a fixed gene order, ranking genes based on expression levels, or combining gene IDs with expression values through simple addition. However, these approaches overlook the biological context of genes and rely on static representations that fail to capture gene activity’s functional dynamics. Here, we present scFiLM, a dynamic fusion framework built upon the Feature-wise Linear Modulation (FiLM) paradigm, designed to integrate prior biological knowledge encoded in pretrained semantic gene embeddings derived from NCBI gene descriptions with expression-driven feature modulation. Specifically, for each gene–expression pair, scFiLM adaptively modulates the corresponding semantic embedding through FiLM layers, utilizing expression-derived scaling and shifting parameters. The resulting modulated embeddings are subsequently fed into a bidirectional Mamba encoder to capture long-range dependencies, thereby generating informative cell-level representations for downstream analyses. Comprehensive cell-level evaluations demonstrate that scFiLM not only effectively mitigates batch effects, but also achieves state-of-the-art performance in B cell type classification. At gene level, scFiLM retains the intrinsic functionality of genes and enables effective categorization of gene functions, thereby enhancing the interpretability of learned representations. The results highlight the potential of scFiLM as a robust and generalizable framework for single-cell transcriptomic analysis.