Named Entity Recognition (NER) is a fundamental task in biomedical natural language processing (BioNLP). Although transformer-based pre-trained language models, such as BioBERT, have significantly improved BioNER performance, they still face challenges in modeling long-range contextual dependencies and require substantial computational resources. Additionally, BioBERT employs Gaussian Error Linear Units (GELUs) as its default activation function, which introduces exponential computations, increasing computational overhead and reducing training and fine-tuning efficiency. To address these challenges, this paper proposes an enhanced BioBERT-BiLSTM-CRF model. This model integrates a Bidirectional Long Short-Term Memory (BiLSTM) network and a Conditional Random Field (CRF) on top of BioBERT to enhance the capture of long-range dependencies and improve entity annotation accuracy. Furthermore, to enhance computational efficiency, we introduce Inverse Square Root Linear Units (ISRLUs) as a replacement for GELUs in BioBERT, reducing computational costs and accelerating the training process. Experiments were conducted on four biomedical NER datasets, demonstrating that the proposed BioBERT-BiLSTM-CRF model improves entity recognition performance across multiple datasets. Moreover, after substituting GELUs with ISRLUs, the training time was reduced by an average of 4.61%, with F1-score improvements observed on some datasets.