In natural language processing (NLP), text classification is a key task for large language models (LLMs). However, current approaches typically depend only on the final layer’s output, neglecting the valuable information embedded in the intermediate layer neurons. To address this limitation, we propose LENS (Linear Exploration and Neuron Selection), a novel method that selects and sparsely integrates prominent neurons from intermediate layers through linear exploration. These selected neurons are then passed to subsequent modules for text classification. This approach effectively reduces noise from irrelevant neurons, enhancing both the accuracy and efficiency of the model. Furthermore, detecting telecommunications fraud text poses a significant challenge in NLP due to its increasingly concealed forms and the limitations of existing detection methods. To address issues of data scarcity and classification accuracy, we developed the LENS-RMHR (Linear Exploration and Neuron Selection with RoBERTa, Multi-head Mechanism, and Residual Connections) model, which builds upon LENS. By integrating these modules, LENS-RMHR strengthens feature representation and training efficiency. Leveraging the CCL2023 telecommunications fraud dataset, we constructed an expanded dataset comprising eight categories covering diverse fraud types. Additionally, we employed a dual-loss function to enhance the model’s multi-class classification performance. Experimental results demonstrate that LENS-RMHR achieves superior performance across multiple benchmark datasets, showcasing its broad potential for applications in text classification and telecommunications fraud detection.