Weak-to-strong boundary learning: leveraging pre-trained models to enhance large language models’ entity boundary recognition
摘要
Entity boundary recognition is a critical component for large language models in natural language processing tasks, playing a vital role in accurately understanding textual semantic structures and extracting key information. This paper addresses the challenges of inaccurate boundary localization in large language models and their inability to fully leverage the specialized advantages of pre-trained models in boundary recognition, proposing an optimization method for entity recognition in large language models based on weak-to-strong boundary learning. We construct a weak-to-strong learning framework based on three stages: “pre-trained model boundary recognition training, boundary alignment signal extraction, and large model boundary learning optimization,” which directly transfers the boundary recognition capabilities and improvement information from pre-trained models to large language models; We design a multi-dimensional fusion boundary improvement metric function that comprehensively quantifies the enhancement of boundary recognition capabilities in pre-trained models from three dimensions: boundary gain, confidence change, and dynamic weight adjustment, ensuring effective transmission of alignment signals through normalization and type-sensitive weight allocation mechanisms; We propose a loss function optimization strategy with embedded boundary alignment signals, combining boundary improvement metrics with the generation probability distribution of large models to achieve precise boundary learning and capability transfer based on alignment signals. Experimental results demonstrate significant differences in the effectiveness of various alignment signal design strategies, with the complete diff_score function achieving an average improvement of 1.8% over single-component approaches, and the dynamic weight mechanism showing the most significant improvement in boundary recognition for organization entities; The adjustment of hyperparameter