Incremental Named Entity Recognition (INER) aims to learn new entity types over time while preserving performance on previously seen types, addressing the challenges of catastrophic forgetting and semantic shift. Existing methods mostly combine pseudo-labeling and knowledge distillation to protect old knowledge, achieving promising results. However, they typically randomly initialize the classification head for new entity types, which creates a significant misalignment between the new classifier and features extracted by the backbone model. This misalignment leads to highly unstable gradient updates during early training, slowing convergence and, more critically, increasing the risk of distorting the shared feature space-ultimately harming performance. To address this issue, we propose New Classifier Pre-IniTialization (NCPT), a strategy that generates well-informed initial parameters for new classifiers before formal incremental training begins. Instead of random initialization, NCPT learns a structured transformation from the weights of old classifiers to initialize new classifiers. This ensures that the new classifier starts from a point that is semantically coherent with the existing feature space, enabling smoother optimization and better integration of new knowledge. Furthermore, we introduce a cross-step class similarity mechanism that leverages semantic relatedness computed from old prototypes and new entity types to guide the transformation. These allow the model to balance stability and plasticity. Experiments on three datasets demonstrate that NCPT significantly enhances performance and training stability across incremental steps when integrated into state-of-the-art methods, offering a principled solution to classifier initialization.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

NCPT: Feature-Aligned New Classifier Initialization for Incremental Named Entity Recognition

  • Zesheng Liu,
  • Qiannan Zhu,
  • Cuiping Li,
  • Hong Chen

摘要

Incremental Named Entity Recognition (INER) aims to learn new entity types over time while preserving performance on previously seen types, addressing the challenges of catastrophic forgetting and semantic shift. Existing methods mostly combine pseudo-labeling and knowledge distillation to protect old knowledge, achieving promising results. However, they typically randomly initialize the classification head for new entity types, which creates a significant misalignment between the new classifier and features extracted by the backbone model. This misalignment leads to highly unstable gradient updates during early training, slowing convergence and, more critically, increasing the risk of distorting the shared feature space-ultimately harming performance. To address this issue, we propose New Classifier Pre-IniTialization (NCPT), a strategy that generates well-informed initial parameters for new classifiers before formal incremental training begins. Instead of random initialization, NCPT learns a structured transformation from the weights of old classifiers to initialize new classifiers. This ensures that the new classifier starts from a point that is semantically coherent with the existing feature space, enabling smoother optimization and better integration of new knowledge. Furthermore, we introduce a cross-step class similarity mechanism that leverages semantic relatedness computed from old prototypes and new entity types to guide the transformation. These allow the model to balance stability and plasticity. Experiments on three datasets demonstrate that NCPT significantly enhances performance and training stability across incremental steps when integrated into state-of-the-art methods, offering a principled solution to classifier initialization.