The objective of Continual Relation Extraction (CRE) is to learn new relations while maintaining good performance on previously learned relations. However, a major challenge in CRE is catastrophic forgetting, which causes a substantial drop in the model’s performance on previously trained tasks after learning new tasks. To mitigate the issue of catastrophic forgetting, some representative instances of previous tasks are stored and are replayed after training on new data so that the model can retain a consistent comprehension of old relations. However, memory-based approaches often struggle with overfitting on the memory samples. Previous studies have identified the presence of classifier bias before replaying previous data, where the model will tend to classify samples belonging to previous relations into new relations. To tackle classifier bias, we propose classifier parameters integration. The classifier parameters of all previous tasks are integrated to mitigate classifier bias. Meanwhile, we utilize the cross-head mechanism, where intermediate hidden representations from the encoder of the new model are fed into the classifier of the old model to obtain cross-head logits. Then, leveraging these logits, we propose Cross-Head Alignment for mitigating overfitting in memory replay. Experimental results on two benchmark datasets show the effectiveness of our model compared to other baselines.