Continual Relation Extraction aims to address catastrophic forgetting of knowledge about established relations in the task. The existing Continual Relation Extraction methods are set up on a sequence task, and the model is trained on each task to update the parameters, and the forgetting is alleviated by storing typical samples and replaying them. However, typical sample selection and storage methods employ clustering algorithm to store identical quantities of samples per relation class, failing to dynamically allocate more storage capacity to hard-to-learn relation classes. This simple strategy also causes unbounded growth of the memory space with sequential tasks. At the same time, the old relation knowledge is implicitly stored in the parameters of the model, which will undoubtedly cause forgetting when the model parameters are constantly updated with the task. To address these issues, we propose a novel Continual Relation Extraction model based on Parameter Regularization and Dynamic Memory (PRDM). Specifically, the dynamic memory method is used to allocate memory space more reasonably, and the test results of the previous task are used as the memory allocation basis for the relations of this task. We also introduce a parameter regularization mechanism to limit the variation of parameters by estimating the importance of the parameters. Experimental results on two public datasets demonstrate the superiority of the model in longer task sequences and validate the effectiveness of our method in alleviating catastrophic forgetting. The code and datasets are available from https://github.com/xueruisxf/PRDM.