Relational Triple Extraction (RTE) mines factual knowledge components as relational triples from unstructured text. However, most triples tested in current datasets are already duplicated in the training set, leading past studies to rely more on memorization than on genuine discovery. In response to this, we suggest a novel benchmark ENT to assess the model’s capability to Extract New Triples, which aligns more closely with the practical objective of RTE such as automatic knowledge graph construction (The dataset is available at https://github.com/Kast-Nora/ENT-Dataset ). We developed the dataset by instructing the large language model to perform text expansion based on preprocessed knowledge graph segments, followed by rule-based and semantic check. The ENT dataset, boasting over 300,000 unique relational triples, encompasses a broad spectrum of knowledge. The proportion of new triples in the test set exceed 60%, and all the samples contain at least one unseen triples, highlighting a strong emphasis on discovering new knowledge. ENT is perceived by human annotators with a low level of hallucination, serving as a valid and valuable dataset. We re-evaluated 9 state-of-the-art RTE methods and found a generalized accuracy decrease on ENT, demonstrating that ENT is a more challenging and meaningful benchmark.