Vast amounts of materials science literature accumulate rich knowledge resources, yet extracting valuable knowledge from this corpus poses significant challenges. Although large language models (LLMs) have demonstrated the capability in general domains, they have not performed well in the highly specialized field of materials science, primarily due to their insufficient knowledge base in this area. In this study, we propose an innovative multi-model automated approach called MatSciES (Materials Science Extraction and Summarization). Specifically, we first extract the texts with keywords filtering and the text context, followed by multi-models to assess the relevance of the texts to materials science knowledge and to summarize the relevant texts efficiently. Due to the absence of datasets in the field, we meticulously labeled multiple datasets combining public datasets for the assessment of relevance, text summarization and question answer. The experimental results demonstrate our approach achieves a high accuracy rate of 99.3% in relevance assessment tasks; in summarization task evaluations, the accuracy rate reaches 95.7% and the average text compression is 56%. Moreover, to validate the capability of integrating LLMs with the knowledge base, we conducted tests on manually annotated and public datasets, resulting in an accuracy improvement of up to 32.4%. Our method can assist materials science researchers with limited computer skills in rapidly constructing a substantial knowledge base. This provides a novel solution for developing LLMs knowledge base in the materials science field.