Dialogue summarization aims to extract key information from dialogue texts into summaries. Current studies are limited due to low ability of recognizing vital text features and implicit commonsense. This work proposes a Knowledge Inference model based on Topic-Coreference encoder and BART (KITC-BART). The model extracts topic and coreference features from the dialogue and replaces attention heads in the encoder’s higher layers with these two feature matrices. KITC-BART introduces a commonsense inference model to inject commonsense into the utterance, enabling the model to generate more accurate summaries by combining implicit knowledge. Additionally, custom difficulty scores are introduced in the training process to implement progressive curriculum learning. Effectiveness of the model is validated on two large-scale dialogue datasets. Experiments show that KITC-BART significantly improves the quality of generated summary compared with existing models in terms of summarizing topics and understanding commonsense.