Pre-trained models have made remarkable strides in fields such as natural language processing and multi-modal learning. However, one limitation of these models is their inability to effectively process extremely long texts, as they often struggle to remember and handle all the essential information. Long document summarization is a challenging task that utilizes a pretrained transformer-based architecture to summarize the contents of documents. However, due to the quadratic cost of self-attention, scaling transformer-based models to long sequences is prohibitively expensive for implementation. Additionally, information locality pattern commonly exists in text summarization, requiring the inductive bias of local concentration. Therefore, we propose a new Transformer variant called LongSum, which addresses challenges by designing a novel Local-Global Attention to efficiently capture and fuse representative information from both local and global views. Extensive experiments are carried out on multiple datasets from different domains. LongSum outperforms baseline models on summarization performance quantitatively and qualitatively, which demonstrates its effectiveness and efficiency.