We present Financial-ICS, a novel method that utilizes 10K reports as textual corpora to generate economic representations of firms, facilitating the identification of economically related peer firms based on cosine similarity. By integrating shifted block attention and global attention, we have reduced the computational and memory complexity of self-attention to O(n), resulting in the LongBERT model. LongBERT is capable of processing texts with a maximum length of 131K tokens, enabling it to handle entire documents from 10K reports in one go. Our approach encompasses a comprehensive unsupervised training strategy, involving continual pre-training of LongBERT and resolving the anisotropy issue of the language model representation through contrastive learning. Additionally, We introduce prototype-based contrastive learning loss to increase the number of negative samples in each batch. Finally, we propose three evaluation metrics for a thorough assessment of Financial-ICS. The experimental results indicate that Financial-ICS outperforms a range of ICSs based on state-of-the-art algorithms across three metrics, as well as SIC, one of the most commonly used expert-designed ICSs. You can find our code at https://github.com/Jintao-Huang/financial-ics .