<p>Texture provides valuable insights into building materials, structure, style, and historical context. However, traditional deep learning features struggle to address architectural textures due to complex inter-class similarities and intra-class variations. To overcome these challenges, this paper proposes a Dual-stream Multi-layer Cross Encoding Network (DMCE-Net). DMCE-Net treats deep feature maps from different layers as experts, each focusing on specific texture attributes. It includes two complementary encoding streams: the intra-layer encoding stream efficiently captures diverse texture perspectives from individual layers through multi-attribute joint encoding, while the inter-layer encoding stream facilitates mutual interaction and knowledge integration across layers using a cross-layer binary encoding mechanism. By leveraging collaborative interactions between both streams, DMCE-Net effectively models and represents complex texture attributes of architectural heritage elements. Extensive experimental evaluations on architectural heritage datasets and three texture databases demonstrate that DMCE-Net achieves superior performance compared to existing deep learning methods and handcrafted features, providing reliable texture representations.</p>