Current face clustering methods rely on global graph structures for feature propagation, which leads to high computational costs and limited constraints about global distribution. We propose MoCo-ANA, a framework that eliminates global graph dependency and decouples local adaptive aggregation from global representation learning. Our method employs a feature-structure co-attention (FS-CoAttn) mechanism for dynamic neighbor selection and introduces a multi-constraint objective including a MoCo-like supervised contrastive loss and hypersphere uniformity regularization. Extensive experiments demonstrate that MoCo-ANA achieves SOTA performance on MS-Celeb-1M and DeepFashion, with ablation studies validating the efficacy of the proposed core components.