Current deep clustering methods face challenges in dynamic functional community detection for smart cities due to inadequate neighborhood association modeling, non-adaptive cluster number decisions, and limited interpretability. We propose Graph-Attentive Dual Deep Q-Network (GraphADQ), a reinforcement learning framework integrating graph attention for adaptive community detection. Key innovations include: 1) Multimodal Heterogeneous Graph Constructor (MHGC) fusing spatial-social-economic features; 2) Dynamic Neighborhood-Attribute Co-Encoder (DNACE) enhancing feature decoupling through attention mechanisms and knowledge distillation; 3) Hierarchical Policy Optimizer (HPO) with multi-indicator reward functions enabling dynamic cluster number determination via double-delay deep Q-network. Dual-stage SHAP Interpretation Mapping (DSIM) further provides end-to-end interpretability from raw features to community labels. Experiments on multi-city datasets (Beijing, Chengdu, Changzhi) demonstrate that DNACE improves clustering effect and cross-city robustness (SC \({>}\) 0.9); HPO enables adaptive clustering optimization through a dual Q architecture; and DSIM identifies recreational facility density and housing prices as key zoning drivers. GraphADQ provides a highly accurate and interpretable decision-making framework for smart city governance.