It is challenging to discriminate autism spectrum disorder (ASD) from a highly heterogeneous database, because there is a great deal of uncontrollable variability in the data from different sites. Recently, prompt learning has received considerable attention in domain adaptation as a promising solution. However, its application to graph data like multi-site brain networks has not been fully studied. It faces two major challenges: (1) complex graph structure; and (2) inter-individual variability. To overcome the issues, we propose a novel prompt-tuning paradigm for multi-site brain network analysis (BrainPrompt) using functional magnetic resonance imaging (fMRI). Specifically, we introduce two tunable soft prompts: (1) a mask prompt to prune noisy edges while preserving important connections, and distill it to reduce domain-specific biases; (2) sample prompts to capture inter-individual variations. Our model outperforms other models on the ABIDE dataset, especially at sites with limited samples (e.g., the Stanford site, which has only 39 samples). BrainPrompt achieves a 35.88% improvement in accuracy compared to the state-of-the-art method, highlighting its superiority in small sites. Furthermore, our results demonstrate the interpretability and generalization of the proposed method. Our code is available at https://github.com/zliuzeng/BrainPrompt .