With the growing menace of Advanced Persistent Threats (APTs) in recent years, provenance graphs have become the focus of studies on various techniques to analyze APTs using Graph Neural Networks (GNNs) for automated host intrusion detection. However, a barrier to the practical adoption of GNN-based intrusion detection systems (IDS) is the lack of interpretation. Although some studies on the interpretation of GNNs have been proposed, they have not been applied to IDS and do not focus on how to interpret anomalies. To overcome these limitations, we introduce an interpreter called GNNexPIDS to interpret provenance-based intrusion detection systems (PIDSes) by adapting and extending GNNExplainer, a generic GNN model interpretation tool. Since GNNExplainer only implements the interpretation on nodes, we extend GNNExplainer to interpret both nodes and edges. By this extension, our GNNexPIDS can interpret PIDS that use link prediction and node classification tasks. Additionally, we introduce a graph reduction method based on anomaly status to optimize interpretation results, aiding in finding interactive behaviors leading to abnormal alerts. Experimental results on two datasets CADETS E3 and THEIA E3 demonstrate fidelity and stability of our proposed method, while comparing to the baselines, e.g., LIME and GraphLime. Our method outperforms baselines on two datasets for 5% to 20% fidelity and 10% to 30% stability.